NEW METRICS FOR EVALUATING MONTE CARLO TOLERANCE ANALYSIS OF ASSEMBLIES

NEW METRICS FOR EVALUATING MONTE CARLO TOLERANCE ANALYSIS OF ASSEMBLIES Robert Cvetko Systems Egieerig Rocketdye Caoga Park, CA Keet W. Case Specer P. Magleby Mecaical Egieerig Departmet Brigam Youg Uiversity Provo, UT ABSTRACT Mote Carlo simulatio may be used by egieerig desigers to predict te effects of maufacturig variatios i mecaical assemblies. Validity of te predictios depeds o te accuracy of te iput variatios, simulatio sample size ad te fit of a statistical distributio to te resultat assembly data. New metrics are preseted for assessig te accuracy of variatio aalysis metods. Errors due to sample size are estimated for te mea, variace, skewess ad kurtosis of te resultat distributio, ad te predicted rejects. Simple algebraic expressios are derived, wic ca be used to predict te error i te assembly variatio parameters witout avig to repeat te simulatio.. INTRODUCTION Variatio aalysis is a importat desig tool for brigig maufacturig cosideratios ito desig decisios. Dimesioal variatios occurrig i productio cause critical assembly dimesios to vary, wic ca adversely affect performace. Mote Carlo simulatio may be used to simulate productio variatios i mecaical assemblies, resultig from te accumulatio of dimesioal variatio, commoly called tolerace stackup. I a Mote Carlo aalysis, sets of sample compoet dimesios are geerated ad iput to a assembly fuctio, wic calculates te resultat size of critical assembly dimesios or features for eac assembly. A statistical distributio may be fit to te resultat assembly data ad used to predict percet rejects due to exceedig te upper ad lower desig limits. Te output of a Mote Carlo aalysis is a set of parameters defiig a statistical distributio tat describes te variatio i a critical assembly dimesio. Te accuracy of tese parameters depeds upo te umber of sample assemblies geerated i te simulatio. Typical sample sizes rage from 000 to 5000. It is ot clear to a desiger wat sample size is required to acieve a desired accuracy. He may repeat te simulatio wit a differet set of radom umbers to see ow muc te parameter values cage, but e as o clear-cut quatitative measures to guide im i coosig te appropriate sample size for eac parameter.. Defiitio of Geeral Terms ad Symbols Te variatio i a set of radomly varyig dimesios x i may be described by a statistical distributio. Te first four momets of dataset x i, defied i Table -, are typically used to defie te sape of te distributio. Table -. First four momets of a statistical distributio. x i i ( x i ) i 3 3 ( x i ) i ( x i ) i st momet: te mea of x i (average) d momet: te variace of x i (spread) 3 rd momet: te skewess of x i (asymmetry) t momet: te kurtosis of x i (peakedess) It is useful to o-dimesioalize eac parameter by te stadard deviatio of te distributio, so te distributio parameters ca be compared to stadard table values. For a stadard Normal distributio, te four momet values are 0,, 0 ad 3, respectively.

Table -. Stadardized distributio momets. α R(x i - )/ 0 α / Stadardized st momet: te mea of x i / Stadardized d momet: te variace of x i / α 3 3/ 3 Stadardized 3 rd momet: te skewess of x i / α / Stadardized t momet: te kurtosis of x i /. Oe-way Clutc Assembly Model A Mote Carlo simulatio of a simple assembly will be used as a case study to illustrate error estimatio procedures. Te oeway clutc assembly sow i figure - illustrates a critical assembly feature dimesio wose variatio is depedet o a cai of compoet dimesios. Te clutc cosists of four differet parts: a ub, a rig, four rollers, ad four sprigs. Te sprigs pus te rollers ito te wedge-saped space betwee te rig ad te ub. If te ub is tured couter-clockwise te rollers bid, causig te rig to tur wit te ub. Turig te ub clockwise causes te rollers to slip ad prevets te trasmissio of torque to te rig. a φ e b φ c c Vector Loop Figure -: Vector Loop for te Clutc Assembly Te cotact agle φ is a depedet assembly variable wic is critical to te performace of te clutc. Te variable b, te locatio of cotact betwee te roller ad te ub, is also a depedet assembly variable. Te variatio i φ ad b are fuctios of te variatios of te compoet dimesios a, c, ad e, as well as a complex assembly fuctio. Te omial cotact agle, we all of te idepedet variables are at teir mea values, is 7.08 degrees. For proper performace, te upper ad lower specificatio limits for φ are based upo tis omial agle ad ± 0.60 degree desig limits. Table -3 below summarizes tese values. Table -3: Cotact Agle Specificatios Cotact Agle Value (degrees) Upper Limit 7.68 Nomial Agle 7.08 Lower Limit 6.8 Te objective of variatio aalysis for te clutc assembly is to determie te variatio of te cotact agle relative to te desig limits. Table - below sows te tree idepedet dimesios ad te stadard deviatio of eac. Eac of te idepedet variables is assumed to be statistically idepedet (ot correlated wit eac oter) ad a ormally distributed radom variable. Table -: Idepedet Dimesios for te Clutc Variable Mea Stadard Deviatio a - ub radius 7.65 mm 0.0666 mm c - roller radius.30 mm 0.00333 mm e - rig radius 50.800 mm 0.006 mm Te cotact agle for te clutc is described by te explicit assembly fuctio of a, c ad e, below, wic is oliear. a + c arccos e c φ Eq. - Eq. - demostrates tat eve a simple assembly ca result i a complex assembly fuctio. Most assemblies are too complicated to be reduced to a explicit equatio. Implicit equatios ca be foud for a assembly usig te vector loop metod [Case 995]..3 Vector Loop Model ad Assembly Fuctio for te Clutc Te vector loop metod uses te assembly drawig as te startig poit. Vectors are draw from part-to-part i te assembly, passig troug te poits of cotact. Te vectors represet te idepedet ad depedet dimesios wic cotribute to tolerace stackup i te assembly. Figure - above sows te resultig vector loop for a quarter sectio of te clutc assembly. Te vectors pass troug te poits of cotact betwee te tree parts i te assembly. Te vector b ad te agles φ ad φ are assembly variables. Sice te roller is taget to te rig, bot te roller radius c ad te rig radius e are coliear. Tus, φ φ. Oce te vector loop is defied, te implicit equatios for te assembly ca easily be extracted. Eq. - sows te set of implicit equatios for te clutc assembly derived from te vectors. x ad y are te sum of vector compoets for te loop ad θ is te sum of relative agles betwee cosecutive vectors. x y θ 0 b + csi( φ 0 a + c + c cos( φ 0 90 90 + 90 φ + φ ) e si( φ ) ) e cos( φ 80 + φ + 90 φ ) Eq. - From te implicit equatios, it is see tat φ ad φ are equal. Terefore, te equatios ca be simplified to just two equatios for te two ukows b ad φ as sow by Eq. -3. Te y equatio may be solved for φ, yieldig te explicit assembly Eq.

-. However, most assembly fuctios are muc more difficult to covert to explicit form. x y 0 b + c si( φ ) e si( φ ) 0 a + c + c cos( φ ) e cos( φ ) Eq. -3 Eac of te two assembly equatios for te clutc equals zero because te vector loop is closed, ad must maitai closure for te parts to assemble. However, vector loop metods may be applied equally well to ope loop assemblies [Gao, et al 998].. MONTE CARLO SIMULATION: OVERVIEW Tis sectio presets a overview of te Mote Carlo simulatio metod for aalysis of variatio i assemblies ad te curret metods for estimatig its accuracy.. Backgroud o Mote Carlo Te Mote Carlo Metod estimates te accumulatio of variatio i a assembly due to dimesioal variatios witi te assembly. Variatios are described by statistical probability distributios. Figure - grapically sows ow te process works. Te output distributio is a fuctio of te distributios of te iput variables ad te assembly fuctio. Tousads of sets of te iput variables are combied to get a reliable measure of te output distributio. Iput Variables Iterative Assembly Fuctio Output Distributio Histogram Mea Std. Dev. Skewess Figure -: Grapical Represetatio of Mote Carlo Metod A assembly fuctio is derived wic describes a assembly dimesio or feature i terms of te compoet dimesios. A Mote Carlo simulatio cosists of selectig radom values for te idepedet dimesios from teir respective probability distributios, ad calculatig te resultat assembly dimesios from te assembly fuctio (a iterative process is required if te fuctio is implicit). By repeatig tis procedure for a large umber of assemblies, a istogram of te predicted variatio i te resultat assembly feature ca be plotted. Te results ca be compared to specified desig limits for te assembly feature. Te rejects (assemblies tat fall outside te specificatio limits) ca be couted durig te simulatio, or a distributio ca be fit to te momets or percetiles of te Mote Carlo output ad te rejects estimated aalytically or from stadard tables. Te most commo distributio used to describe assembly variatio is te ormal or gaussia. Skewed, flat or peaked data may be approximated by more geeral distributios [Early 989]. Te umber of samples required depeds o te desired accuracy of te output. Very large samples may be required for accurate results. Te relatiosip betwee sample size ad accuracy will be examied i detail.. Mote Carlo Simulatio of te Clutc Te oe-way clutc assembly was easily modeled ad aalyzed wit te Mote Carlo spreadseet program Crystal Ball (sice te equatio for te cotact agle is explicit). More complex assemblies wit implicit assembly fuctios, requirig a iterative solutio for eac assembly, would be more difficult to model, ad would be impractical to aalyze wit a spreadseet. Figure - below sows sample output of a Mote Carlo simulatio of te clutc assembly. Te istogram estimates te probability distributio for te cotact agle φ. 0,000 Trials.03.00.007 Forecast: φ Frequecy Cart 99.5.003 66.5 Certaity is 99.8% from 6.8 to 7.68 Degrees.000 0 6.500 6.650 7.0000 7.3750 7.7500 Cotact Agle Figure -: Mote Carlo Output from Crystal Ball Te istogram sows a estimate of te output distributio for te cotact agle. Te tails of te istogram represet te assemblies tat fell outside te upper ad lower specificatio limits (rejects). Wit te specificatio limits as idicated, 99.8% of te sample assemblies are good, ad 0.7% of te assemblies are outside te limits. Te sample size was 0,000 for tis ru. A study was made of te effect of sample size o te results of te clutc simulatio. Te first four momets of te resultat cotact agle distributio were calculated as te sample size was successively doubled. Data was accumulated for successive simulatios for efficiecy. A custom Mote Carlo simulatio program, usig a radom umber geerator, was used to provide greater flexibility ad speed i outputtig te distributio momet iformatio. Tis simulatio of over a billio samples took 8.6 ours to complete. Figure -3 sows ow te value of te distributio momets caged durig te simulatio as te sample size icreased. Te fial momet values are used as te becmark for calculatig te error trougout te followig sectios. A earlier study of te effect of sample size compared eigt differet test problems, usig 00,000 samples as te becmark [Gao 995]. We sall examie te comparative cofidece levels of tese widely differig sample sizes i te ext sectio. φ 66 33

- 0% 30% 0% 0% 0% -0% -0% %Cage i Momets for Mote Carlo α3 α 95% of our repeated Mote Carlo rus would fall witi te ± limits. Te cofidece iterval i usig Mote Carlo to predict a assembly parameter may be estimated by meas of te biomial distributio [Sapiro 98]. For example, te reject rate is expressed i parts-per-millio (PPM), tat is, te umber of reject assemblies per millio. Te stadard deviatio i predicted PPM rejects may be calculated from Eq. -. A equivalet o-dimesioal form, percet of te rejects, is also sow. As te sample size () doubles, te error estimatig te rejects decreases by (for >>). -30% -0%.E+03.E+0.E+05.E+06.E+07.E+08.E+09 Figure -3: Percet Cage i Momets for Mote Carlo Simulatio as te Sample Size Doubles Usig te percet cage i momets as a error measure distorts te results. Tis is because percet error is amplified we te omial value is ear zero. For example, te percet cage of te mea ( ) is very small due to te large value of te mea. Accordig to te grap, te error i estimatig te skewess appears greater ta te error i kurtosis. Sice α 3 is ear zero, te percet cage is large, wile α, beig muc larger, as a muc smaller percet cage. Usig a percet cage i te raw momets or te percet error i te ormalized momets is ot a good estimate for accuracy. A better metod of calculatig errors will be preseted i Sec. 3. As Figure -3 sows, te percet cage i te momets as te sample size is doubled approaces zero as te sample size reaces a billio. Te becmark momets ad rejects for te clutc assembly are preseted below i Table -. Table -: Becmark Results from Mote Carlo Simulatio at Oe Billio Samples Cotact Agle for te Clutc Value (Mea)...7.0953 (Stadard Deviatio)... 0.9668 α3 (Skewess)... -0.099 α (Kurtosis)... 3.0386 Lower Rejects (ppm)... Upper Rejects (ppm)... Total Rejects (ppm)...,06,66 6,57.3 Estimatig te Effectiveess of Mote Carlo How ca we estimate te error i a Mote Carlo simulatio witout avig to ru a billio samples? If a Mote Carlo simulatio is perfomed for 0,000 sample assemblies, te repeated, te secod assembly parameters will ave differet values from te first, due to te radom ature of te simulatio process. Eac time we repeat te simulatio, te results will vary, givig us a "feel" for te reliability of te estimated values. Te scatter or probable error may be expressed quatitatively i terms of te stadard deviatio of te repeated values, wic correspods to te spread of te resultat istogram plot of te repeated values. A iterval of ± defies a cofidece iterval of 68%, ± correspods to 95%, etc, wic meas tat PPM 6 PPM ( 0 PPM) 0 6 PPM % PPM PPM PPM PPM ( ) Were: Eq. - PPM te estimated parts per millio rejects PPM te oe-sigma boud o te error i predicted rejects %PPM te oe-sigma boud o percet error i PPM rejects. Te oe-sigma cofidece limits calculated by Eq. - ca be multiplied by a appropriate value to obtai ay desired cofidece level for te estimate of accuracy. Te figures preseted i tis paper will use te oe-sigma cofidece limits for te errors for easy compariso. To explai wat te oe-sigma error meas, cosider te lower PPM rejects for te oe-way clutc assembly. If a Mote Carlo simulatio of 0,000 clutc assemblies is ru, te rejects correspod to assemblies wit cotact agles outside of te specificatio limits ±0.6 degrees, preseted i Table.. Te lower PPM rejects correspod to about a.6 sigma quality level, or,600 PPM rejects. Figure - below sows a istogram of te percet error for repeated simulatios. Eac data poit o te grap represets oe cycle (or ru) of 0,000 Mote Carlo samples. A total of,000 suc rus were performed to geerate te istogram. Te istogram sows te spread i values obtaied by Mote Carlo simulatio for a sample size of 0,000. A sample size of 0,000 would be cosidered by most Mote Carlo practitioers to be large eoug to gauratee accurate results. However, we see from te istogram of repeated rus tat tere is cosiderable variatio i te results.

Output Distributio for %Error i Lower PPM Rejects 0 Oe-Sigma Limits 0 00 80 60 0 0 0 - -36-9 -3-6 -0-3 3 0 6 3 9 36 8 55 %Error i Lower PPM Rejects Figure -: Histogram of %Error for Lower Rejects for,000 Simulatios of 0,000 Samples Eac Te istogram portrays te oe-sigma limits for te percet error, as calculated wit Eq. -. Te mea of te percet error for all,000 rus (or cycles of Mote Carlo) was about two percet. Te oe-sigma limits are about ±6%. If te sample size were doubled (0,000), te te oe-sigma boud o te percet error ( %PPM ) would be divided by te square root of two, ad would te be reduced to %. Tus, as te sample size icreases, te istogram becomes more arrow, ad te accuracy icreases. Table - sows te mea ad stadard deviatio for te percet error of te lower, upper, ad total PPM rejects calculated directly from te,000 Mote Carlo simulatios. Te estimates of stadard deviatio of te percet errors are also compared as calculated by Eq. -. Table -: Distributio for %Error i PPM Rejects for 0,000 Samples %PPM Lower PPM %Error Upper PPM Total PPM Mea Histogram.% -9.9% -.% Stdev Histogram 5.8% 0.0%.6% Stdev Calculated from Eq - 5.% 0.%.% Eve at 0,000 samples te oe-sigma boud o te percet error for te lower PPM rejects is still about fiftee percet. Te importat result i te table is tat Eq. - gives a very close estimate of te stadard deviatio witout te eed of repeatig te simulatio.. Error i Estimatig First ad Secod Momets wit Mote Carlo Te metod described above for estimatig te error i assembly rejects obtaied from Mote Carlo simulatio may be exteded to oter assembly parameters of iterest to desigers. Te error i te estimate of te mea of te output distributio is preseted i Eq. -. ˆ Stadard Normal Variable Eq. - Were: ^ te estimate of te first momet of a distributio Te becmark value of te mea te oe-sigma boud o te error of te mea estimate. Te stadard Normal variable is a o-dimesioalized error estimate. It as bee used as a estimate for te error i te first momet [Sapiro 98, Crevelig 997]. Te quatity is also called te "Stadard Error of te Meas" [Jamieso 98]. It is a fudametal statistical parameter wic requires some explaatio. Suppose a large productio ru is sampled parts at a time, ad te mea of eac sample is calculated ad a istogram plotted, as was doe i te previous sectio for PPM rejects. As we accumulate samples, te istogram will approac a ormal distributio, wose mea (tat is, te mea of sample meas) approaces te mea of te wole productio ru. Te spread, or stadard deviatio, of tis istogram of mea values is called te stadard error to distiguis it from te stadard deviatio of te wole populatio. As see i Eq. -, te stadard error for te mea decreases wit te square root of (sample size). By extedig tis cocept to oter assembly parameters besides te mea value of te assembly resultat, ew metrics for evaluatig Mote Carlo accuracy ave bee developed, wic are preseted i te ext sectio. Estimates of te variability i te secod momet of te assembly distributio, te variace, as calculated by Mote Carlo simulatio, ave bee made by applyig a Ci-Square distributio. Te Ci-Square distributio for variace ca be approximated by te ormal distributio (wit te same mea ad variace) for large sample sizes [Vardema 99]. Tis is appropriate, sice Mote Carlo geerally uses large sample sizes. Tus, te accuracy of coutig rejects, ad estimatig te mea ad te variace of a assembly distributio ca be evaluated quatitatively. Te accuracy of eac of tese parameters improves as te umber of rus icreases. Sectio 3 will expad upo tis to estimate te accuracy of te skewess ad kurtosis. 3. NEW METRICS FOR ESTIMATING THE ACCURACY OF MONTE CARLO SIMULATION Sectio preseted te state-of-te-art i terms of estimatig te accuracy of Mote Carlo simulatio. Eq. - was used to estimate te oe-sigma boud o te error of estimatig te PPM rejects. Eq. - was used to estimate te error for te mea. Tis sectio will ivestigate te stadard errors for te first four momets of a distributio as a fuctio of sample size. 3. Stadard Errors for Estimatig Momets wit Mote Carlo Te estimate for all of te momets of te output distributio improves as te sample size icreases. Te oe sigma boud curve is a good idicator. Te accuracy of te estimate of te mea ad PPM rejects of te output distributio was preseted i Sectio. Four error measures for eac of te four momets of a distributio are defied i Table 3-. Eac is odimesioalized i terms of te distributio variace raised to a appropriate power.

Table 3-. Stadard Error Metrics for te Momets of a Distributio Stadard Error Stadard Deviatio of Variable te Error SER ^ - SER Eq. 3- SER ^ - Eq. 3- SER ( ) SER3 ^3-3 Eq. 3-3 SER3 3/ 00 SER ^ - Eq. 3- SER 6 were ^ i te it momet of te distributio estimated from measured samples, simulatio, or oter variatioal aalysis metods. i Te becmark value of te momet SERi te o-dimesioal stadardized error SERi te Stadard Error of te Momet. Te stadard momet error for eac of te four momets decreases wit (sample size). Usig stadard error, istead of just te error i te momet itself, elimiates avig to scale te estimate of error by te stadard deviatio or percet. ˆ Stadard Normal Variable SER SER ˆ Variace Eq. 3-5 Were SER te oe sigma boud of te estimated mea. Te oe-sigma error boud for te estimate of te variace is a little more complicated. Te distributio for estimatig te accuracy of te variace is te Ci-Square, wose mea ad variace is sow i Eq. 3-6. After a little maipulatio, te stadard error of te variace SER is sow as te last equatio. ( ) ˆ Mea Variace ˆ Variace SER Ci - Square Variable ( ) ( ) ( ) ˆ ( ) ( ) Eq. 3-6 Te errors for te skewess ad kurtosis are ot as easy to calculate as wit te mea ad variace. Usig data from several Mote Carlo simulatios, te equatios for te oe-sigma error bouds for SER3 ad SER were foud empirically to be described by equatios 3-3 ad 3-. To visualize te probability distributio for SER of te clutc assembly at 0,000, figure 3- sows a istogram of SER for,000 rus of Mote Carlo (eac at 0,000 samples). 0 0 00 80 60 0 0 0 Output Distributio for SER Oe-Sigma Limits -0.33-0.8-0.-0.9-0.5-0.-0.06-0.0 0.03 0.07 0. 0.6 0. 0.5 0.9 0.3 SER Figure 3-: Histogram of SER for,000 Cycles of Mote Carlo, eac wit 0,000 Samples Te mea SER for te,000 cycles is very close to zero. Te stadard deviatio of SER is about 0.. Te distributio looks sligtly skewed to te rigt, but overall it is ot far from ormal. Table 3-: Probability Distributios for Estimatig SER SER at 0,000 Samples Calc. from SER SER SER3 SER Mea Histogram 0.0003-0.000-0.0005-0.000 Stdev Histogram 0.00 0.05 0.05 0.08 Stdev Eq. 3-- 0.00 0.0 0.00 0.00 Te oe-sigma error bouds for te stadard errors from te istogram are very close to tose predicted by Eq. 3- troug Eq. 3-. SER is sigificatly larger ta te oter stadard errors (te times greater ta SER). To covert te oe-sigma error bouds i te stadardized momets back to oe-sigma bouds o te origial momets, Eq. 3-7 below is used. i SERi Eq. 3-7 i

Were: i i t power of te stadard deviatio of te distributio (if te actual stadard deviatio is ot kow, te estimate ca be used, if te sample size is ot too small) 3. Grapically Represetig Error versus Sample Size A useful grapical demostratio of te error will ow be preseted. Te percet error i PPM rejects for te clutc problem, alog wit te oe-sigma bouds o te error, are plotted i Figure 3- versus te log of sample size. 50% Actual ad Sigma Limits for %Error of Couted Lower PPM Rejects usig Mote Carlo to cout rejects. Te term cout refers to te metod of coutig rejects durig te Mote Carlo simulatio ad te calculatig te resultig PPM rejects. Te teoretical oe-sigma limits versus sample size for te stadard errors SER troug SER are sow below i Figure 3-. Te actual errors sow are from two differet Mote Carlo simulatios of a billio samples. Additioally, te oe-sigma bouds o error are sow for compariso..e+0.e+00 SER versus Oe-Sigma Boud -- 00% 50% 0% +Sigma Actual -Sigma.E-0.E-0.E-03.E-0 Simga SERa SERb -50% -00%.E+00.E+0.E+0.E+06.E+08.E+0.E-05.E+00.E+0.E+0.E+06.E+08 F igure 3-: Percet Error of Couted Lower Rejects wit Mote Carlo Sowig Oe-Sigma Bouds for te Error Te + ad sigma limits i te figure above sow te sixty-eigt percet cofidece rage for te percet error of PPM rejects. Te absolute lower boud is 00 percet, as ay lower would be predictig egative rejects. Te oe-sigma boud estimates are calculated directly from Eq. -. Figure 3- illustrates te percet error versus te cofidece limits up to a sample size of 00,000; afterward te error is too small to be see o te grap clearly. To improve te resolutio, te same iformatio is displayed o a log-log scale grap below i Figure 3-3. I order to plot te percet error o a log scale, te absolute value of te error was take. Additioally, te cofidece limit bouds are symmetrical, ad tus form a sigle boud. All subsequet graps of error versus sample size use tis same format. -.E+0.E+00.E-0.E-0.E-03.E-0 SER versus Oe-Sigma Boud.E-05.E+00.E+0.E+0.E+06.E+08.E+0 SER3 versus Oe-Sigma Boud Simga SERa SERb -- 000.00% 00.00% %Error i Coutig Lower PPM Rejects for a Mote Carlo Simulatio -.E+00.E-0.E-0.E-03 Simga SER3a SER3b 0.00% Cout Sigma.E-0.00% 0.0%.E-05.E+00.E+0.E+0.E+06.E+08 0.0%.E+00.E+0.E+0.E+06.E+08.E+0 Figure 3-3: Percet Error for Coutig Rejects wit Mote Carlo versus Sample Size Tis grap allows te errors for all of te samples to be viewed togeter relative to te oe-sigma boud. It appears tat Eq. - for te oe sigma boud geerally describes te error of

--.E+0.E+00.E-0.E-0.E-03.E-0 SER versus Oe-Sigma Boud.E-05.E+00.E+0.E+0.E+06.E+08 Simga SERa SERb Figure 3-: Error i Momets versus Sample Size ad Oe- Sigma Boud Te oe-sigma boud curves o te graps are calculated by Eq. 3- troug Eq. 3-. Eac of te oe-sigma boud curves as te same geeral slope, but is sifted up. Figures 3- sow tat te equatios for te oe-sigma bouds for SER troug SER estimate te Mote Carlo error well for te wole rage of sample sizes. Havig a estimate for te error i te output distributio momets as a fuctio of oly te sample size is a valuable tool to determie te umber of samples to ru. Figure 3-5 below sows te four oe-sigma bouds o te same grap to allow for easy compariso. -.E+00.E-0.E-0.E-03 Oe-Sigma Boud for SER- versus Sample Size.E-0.E+0.E+03.E+0.E+05.E+06.E+07.E+08 SER SER3 SER SER Figure 3-5: Te Oe-Sigma Boud o SER- versus Sample Size Te oe-sigma cofidece itervals for all four stadard momet errors ave a very similar slope (decreasig as te square root of, wic is -/ o a log-log plot). Te error i SER is 0 times te error i SER. But, to acieve a error level of.0 for SER, oe millio samples is required, versus 0,000 for SER. Te plot ca be used to estimate te error i te momets for ay sample size. Oter sources of error ave bee cosidered: )te error ieret i fittig a distributio to te assembly data, )te error ieret i te iput data. Te results are preseted i [Cvetko 997]. 3.3 Applicatio of te Stadard Momet Errors Te Stadard Momet Errors, SERi, may be used to estimate te error due to sample size i a Mote Carlo simulatio witout performig a large sample becmark test ad witout repeatig te simulatio to see ow muc te results cage. Equatios 3- troug 3- ad 3-7 ca predict te Stadard Momet Error for eac momet ad covert it to dimesioal quatities. Te actual value will be witi oe stadard deviatio of te estimate wit a 68% cofidece level. est ± est ± SER est est ± est Similarly, est ± 3 3est ± - est Eqs 3-8 - est 3 00 est ± -6 est were i te i t momet of te distributio for sample size iest value of te i t momet estimated by Mote Carlo i te 68% cofidece iterval of te momet for repeated simulatios est est, estimated by Mote Carlo Table 3-3 sows te estimated oe sigma error bouds usig Eqs. 3-8 for te clutc assembly, compared to 000 repeated simulatios of 0,000 samples eac. Te stadard deviatio is calculated from 000 values of te momets obtaied from te 000 Mote Carlo rus. Te 68% cofidece iterval is estimated from te Stadard Momet Errors. Mote Carlo Ru # is sow as a typical example of te results of a sigle ru. α 3 ad α are stadardized momets ad ave more recogizable values. α 3 ad α tell us ow close te distributio is to a ormal distributio ( α 3 0 ad α 3.0). Table 3-3. Compariso of Errors for te Clutc Assembly (Sample Size0,000) Mote Carlo Mote Carlo - 000 Rus 68% Ru # Max/Mi Std Dev Cof It 7.0 7.088/ 7.0086.0003 ±.00 0.0893 0.05036/0.0598.00077 ±.00069 3 -.00086 -.000/-.008.00063 ±.000 0.0073 0.00788/ 0.0068.0005 ±.00033 α 3 -.0795 -.00/-.6833.065 α 3.0590 3.05/.8578.053057 Most aalysts would assume a sample size of 0,000 to be large eoug to obtai accurate results. But, te Max/Mi values sow i Table 3-3 demostrate te wide rage of values wic result from repeated simulatios. A sigle ru could be aywere i tis rage. Tere is close agreemet betwee te stadard deviatio calculated from repeated Mote Carlo rus ad te ± cofidece itervals predicted from te Stadard Momet Errors. Comparig te iterval variatio estimates to te Ru # momets, ad appear to be sufficietly accurate, wile α 3

ad α may ot be. To reduce te error itervals by a factor of would require times te sample size, or 0,000.. CONCLUSION Te priciple cotributio of tis paper is te demostratio of ew metrics for estimatig te accuracy vs. sample size for five commo assembly parameters obtaied by Mote Carlo simulatio. Furter applicatio to a variety of assemblies is eeded to validate te ew metrics. Te cofidece iterval predicted from te Stadard Momet Error offers a alterative metod of estimatig te error, witout avig to repeat te simulatio. It is bot simple ad accurate. It also elps i selectig te sample size, if error reductio is desired. REFERENCES [Case 99] Case, Keet W. ad Ala R. Parkiso. A Survey of Researc i te Applicatio of Tolerace aalysis to te Desig of Mecaical Assemblies. Researc i Egieerig Desig, 3 (99): 3--37. [Case 995] Case, K. W., J. Gao, ad S.P. Magleby. Geeral -D Tolerace Aalysis of Mecaical Assemblies wit Small Kiematic Adjustmets. J. of Desig ad Maufacturig, 5 (995): 63--7. [Case 997] Case, K. W., J. Gao, ad S. P. Magleby. Tolerace Aalysis of - ad 3-D Mecaical Assemblies wit Small Kiematic Adjustmets. Advaced Toleracig Teciques, Jo Wiley (997), Ed. by H-C.Zag, Cap. 5, 03--37. [Crevelig 997] Crevelig, C. M. Tolerace Desig: a Hadbook for Developig Optimal Specificatios. Addiso- Wesley, 997. [Cvetko 997] Cvetko, Robert, "Caracterizatio of Assembly Variatio Aalysis Metods", MS Tesis, Brigam Youg Uiversity, Dec. 997. [Early 989] Early, R. ad Tompso, J., Variatio Simulatio Modelig-Variatio Aalysis Usig Mote Carlo Simulatio, Failure Prevetio ad Reliability -989, ASME Publ. DE-Vol. 6, pp.39-. [Gao 995] Gao, J., Case, K. W., ad S. P. Magleby, "Compariso of Assembly Tolerace Aalysis by te Direct Liearizatio ad Modified Mote Carlo Simulatio Metods," Proc. of te ASME Desig Egieerig Tec. Cof., 995, 353-360. [Jamieso 98] Jamieso, A., Itroductio to Quality Cotrol, Resto Publisig, 98. [Sapiro 98] Sapiro, S. S. ad A. Gross. Statistical Modelig Teciques. Marcel Dekker, 98. [Vardema 99] Vardema, S. B. Statistics for Egieerig Problem Solvig. PWS Publisig Co., 99.