Monte Carlo Question

[DanStro]

I work with optical designers and as one of the checks for expected performance they perform a monte carlo simulation of the system they are designing.

Normally these systems have a fair number of variables that are being simulated (usually ~100) simultaneously. Each one is changed, within bounds, and the system performance is evaluated. Because of the amount of time it takes to do each run they usually limit the simulation to 100 runs. They insist that this means the error of the predicted performance range is within 1% of the actual.

However I am not so sure since there are so many combinations possible because of the number of variables that 100 runs seems way too small to get a decent confidence. Am I just too paranoid or are they underestimating the number of runs needed?

Any help is appreciated.
Dan

 

[GregLocock]

Gut feeling is that 100 runs for 100 variables is nowhere near enough. I analyse non linear systems, typically with 10 factors at 3 levels, and typically use 200 to 400 runs.

However, your system is probably linear, and you are not trying to identify the interactions and so on, so it may be that you can get away with comparitively few runs.

As a quick check you could run your 100 runs experiements several times (with different sets of random numbers obviously) and see how the predicted performance changes. That is a very robust approach. You would expect a normal ditribution of scores.


Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[PSE]

My guess is that the optical designers are trading off results accuracy for computational speed. For our guys, a single typical analytical run takes a weekend. This would be on a system say with some dozen elements or so. If you allow variations in curvature, center thickness, surface reflectivity and element spacing, that would easily reach the 50+ variables mark. I would be a bit surprised that they could achieve that accuracy in predicted performance. Either they have dropped a lot of rays out of the tracing or placed some significant limitations on the ray paths.

A lot depends upon the types of optical systems you are working with. Monochromatic systems are easier than polychromatic. If you do not have to "worry" about stray light analysis, that makes things simpler as well. I disagree with Greg's assertion that the system is probably linear but again it depends on the level of detail necessary. Aberration coefficients are highly non linear and interactions are highly significant.

The devil as they say, is in the details.

Regards,

 

[IRstuff]

Maybe it uses DOE to maximize its performance?

TTFN

FAQ731-376

 

[DanStro]

Unfortunately they aren't using a DOE, it really is as PSE says. They are just trying to balance time with accuracy. After discussing this with them more they are going to be running more simulations in general, but how many depends on the time frame.

At least now I know that I am not being too paranoid in doubting this.

Thanks everyone.
Dan

 

[GregLocock]

OK, why not look at the data you already have?

Split each set of 100 runs in two lots of 50. Do they agree with each other?



Cheers

Greg Locock

SIGlease see FAQ731-376 for tips on how to make the best use of Eng-Tips.

 

[DanStro]

GregLocock , I am not sure they have done that. I'll ask them. My hope is that they have and determined that they were close enough to each other.