Confidence level in reliability requirements?

[McLeod]

How does your company use confidence level in setting its reliability design inputs and test acceptance criteria?

Given that the confidence level approaches zero as the bounds approach the nominal Weibull curve, what do you consider a minimum confidence level to set as the acceptance criteria? 90%? 70%? 50%? Lower...? Do you use the nominal estimate as the criterion?

While we understand the meaning and application of confidence level in statistical analysis, we also understand that the most commonly used value of 95% is an arbitrary historical choice going back to R. Fisher. The main problem is twofold:

[ol]
[li]The typical samples sizes that have been regarded as economically acceptable for tests of our more expensive products result in 95% confidence bounds that are rather wide in the area of interest.
[/li]

[li]The stress level applied by the users in the field varies so widely that a high degree of statistical confidence about our simulation test results is probably not as meaningful as one might think. (It's analoguous to trying to achieve 99% mesh convergence with a finite element model when your load case may vary by ±50%.)[/li]
[/ol]


Given all that, I've identified several options:

[ol]
[li]Set a minimum confidence level for reliability design inputs and test criteria at 90%. Everyone will just have to live with the higher costs associated with larger sample sizes and/or longer test times, or use more conservative (lower) reliability targets and claims.[/li]

[li]Allow any confidence level to be used, as long as it's stated in the design inputs and the test protocol before testing is initiated. Levels less than 70% will probably sound less rigorous than just using the nominal estimate.[/li]

[li]Omit confidence levels from the design inputs and test criteria and use the nominal estimate for pass/fail decisions, but require that 90% confidence bounds still be reported for informational purposes and a gut check for Quality and management.[/li]

[li]Write the design inputs and test acceptance criteria such that the lower 90% confidence bound is the limit for a clear pass, the nominal estimate is the limit for a clear fail, and in between is a yellow caution zone that triggers a management review and pass/fail decision. In practice, this would probably be equivalent to option #3.[/li]
[/ol]

Comments?

 

[IRstuff]

I'm not clear what you mean by confidence level. Military equipment generally uses MTBF, without any further stipulations or conditions. The only constraints are the actual environmental conditions that impact the failure rate.

The actual times to first failure vary by at least 50% of the predicted value in the BEST CASE, so are you saying that you'll derate the predicted MTBF value by 50%?

TTFN

FAQ731-376

 

[McLeod]

IRstuff,

This is not the same as derating. Whenever a parameter of a statistical distribution is calculated from sample data, there is some uncertainty associated with the estimate. A second sample taken from the exact same population will not give the exact same estimate as the first sample. Confidence intervals are an expression of that uncertainty.

This applies whether your parameter of interest is MTBF, shape factor, scale factor, or other.

See

http://en.wikipedia.org/wiki/Confidence_level

for more background details.

 

[IRstuff]

I didn't say it was the same, I was asking if you were going to derate the MTBF to get the confidence level up.

TTFN

FAQ731-376

 

[PSE]

Confidence level should have little or no bearing on the actual test being done (number of samples, method etc). You basically afford to do what you can with your test budget. It simply sets the bounds on whether or not to reject the "Null" hypothesis that you are testing against. I agree with your option 2 in that confidence level needs to be selected and indicated before the data analysis and NOT CHANGED afterward. Changing confidence levels after the fact can lead to bias.

Regards