MTBF & the Land of Make Believe 1

[jOmega]

JBartos, in a previous thread (#237-69725), stated:

"Suggestion: There are VFD manufacturers that offer Mean Time Between Failure (MBTF) values for their products off the assembly line; however, not for the custom-made products, e.g. ABB"

To which we replied that MTBF is a crafted fairy tale.

That which follows is offered in support thereof.

For more than the last 4 decades,......
MIL-HDBK-217, Reliability Prediction of Electronic Equipment, U.S. Department of Defense, (sometimes referred to as MIL-Std-217)
.....has been widely used to predict product reliability and has been the de facto authority.

From

http://www.t-cubed.com/faq_217.htm

one learns that:

What is MIL-HDBK-217?

The original reliability prediction handbook was MIL-HDBK-217, the Military Handbook for "Reliability Prediction of Electronic Equipment". MIL-HDBK-217 is published by the Department of Defense, based on work done by the Reliability Analysis Center and Rome Laboratory at Griffiss AFB, NY.

The MIL-HDBK-217 handbook contains failure rate models for the various part types used in electronic systems, such as ICs, transistors, diodes, resistors, capacitors, relays, switches, connectors, etc. These failure rate models are based on the best field data that could be obtained for a wide variety of parts and systems; this data is then analyzed and massaged, with many simplifying assumptions thrown in, to create usable models.

MIL-HDBK-217 champions two methods of reliability prediction:
-- Part Stress Analysis and
-- Parts Count.
Part Stress Analysis requires more detailed information and is usually applicable later in the design phase.

Parts Count generally requires less information such as part quantities, quality levels and the application environment. It is most applicable early in the design phase and proposal formulation.

And therein lies the first fallacy: Most VFD manufacturers arrive at their MTBF numbers by means of the Parts Count method; it is much easier to do and much less time consuming.

Next I refer you to a paper written by Barry Ma and Mekonen Buzuayene; both of whom are impressively credentialed and eminently qualified to speak to the issue of MTBF and its weaknesses.

The paper is entitled: MIL-HDBK-217 vs. HALT/HASS and is found at

http://www.evaluationengineering.com/archive/articles/1100reliability.htm

(Out of respect for the copyright, I refrain from clipping and pasting any parts of their article herein.)

In the article you will find well pointed discussion of the following:

- reliability predictions not substantiated by real world failure data that often shows such predictions to be highly inaccurate
-discussion of Mil-Hdbk-217, Parts-Count Analysis, Part-Stress Prediction, and BellcoreTR-232
-Why MIL-HDBK-217 Turns Out Inaccurate Predictions
-A discussion of HALT/HASS methodology as a better model for failure prediction. (Highly Accelerated Life Testing and Highly Accelerated Stress Screening).

So, the bottom line.... if a VFD mfgr is stating MTBF numbers as representative of their product quality and reliability...... you need to ask......

- What methodology was used to obtain those numbers ?
- If he cites Mil-Hdbk-217 or Mil-Std-217.... ask which method..... Parts Count ... or Parts Stress.
- Also ask if the Reliability Analysis is revisited whenever a part change is made during the life of the product and if new MTBF numbers are issued ?

Here's the hard one....
- Ask what the actual failure rate is (for the product rating that you are considering) ..... and, be prepared to hear the orchestra start up ... and the tap-dance music to begin.... Failure rate should be stated as a percentage .....(number of installed failed units / number of units shipped) x 100. Oh, and don't forget to ask how old the numbers are..... are they recent....>6 months old... > 1-yr old ? .....etc.

Final comment: JB with regard to ABB.... their custom products use the same basic hardware as their standard off-the-line products.... only difference is that they are assembled into a system and there might be some ancillary components integrated into the system as shipped. The MTBF numbers that they would offer, would be for the standard hardware... not for the system. I don't know of anyone that does an MTBF analysis for a system when each system is unique.

 

[electricpete]

Good points.

My observation of similar attitude is in the area of motor lifetime or winding lifetime. When I am called to analyse a motor failure, or to recommend a proactive refurbishment or rewind interval, I am faced with a mentality that managers believe the equipment has a definite predefined lifetime.... let's say 20 years. Up to 20 years it should work perfectly if properly maintained. At 20.1 years we had better do something to refurbish or replace it because it will surely fail by 21 years ;-)

But the items which contribute to motor failure have various different causes with many different contributing factors unique to each motor. The manufacturer cannot possibly predict it ahead of time. The best we can do is get actual field experience data. But attempts at scientific large scale collection of that data I believe have fallen far short. The most comprehensive attempt I know of for motors is published in an appendix to the IEEE Gold book... But on close scrutiny there are reasons to be skeptical about that document as well.

 

[Marke]

I was recently requested to provide MTBF figures for a piece of equipment that is locally made in reasonable quantities. I located a specialist in this field, and he was able to provide the required calculations, but as the equipment uses commodity type components and not MIL Spec components, the calculations were based on generic values rather than values for the actual parts used or the actual operating conditions. A more sensible calculation could have been done if the FITs for the actual components used were available, but you can generally only get those for MIL Spec parts. Of course the customer is putting severe pressure on driving the price down and just would not entertain the use of MIL spec prices. The thing that really surprised me, was that the customer would not entertain the use of three years of history with over 100,000 peices out there. I would have thought that that was relevant as well. An accelerated lab test was also rejected because not enough units actually failed. To me, from the lab test and field experience, we could have said that the MTBF was at least .... but no, it is calculations or calculations based on fairyland data.
All it takes in a competitive environment, is for one supplier to quote a number for MTBF and all of a sudden, the purchasing guys latch on to that as yet another specification to insist on.
Best regards

Mark Empson

http://www.lmphotonics.com

 

[jbartos]

Suggestion: There are several ways to assess the quality of a product, namely,
1. To follow the reliability procedure, use a suitable reliability software, and reliability data from the manufacturer based on this approach.
2. Use an empirical approach that is often based on the service of the equipment and the simple use of probability.
3. Use various and appropriate variable signatures, e.g. current, power input, etc. and predict the product expected life.
4. Rely on the product reputation, awards, won prices, past experiences, etc.
5. If possible, actually test the product and assess it by comparisons to competing products by selected important items, functions, features, etc.

Old reliability, life-expectancy, etc. criterion, when it came to horses, was the check of the horse teeth.

Nowadays, it is important to find out what "teeth" to check at electrical products, e.g. VFDs, electric motors, transformers, etc.

 

[dadfap]

jOmega,

I have never believed the numbers given for MTBF, but here's how from an operational standpoint I have tracked increases in MTBF and the influencing factors for the VSD systems I sell.
1. Improved power supply in remote areas, desert oilfield, resulted in an overnight reduction in callouts for service personnel.
2. Improved training of customer's personnel resulted in two things
a. Better set up of the drive on electrical submersible pumps operating in oil wells which resulted in less premature failures of components due to bad set up procedures.
b. Better fault finding techniques resulting in less downtime and less wastage of good components through three things 1) more of a methodical approach and less "shotgunning" 2) better identifiction of suspect components 3) I actually got the guys to read the damn manual!

How did I track this? Over a two year period, for the same numbers of drives in the field, I sold substantially less parts!

So, MTBF needs to be viewed with an understaanding of the "human" & operational factors.

all the best

dadfap

 

[jOmega]

Gentlemen:

Excellent comments, all.

Much appreciated.

MTBF is one of those subjects that doesn't see the light of discussion very often... and needs to ..as there is widespread misconception of just what it is..... and isn't.

Kind regards,
jOmega

 

[IRstuff]

Should point out the HALT/HASS is an approach that does not necessarily have much meaning, unless you have the luxury of killing an army of parts.

We had an interesting exercise with a customer that demanded HALT/HASS on one of two units we were contracted to design and build. The system contained approximately 50 CCA's with a total of over 5000 components, mostly COTS, at that. After a while, they realized the absurdity of the requirement.

TTFN

 

[jOmega]

Y'all might want to take a look at another thread on MTBF.

Some good info there..

jO

thread248-72202

 

[geezer]

I have used parts count MTBF's with considerable success in predicting warranty return rates. The MTBF predictions have fallen within the 90% chi-square confidence band of the actual return rate. If appropriate derating is used (90% of junction temperature, voltage and current ratings and 50% of resistor power ratings)the additional "accuracy" of the parts stress MTBF method does not justify the effort needed to carry it out. Any more than two significant figures is an exercise in self-delusion, while getting the right order of magnitude is cause for rejoicing. I also ignore the quality factor for commercial parts as merely being an incentive for military contractors to use Mil-spec parts in their designs and use that for Mil-spec parts.

 

[jOmega]

Geezer,

Appreciate your comments.

Have spent time in both the Military/Aerospace and Industrial environments, I fully understand.

I would offer, though, that the industrial environment doesn't derate to anywhere near the Military/Aerospace practices. In fact... I know of some manufacturers products where you are lucky if they derate to 85-90%!... and at that, it's not across the board...

Over many decades, I've never seen the MTBF numbers to be anything more than a fairytale... I've known some real artists who could make the numbers anything you want them to be... Kind of like three card Monty on the streets of New York...

I've also seen burnin testing reduced to an absolute minimum or go to none at all. (saves cost, don't you know). There preference is to let the customers find the "infant mortality" failures.... hoping that the field failure rate doesn't spike on them... Some burnin testing isn't done with a motor load ... And some (575-600v class) don't see rated voltage until the customer connects them up in his facility.

And yet.... they'll all tell you that their MTBF numbers are in excess of 10,000 hours.... Seems like that's the magic number that they all claim to meet.

Yeah, Sure !

As a customer investing thousands of dollars in a piece of equipment, I sure want to know what the field failure rate is as a percentage of units shipped.... MTBF isn't worth the paper its printed on.... really. Accelerated life testing is worthwhile... as is accelerated stress testing. But then, who does it.... because it is expensive....

One drive manufacturer I know of first hand, had a closed door sign-off party for the Qual Test documents associated with their drive product some years ago. Closed door, because the tests were never performed..... and the consequent field failure rate was alarming. But yet, they espoused MTBF numbers in excess of 10,000 hours.

MTBF in the Industrial theater ?

Caveat Emptor.

 

[jbartos]

Suggestion: MTBF is found among factors that are used for the product evaluation / rating within the procurement stage. It is just there procedurally and may count as other criteria.

 

[IRstuff]

Actually, most military contracts no longer impose the strict derating factors anymore, since it's essentially impossible to get military qualified processor boards anyway.

TTFN

 

[jOmega]

IRstuff

Is that an across the board policy?

What about Custom Designed equipment ?

 

[IRstuff]

Not specifically. Even with custom design, where would you find enough QPL parts that have sufficient to actually derate from?

If you're lucky, you might find industrial temp grade parts, which would have essentially no margin for military temp applications.

TTFN