Trouble shooting aids. 4

[Nehemiah]

At the company that I work for, we have several cards that go bad routinely. It costs us anywhere from $2k to $10k to have repaired. These are fixed prices independent of the what is broken. I know these are standard practices regardless of the industry, but I would like to try and do the repairs inhouse if possible.

First, does anyone have any experience to tell me if it is typically a good idea to try and do in house, component level repairs?

Second, I have done some looking and found that since I was trained in electronics troubleshooting and repair that they have lots of software that looks like it would help. Some of the software allows you enter in a wiring schematic and it will turn out a simplified schematic and you can do simulations to aid you in trouble shooting. Does anyone know if they have software that will allow you to do enter in your faulty readings and it will suggest faulty components or do you still have to do this by hand?

Thank you for any input on this.

 

[BrianG]

Hi there. I take it that these cards are bought-in for use in your own systems (PC-based or PLC?). If these fail so regularly as you say then wouldn't it be worth investigating if they are being overstressed by your application demands, or are being damaged by static discharge as a result of poor storage or handling?

If you really want to repair the cards in-house one of the problems that you could encounter are that the card has some custom chips, and you may not be able to obtain replacements from the manufacturer.

Secondly, unless you are manufacturing other circuit cards of your own, or otherwise geared up with using the latest repair techniques, how will you deal with any defective surface-mount chips or other components that the card vendor might use?

Regarding software for testing, yes, simulation software allows allows you to draw the schematic and then simulate the functions (we use a package called Simetrix). However as far as I know all such simualtors assume the models of the chips are perfect - I don't know how you could realistically specify a faulty chip - so you can really only vary things like resistors and capacitor values to see what happens. I think even current ATE testing of circuit cards still only manages to get down to a likely faulty component without guarantees - you still sometimes have to resort to using a 'scope or testmeter to be sure!.

 

[busbar]

Unfortunately, details about service documentation should have included in the original procurement contract. There is no clue as to equipment type, or the contractual responsibility of a vendor. Until the maintenance costs become prohibitive so far as the “main office” is concerned, there is probably no inexpensive solution.

Remember who not to buy from next time.

 

[Nehemiah]

I wish we had a real option, busbar. Our purchasing dept. is notoriously closed minded. They ALWAYS buy from the cheapest bidder. We have an amazing array of sensors and equipement because they will not specify in contracts the maker of sensors for future equipement. Every company gets to put on the equipment whatever is on hand.

Almost all of this is for drivers or powersupplies. The reason these cards fail so often is no that they are necessarily being used improperly but that we have 30-50 of them scattered amount our plant.

All of the IC's are mounted in sockets for easy removal and replacement and with the exception of one EEPROM they appear to be standard logic chips. Typically what goes wrong is a capacitor or resistor fails. The cards are all hand soldiered to start with and are not that difficult to repair.

We have most of the troubleshooting eqiupment on hand all ready because on occation, when we do not have a spare on hand, we have to do the repair ourselves. But that takes hours, so if there was a way to speed this up, it would be great.

Well, looks like we will probably stick with our current methods for a while. Thank you all for the input and assistance.

 

[BrianG]

I understand that the cards are relatively simple logic, which is a plus point, but I am not quite clear from your statement about purchaing policy whether you have many different cards from a variety of manufacturers or lots of cards all the same. Are they "industry standard" cards?

I don't know how you go about fault-finding at the moment, but if you have lots of cards of relatively few types would it be worth making up some simple test jigs to provide known inputs so that you could then measure the corresponding outputs?

 

[dpc]

Board-level repair might seem to be a great idea, but I doubt if it would ever be cost-effective, unless you were dealing with hundreds of similar boards.

With more and more tiny surface-mounted components and multi-layer PC boards, repair is exceedingly difficult assuming the problem can be determined. It also requires a lot of expensive equipment.

Your situation of having multiple vendors for similar items makes board-level repair even more difficult.

There is probably someone above your Purchasing Department that could be persuaded that the life-cycle cost of inferior products is greater than a quality product.

 

[busbar]

Management support is too many times hard won, but, however
slowly, buyers sometimes are very slow to understand that low bid is not always practical and may significantly cut into production uptime.

Just don't box yourself in too much. The rampant "What have you done for me TODAY?" point of view sometimes lasts far longer than I care for.

 

[IRstuff]

Ditto DPC's remarks...

"penny-wise and pound-foolish" should be applied.

Perhaps something equivalent to a blacklist/probation list needs to be developed by Engineering and sent officially to Procurement stating why certain suppliers are blacklisted. Suppliers can petition to be removed, based on their agreement to improve quality and service.

One thing that needs to be immediately implemented is a detailed documentation of the downtime/repair cost of each and every failure. This is the foundation of DPC's suggestion about life-cycle cost. Then, you'll be able to compare suppliers and individual parts to show how the immediate cost savings results in massive support and downtime costs. Ultimately, Procurement and other organizations are responsible for bottomline cost and profit. If it can be clearly shown that Procurement's lowest bidder approach actually hurts profits, things should change quite dramatically, although, there's no guarantee that the messenger won't get shot ;-) TTFN

 

[plasgears]

Are the boards two sided conductors? That would aid reliability. Ask if this is the case.

 

[Nehemiah]

The boards are two sided conductors for the most part.

The cards in question, that are really expensive to repair and therefore the ones that I am interested and willing to try doing in house, are only about 10 different cards. They are used in alot of the equipment though. If there was software that would really aid, I was considering adding more cards that are only single use and such. But since there is not, I will not worry about it.

A few of the cards only have about 20 components on them so they really are not THAT hard to trouble shoot. But one in particular is a module that has 4 cards and cost us $7k to repair last time. That module is one that I do not even want to try....I am not great at trouble shooting cards and am the only one here who has any training/experience in component troubleshooting.

I am going to start tracking the downtime associated with the cards themselves and see if I can make changes with corparate buyers.

Thank you all for the input. It has been most helpful.

 

[SuperE]

Nehemiah,

Your situtation sounds familiar to a lot of work I was involved in while in the military. You have some legacy technology PCBs that break too often and you are tired of paying someone to do relatively straight forward work, if you knew how to do it.

Well given that you have trained electronic technicians, they may not be trained in this area, all you would need to add is the proper equipment. I do not know of any R2D2 computerized solution but a good PCB can be directly comparied to a a bad PCB with a product called a huntron tracker. It uses low voltage and varying signals to probe and help locate faulty components. Coupled with a schematic (optional) a good electronics technician can repair most faults, depending on the PCB technology. The first course of action is your eyes and your nose, also look over the vendor repaired cards to see what was replaced. As you stated you may have a couple of components that fail regularly, experience counts.

If you have a trained technician great, get approval to repair a board to show you can. Otherwise consider hiring someone with the skills needed. If you are sending out enough boards, the cost savings may pay for the new position and benefit your bottom line.

There is not much component level maintenance in the military any more for most of the reasons mentioned by other responders. However a two-sided board with 1980s style components may lend itself to something you could fix in house. The cost savings you would realize by repairing one board may pay for your equipment.

If you have the expertise in house, great and good luck, otherwise you may visit your local community college electronics department or seek a vetran with component level skills.

Regards

 

[Nehemiah]

Thanks SuperE. I have used a Huntron Tracker when I was in the Navy as well. Was hoping for a better method than easter-egging though. Will have to see. And thanks

 

[philiptechie]

You don't state the complexity of the boards or the generation of technology employed.

However assuming they employ Surface Mount and Ball Grid Array packaging, the two aspects to be considered are fault finding and component replacement.

Many Board manufacturers provide Diagnostic Software and Circuit Diagrams. Armed with these, some basic test equipment and a modicum of experience it is possible to fault find down to component level.

The second issue regarding component replacement with todays ic packaging, presents the Greater problem.
For the removal and replacement of surface mount components should only be performed by trained operators to avoid damage to both the board and component. In the case of Ball Grid Array(BGA) packages specialist equipment is required.
It may also be a good idea to invest in an X-Ray workstation to be able to view the BGA connections, in the event of the pins being misaligned with the tracks.

All in all, the repair of modern boards as you can see is neither easy or cheap, and can only be justified by the economies of scale.

 

[philiptechie]

Sorry, I had'nt read you subsequent message, which states that the chips are mostly mounted on sockets with standard logic.

In that case, I would suggest you have a go!
Since the devices are easily available and can be easily replaced its worth a try. All you need is some diagnositc software and basic test equipment.

Hope my second reply is more helpful and encouraging

 

[lucino]

Good day.
Doing in house repairs at card level makes a lot of sense only if you don't have to troubleshoot which component is defective.
Nowadays electronic cards circuitry gets complicated and components smaller which is hard to be replaced without aid of sophiticated tools.
Hiring and/or assigning people to repair the card helps the economy but is additional operating cost to the company. Sometimes people tends to be inefficient to routinary activities.
I suggest whenever a card is broken a new replacementwill be installed and send the defective cards to the manufacturer for repair.

Thanks for your time.

Lucino

 

[nbucska]

If all IC's are in socket most likely error is IC or contact with the socket: If digital, I suggest
Replace all IC's with good ones. Then replace back
half. If works then the other 1/4. and so on in binary
fashion.





<nbucska@pcperipherals.com>

 

[Warpspeed]

There are several ways of approaching this, if all the devices are socketed and are simple common logic chips.

It might sound silly, but why not just replace every chip on the board and see if that fixes it? If the minimum repair cost is $2K, that will buy an awful lot of replacement integrated circuits ! The only thing you will probably need is some fairly basic anti-static equipment.

At the next level, you might be able to find someone with the required skills and test equipment. Perhaps a retired engineer, or someone moonlighting.

If all else fails, you can always return the chip to the supplier as you are now doing, but you still might be able to save a fairly high proportion of boards yourself with minimal skills or test equipment.

 

[electricuwe]

There are several reasons for high repair costs for electronic cards:

- The supplier earns his money from repairs so that he can sell his equipment at low price:

In this case you should consider if such suppliers are really the best choice for your company. I worked in the induction heating and there were several competitors who used parts with custom part numbers or who removed part numbers to make board level repair impossible to customers.

-The equipment is very old:
In this case repairs are very expensive because the suppliers needs to search for old schematics in the darkest part of his archive, needs to search for replacement for obsolete parts, needs to build new test setups for calibration. We usually recomended to our customers to grade up the control to the newest generation. This is usually more cost effective than doing expensive repairs again and again. One big advantage is that such an upgrade can be as a scheduled maintenance whereas the repairs have to be done when the cards have failed.

 

[Nehemiah]

I had considered just replacing the IC's. I suspect that they are in sockets for a reason. I have not seen any evidence of the board being resoldiered so I suspect that they just swap those out. The reason that the repairs are so expensive is they use the repair cost to lower the up front cost on their equipement. We now no longer buy their equipment but just have to deal with what we have on hand. Most people here at the company believe that it is not worth it in the long run. It had been tried along time ago by someone who, it sounds like, was not that skilled. They ended up sending all the cards in to get repaired because of his work.