Relay Redundancy and fault tollerance

[Noway2]

During a project review with a high level manager, it was stated that we need to investigate the concept of using fault tollerant / redundant relays in out system. The idea behind the concept, I believe is to gain a marketing edge. This person suggested the use of Allen Bradley Safety Relays, though I am not certain how they would really be applicable.

The concept of the project, as it stood, was to use inexepnsive dumb IO (relay) cards with as little hardware on them as possible to minimize the chance of items going bad. There are two flavors of the IO cards, one of which is used predominantly for alarm panel contacts and the other is used to drive engine control solenoids (water, fuel, starter, etc).
From what I have been able to find, there are safety relays which are mechanically designed to not actuate one set of contacts if the other becomes fused. The cost of these relays ranges between $7 and $30. In contrast, the AB safety relays that tbrought up in the meeting look like they take a contact input and provide an electronically monitored contact with a redundant backup circuit at a cost of about $60. This is in contrast to the $1 DPDT relays we have always used.

I am wondering if a form of Solid State Relay or two FETs would be of benefit and if it would be possible to monitor any leakage currents or voltages to determine if a failure has occured, assuming this might be a possible approach. The idea being to implementations I have seen on SCRs to detect an open or shorted SCR.

While I am not certain if the market for the product would even want or need these levels of redundancy, ie be willing to pay for it remains an other issue.

I am trying to evaluate my options in this regard to present to my boss so he can determine if the cost is justifiable or not. My question to everyone here is, do any of you have any ideas or suggestions as to how you would impliment a fault tollerant "relay" type circuit providing a NO and NC circuit upto about 8A?

 

[VEBill]

Can you use simple feedback (monitoring the relay) and software, and then a separate relay to shut the whole system down (into a safe mode) in the event of any failure?

The problem with redundancy (implemented simply) is that the failures could go either way (series contacts sticking open, parallel contacts sticking closed).

 

[Noway2]

You are very correct abou the two failure modes of the relay and this is where I am running into a sticking point.

When you suggest monitoring the relay, I am assuming you mean monitoring the relay contacts. At this point, the hardware is still on paper and changes are feasible. What I don't understand, though, is how I could monitor the contact outputs without interfering with the user's circuitry that is anticipating a set of dry contacts.

For most of the points to be monitored, between 13 and 30 relays depending on the options, an alarm message would suffice. For the case of where I would want to shut things down, which would mean stopping the engine, I could put circuitry in series with the water / fuel solenoid to kill the engine.

 

[sreid]

Fault tolerant in the commercial marketplace can rarely be justified. The concept usually involves duplicated hardware such as two computers where one takes over if one goes down.

The phrase "Safety Relay" to me means a relay that kills AC power if an e-stop is hit or a safety interlock opens. The relay contcts are "Positive Break." The relay can only be re-energized if the e-stop and interlocks are reset and the operator pushes a specific switch to re-energize power.

 

[Noway2]

Sreid,

I agree 100% with your statement about fault tollerance in the commerical marketplace. Additionally, what you indicated about the application of safety relays is in accord with my understanding of their application too. I don't see how they are applicable.

Unfortunately, I am in a situation where the person who is suggesting these items and told me to investigate them is not going to want to hear that (above). I have made that mistake before and suffering the resulting wrath once was enough.

The only thing I can do, is present the option from the standpoint of this is what it would cost and this is what it would accomplish, and hope that it gets decided that it isn't worth the cost.

Before I do that, though, I thought I would ask if anyone here has any suggestions of how I might accomplish the fault detection goal as I don't have any good ideas on this one.

 

[VEBill]

"...how I could monitor the contact outputs...?"

Assuming (for example) that the relay contacts are switching some power supply, then simply monitor the switched contact for the proper switched voltage. If the voltage is there when it shouldn't be, then perhaps the contacts are welded shut and it might be time to shut down the system. If the voltage is not there after closing the contact (and optionally it IS on the input contact), then the relay is not making contact and it might be time to shut down the whole system.

Given a suitable controller, the sensing line(s) might be as simple as a resistor into analog input(s). There will probably be saftey and isolation issues to address and the whole thing can easily spiral out away from simplicity.

That's one reason why military systems are so expensive. They do typically demand IBIT, PBIT and CBIT (BIT = Built in test).

 

[Noway2]

Ve1bll,

Your hitting the nail right on the head when you say that there will probably be safety and isolation issues and I can guarantee that the thing WILL spiral out of control.

The issue there is that I have no idea what so ever what is on the contacts. It could be 5vdc to 250vac and anywhere in between, with god knows what as a reference. Not to mention the issue of potential interference with the customer's equipment as these are the customers interface points.

If I were in control or at least had known restrictions on what was going to be applied to the contacts, I think your suggestion would definately be the way to go and I certainly do appreciate your input by the way!

 

[VEBill]

With 'user' contacts, you probably don't know what is safer: open or closed. Do you provide series redundancy to ensure that you can open the circuit one way or the other, or do you provide parallel redundancy so that you can close the circuit one way or the other? Which is safer?

Do you need to do both redundant series and parallel? There is probably a complicated wiring scheme that will get you the 1950's relay equivalent of best-of-3 voting(*).

(* This is the point where you get fired for being a smart@$$.)

That said, there are relays with positive action and "tell-back" secondary contacts (I think that is what they're called). I believe that if they stick, then there's a good chance (not 100%) that the secondary contacts will reflect the status of the primary contacts. I'm not sure if they work both ways.

One problem with simple redundancy is that you need to detect when you're relying on the redundant item. Otherwise, you've simply delayed the failure, not prevented it.

I'd certainly think about including an internal fuse (soldered in), if you haven't already, to protect the contacts that might be abused by the user.

 

[sreid]

Noway2,

Thank you for being so candid. I had to look up Fault Tolerant on the web.

http://www.webopedia.com/TERM/F/fault_tolerance.html

Perhaps you could define Fault Tolerant to make some sense for your product. If management wants some feature that differentiates your product from compedetors, perhaps you could suggest something like reading back the state of the relay coil.

 

[cbarn24050]

If you need extra reliability you can use 4 contacts in series parallel to replace each single contact. In most applications contacts in series or parallel sufices depending on which condition your trying to guard against. Force guided relays only give an indication of welded contacts they dont show up failed contacts.

 

[Noway2]

The responses to my inquiry are pretty much confirming what I suspected, but I wanted to ask the question to see if there were an approach that I had overlooked.

One of my key design goals, for other reasons, was to make all of the IO dynamically configurable with regards to its function. Therefore, it shouldn't be too difficult to wire a set of contact in parallel or series depending on if an NO or NC contact is desired. Granted, this will only delay a fault not prevent it, but I feel it is better than designing in extra circuitry on all the contacts when I don't believe that the customer will require it.

Monitoring the relay coils, however, is a feasible option. It would likely require an approach of putting logic on the IO cards, which we had avoided but I can at least present that as an alternative.

Again, I want to thank everyone for their input and taking the time to respond and to say that the advice I have been receiving is very helpfull.

 

[itsmoked]

I'd just add a micro that scans all the contacts and their coil commands and then can offer up a useful display like "relay 6 failed". This allows you to essentially add a board that is separate from the rest of the logic.

 

[sreid]

Another way to claim "Fault Tolerance" (and you may already be doing this) is to have an under voltage detector and a watch dog timer for the microprocessor control (and pad unused memory with No-Ops). This is to try keep the processor from getting hung and to cause an automatic code restart.

 

[djs]

It may be possible using an optical isolator (LED/phototransistor combo) to monitor the status of a contact. Connect the LED part in series with a resistor across the contacts. If the contact is closed, a small current will flow through the LED turning on the transistor. If closed, no current will flow, turning off the transistor.
Two down sides: If the contact is not connected or the external voltage supply is off, open status is invalid. Two it is no longer a "dry" contact, there will be a small "leakage" current.
By changing the specifications of the new unit, some of this can be bypassed.

 

[Noway2]

The power supplies, all four of them, each have a power monitoring device associated with them that will generate a valid system reset in the event of a brownout. While there is a watchdog timer on the processor I am not a real big fan of these because as the datahseets says, if the processor clock fails the watchdog won't do you a bit of good. Instead, I have utilized a watchdog circuit, based on a seperate clock source, that can reset the system.

Padding the unused code with no-ops, however, is a good idea as it would keep the unit from doing any unusual activities like firing off IO ports if it were to go into the weeds.

I was just discussing the fault tollerance concept with my boss and how I am concluding that it isn't going to be practical. He raised a point, about adding any circuitry that could interefer with the user contacts. This product, due to its nature, has to be approved by Factory Mutual. He feels that it is likely that they would take issues with anything that could possibly interfer with the dry contacts as they tend to view the machine as sacrificial equipment and would be more concerned about anything affecting the operation.