ControlLogix 5000 EtherNet/IP Mystery

[RajeevCell]

We have an Allen Bradley ControlLogix 5000 system operating for about eight months. It controls a stamping machine. anyway, it broke down on December 28, 2009. Then suddenly started to work from the next day morning. No surprise that it has broken down again two days ago, after running for about 12 days. I'll explain the symptom of the breakdown. But let me describe a strange thing that I noticed here.

Description of the PLC.
1. Power supply
2. CPU ControlLogix 5555 (slot 0, catalog 1756-L55 A)
3. 2-Axis servo controller (slot 1, catalog 1756-M02AE/A)
4. DC Input card (slot 5, catalog 1756-IB32)
5. DC Output card (slot 7, catalog 1756-OB32)
6. EtherNet/IP communication bridge (slot 8, catalog 1756-ENBT A)
7. DeviceNet card (slot 9, catalog 1756-DNB)


Now, from my limited IT knowledge I think the EtherNet/IP bridge should connect the PLC system with some external IP network. But strangely enough, there is absolutely no connection to this module, except the backplane.

There is an HMI connection in the system, but that is on the DeviceNet card, and through a CANbus cable.

Does anyone know why the EtherNet/IP card is there? Can it have any function I'm missing?


Now about the breakdown. The symptoms are . . .

- CPU starts up. "OK" and "RUN" LEDS glow right. "I/O" LED blinks.

- Servo controller - "OK" LED blinks. "FDBK" and "DRIVE" LEDs for both channels remain off.

- DC Input card - "OK" blinks.

- DC Output card - "OK" blinks.

- EtherNet/IP card - "OK" on. "NET" blinks. "Link" off. Scrolling display says "Link Lost".

- DeviceNet card - "OK" blinks, "I/O" blinks, "MOD/NET" on.

- HMI starts up, cannot read any param from the PLC


I need to fix the breakdown and get the production line back immediately. But I cannot find any clue. Is it possible that the EtherNet/IP card is looking for a link and finding none, it is stopping the CPU?

We have two other machines like this, and both have similar Ethernet card with NO external connections. None of them are reporting "Link Lost".

Thanks much for your time. I really need some insight into this mystery. Best regards.

 

[amptramp]

You did not tell us the color of the flashing LEDs.

I do not think the Ethernet card is faulting the system. You would have a solid red LED on the Processor card. Is the Processor faulting? If so, can you go online and see what fault is being enumerated?

The Processor I/O LED flashes green when the Processor has been configured for a certain I/O layout but it is having problems communicating with one or more I/O modules.

I would guess the Ethernet card is an unused part of the system - installed but unused - maybe for future data collection. Or it could be configured as an adapter for remote I/O. This scenario might explain most or all of the flashing LED's.

Go to these links and download the user manuals for the Processor and Ethernet card:

http://literature.rockwellautomation.com/idc/groups/literature/documents/um/1756-um001_-en-p.pdf

http://literature.rockwellautomation.com/idc/groups/literature/documents/um/enet-um001_-en-p.pdf

 

[djs]

The blinking "OK" lights on the various I/O cards means that they are not being configured by the controller. If the "RUN" and "OK" lights on the processor (slot 0) are red then that means that the processor is faulted. A fault can sometimes be cleared by switching the processor to and from run to program mode several times. This is performed by using the key switch on the front of the processor. Turn it to the PRGM position, then to the run and back several times. Leave it in the run position.

However while that process will clear a processor fault, it will not fix the issue that caused that fault. In order to do that, you will need to get a working copy of the RsLogix 5000 programming software. Or call the people that built the machine.

 

[Agfa]

I agree with amptramp's opinion, the EtherNet/IP is idle for future connection. You OIU communication with the PLC through the DeviceNet module. Nothing wrong with the E/I module.
Nothing wrong with the PLC system according to your description.
I suggust that: Check the linkage between Servo controller and the stamping machine, and the status of the machine.

 

[PLCMentor]

As everyone has agreed, the ethernet card seems ok. The link is lost because nothing is connected to it. It may have just been used for programming. You do have an I/O issue. It would be good to go online and check this first. Once online, you can scroll down to the I/O and look for small yellow triangles. If you cant get online, check all your I/O. I would assume that the problem is I/O that is attached via a communication link. Device net seems to be your only active comm link; however, I am not familiar with the servo card you are using so there may be an issue there. Check the manual on that card to make sure the indicators are indicating correctly. The fact that your DeviceNet I/O light is blinking I believe indicates a problem there. Also, the OK light should be solid green. I am not sure how the Operator Interface looks to the card via that interface, but the fact that it has a problem may be your whole issue. Check the Device net card again. If there is a problem with a node, it should indicate the node on the little display. If that node is also your Operator Interface, then you have verified your problem. Since the problem is sporadic, check for terminating resistors on either end of the devicenet link. If you dont have any, they need to be added. That in itself may make the problem go away. Thats the easy checks. After that, you will need to be able to get online with RSLogix and RSNetworx for devicenet to further troubleshoot the problem.

Russell White, P.E.
Automation Technologies, Inc.

http://www.AutomationNC.com

Automation Help

http://www.PLCMentor.com

 

[RajeevCell]

Thanks everyone. That was helpful.

As indicated by all contributors here, the EtherNet/IP card was not a problem. The OEM placed it there to download their program into the system (faster than the serial link). It did not do anything useful in my system, and it was NOT the cause of the failure I faced.

I made a gross mistake in evaluating the AB system's health. I thought the CPU still has its program in place. The "OK" LED was green, indeed. And on a couple of occasions, the system came back to life, after the failure. But the revivals did not last long. I was hopeful that the program inside is OK. I was wrong.

ControlLogix CPU will glow the OK led green, even if it doesn't have a program but the CPU hardware is OK. It has a flash backup, and it does try to restore the application from flash, if it detects any corruption in the main copy of the app.

So, I speculate that, somehow my application memory got corrupted, and the system went down. But upon next power up, the CPU tried to copy the app into the main memory, and the system cam back to life. However, the main problem in the CPU memory was not rectified, naturally and the system failed repeatedly.

All these conclusions were reached externally, because we do not have AB software (RSLogix 5000). I did download free copy of RSLinx which showed that the whole system backplane and the DeviceNet network are fine.

The OEM finally sent another CPU with program loaded, and plugging that in solved everything. We lost more than a week of productivity in that line.

 

[amptramp]

RajeevCell:

Thanks for the follow-up. The solution to the original problem is always nice to know.

 

[PLCMentor]

RageevCell,
Yes thank you for letting us know what the problem was. Your next step is very important also. Most people will mark this off as problem solved and move on. I would suggest that your team get together and evaluate the reaction to this issue and what methods would have brought the line up faster. It is much easier to look back at reactions to an issue and see ways to do it better than it is to evaluate things in the middle of the problem. I suspect when you figure your lost revenue due to downtime, it would have been cost effective to have the OEM tackle the issue. At the bare minimum, the cost of this downtime should justify you having a copy of RSLogix5000 onsite. You may also want to look into giving your OEM remote access if they dont already have it. I am glad you are back up, but dont forget to look back and see what hampered your efforts.

Russell White, P.E.
Automation Technologies, Inc.

http://www.AutomationNC.com

Automation Help

http://www.PLCMentor.com