81c thinks it's too hot and shuts down

[glarkin]

One day out of the blue our 81c (RLS 25) started getting PWR007 (high internal temperature) and PWR047 (temperature normal) errors, rapid fire, one after the other, for a couple of minutes. Our vendor said the heat sensor in the top cap was probably flaky, and since it only lasted a few minutes they said to just keep an eye on it. Next morning, same thing happened but the system monitor threw the breaker and shut down the switch. The heat sensor was replaced, and all was well for about two weeks when the same thing happened again. This time the system monitor itself was replaced, and everything was fine for another two weeks when it happened yet again! This time we're being told to replace the fan unit and the air flow sensor. The switch is in our network ops center and the room is temperature controlled so I don't know if the high temp readings are true or not. It's also been suggested that a paddleboard (I/0 board?) should be replaced. Has anyone experienced anything similar? Any input at all would be greatly appreciated. Thanks,

Greg

 

[JMKyle]

Have you consider replacing the Vendor?

 

[johnpoole]

agree with kyle.. the paddle board is not the 1st choice, check your filters.. theere is a reed switch on the top of your switch that could be bad, the xms monitor could be bad and yes it is a chance of a bad paddle board..

john poole
bellsouth business
columbia,sc

 

[Avaya911PLM]

The FIRST thing I would do at this point is validate the temperature.

Take a reading with a thermometer inside the top cap. You will find either:

A: You have a high temp problem. Look at the airflow and cooling.

B: You are getting a 'false' high reading. Look at the hardware.


You need to know where the problem really is before you can troubleshoot.

--
Fletch
Visit the Nortel Technical Support Portal at:

http://www.nortelnetworks.com/support

 

[glarkin]

It's been 3 weeks since my original post (and almost 2 months since this began) and still not much progress has been made. After some troubleshooting our vendor is now having us replace the PDU and some ribbon cables in that column, but the PDU they sent us was damaged so we have to wait another week for parts. After our initial crash we could run a couple of weeks before the system monitor would throw the breaker, now we're lucky to run for 20 seconds before it dumps. Another thing I've noticed is that when you do an EDD, or the nightly routine backs up the database to floppy, it will only write to the floppy drive on CMDU 1. Before we starting having trouble it would write to the drive on 0 or 1 depending on which core was active. Now it only writes to the drive on 1 and never to 0. I'll be glad when everything is back to normal. At least with the system monitor backed out the switch hasn't been crashing. Anyway thanks for reading...

Greg

 

[janaya]

as jmkyle said, I would look for another vendor. You should not have to wait this long if your pbx is bouncing like this.

 

[glarkin]

I would like to explore other vendors, but those decisions are made at the corporate level and a peon like myself has no say in the matter.

Greg

 

[johnpoole]

what do you get when you do a load 135 stat? swap processors and stat again. the cmdu should swap with the processor. i would pull the xms monitor cards until it was fixed, what do you get when you stat your xms's

john poole
bellsouth business
columbia,sc

 

[glarkin]

It turns out CMDU 0 was disabled so it has been enabled. The system monitor has been pulled until we get this issue ironed out. Thanks,

Greg

 

[johnpoole]

again great vendor, let the upper food chain know that your switch is now redundent because you are doing your job and the vendor is a dork

john poole
bellsouth business
columbia,sc

 

[janaya]

You mean to tell us CMDU 0 has been disabled all this time and your vendor did not catch this?

They did not get an alarm when the cores couldn't swap. This is basic monitoring stuff, IRIS should have caught.

Vendor is a dork - I like that.

John

 

[technickelone]

Old post but we are experiencing this now.. 6 times this 81 has tripped.. Tonight we are replacing all the sys monitors and a pdu. CNI card blew the first time which made us think that was it but it was replaced. So I will keep an eye on it.

 

[johnpoole]

i would pull the sys monitor cards, if it happens again. i've had dirty filters or a thermal switch do that. are you getting a xms alarm prior to power down, or do you get an actual temp alarm.. i've seen the one that says the temp has exceeded x degrees and the system will shut down in 60 sec. not a great alarm on a busy day, i pulled the xms monitor cards, very quickly. scheduled a replacement time

john poole
bellsouth business
columbia,sc

 

[ArchTelecom]

I hope you are keeping a written log of this so you can express your concern to Nortel about your vendor.

 

[johnpoole]

nortel cares not... carve that in stone

john poole
bellsouth business
columbia,sc

 

[glarkin]

Wow, this brings back some bad memories. In the end, we had to get our vendor to escalate the issue to Nortel to discover that some ribbon cables deep inside the column were not seated properly. This took just over two months, during which time our PBX crashed four times during production hours. Here's a list of parts that were replaced:

1 heat sensor
1 air-flow sensor
3 fan units (2 were DOA out of the box)
2 PDUs (1 was DOA out of the box)
2 system monitors (1 DOA out of the box)
a multitude of cabling

Thankfully things have been fairly calm since then. I did end up learning a lot about the hardware side of things from this experience, but I could have done without the downtime and frustration.

I hope you get everything sorted out soon. Good luck,

Greg

 

[johnpoole]

it is almost impossible to not seat those cables, they go in with a good hard lock. i would have never thought to check that. those ribbons have ram like locks on each side, sorta a no brainer

john poole
bellsouth business
columbia,sc