CS1000E CPU-0 - System / Power light is RED

[mrclean0613]

I discover tonight that we have a RED light on our CS1000E -

it's on CPU-0 Very Top Cabinet
System = RED
Power = RED

The system appears to be running fine - was able to swap CPU's - just not certain to the cause

Not too certain how to troubleshoot this issue - sure not very comfortable with a power cycle

any suggestion are appreciated

Thanks,

mrclean613

 

[pbxn]

Take a look at your history file, specifically what happens around your midn's. Look for some PWR codes, sys monitor codes etc. Do you have a vendor that provides monitoring/maint contract? If so, time to have them earn their contract pay and have them come in a take a look

 

[mrclean0613]

There are no PWR codes in the history file

We're on T&M with a local vendor - The cs1000e appears to be running ok at the time - CPU-0 is the cabinet showing the RED light on both System and Power LED's

So it appears to be a power issue on CPU-0 - just not certain how to isolate

 

[Tman45]

Did you stat the cpu in load 135? Should look something like this (our card power light is green and our Redundancy light is yellow) - however the remote system that waits for failure of both cores at our main site is Red):

>ld 135
CCED000
.stat cpu

cp 1 9 PASS -- ENBL

SYSTEM STATE = REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 14
VERSION = May 23 2007, 00:18:36
Side = 1, DRAM SIZE = 1024 MBytes

CP[1] located at IPMG [8 0 9]


cp 0 8 PASS -- STDBY

SYSTEM STATE = REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 14
VERSION = May 23 2007, 00:18:36
Side = 0, DRAM SIZE = 1024 MBytes

CP[0] located at IPMG [4 0 8]

 

[mrclean0613]

>ld 135
CCED000
.stat cpu

cp 1 16 PASS -- ENBL

TRUE REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 20
VERSION = Jul 28 2005, 04:14:02
Side = 1, DRAM SIZE = 512 MBytes


cp 0 16 PASS -- STDBY

TRUE REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 20
VERSION = Jul 28 2005, 04:14:02
Side = 0, DRAM SIZE = 512 MBytes

****************
****************

Are you saying this lamp condition is normal? I don't recall seeing these lights being RED

 

[Tman45]

Not sure, but your stat cpu doesn't show anything as disabled. If you are really concerned perhaps do an INI when you can to see what happens

What is the light status on the 1 16 cpu? Green/Green?

 

[mrclean0613]

CPU-0 side is all GREEN lights

So, you suggest an INI - do I need to swap CPU's to get it on CPU-0, then INI the switch? Or can I initiate the INI on CPU-1?

 

[Tman45]

You can do both or either. If it were me I would likely INI on CPU-1 first to see if there are unexpected results (because you want to test your failover CPU anyway). Then if you run into trouble get CPU-0 active and INI again

Someone else may have a better opinion.

 

[janaya]

go to ld 37 and do:

.stat xsm

and let us know what you get.

 

[mrclean0613]

OK - Here's what I get - I don't believe there is no System Monitor on the CS1000E Call Server

stat cpu

cp 1 16 PASS -- ENBL

TRUE REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 20
VERSION = Jul 28 2005, 04:14:02
Side = 1, DRAM SIZE = 512 MBytes


cp 0 16 PASS -- STDBY

TRUE REDUNDANT
DISK STATE = REDUNDANT
HEALTH = 20
VERSION = Jul 28 2005, 04:14:02
Side = 0, DRAM SIZE = 512 MBytes

**************************
**************************

>ld 37
IOD000
.stat

TTY 0 : DSBL
TTY 1 : DSBL
TTY 2 : DSBL
TTY 3 : DSBL
TTY 4 : ENBL
TTY 5 : ENBL DES: ION_MODEM
TTY 11 : ENBL DES: CDR
TTY 13 : DSBL DES: ether1
TTY 14 : DSBL DES: ether2
TTY 15 : ENBL DES: ether3


.stat xsm


IOD300

0
.

 

[janaya]

Ok, your system is not reading your System Monitor (IOD300). What type of circuit card is connected to your System Monitor? Is is a 2 or 4 port paddle board. (After Release 25 you should be using a 4 port paddle board as the 2-port was MD and is not supported on the Succession Line) You may need to change this out as well as your system monitor to clear these issues. The system monitor is the lovely little card that controls the power functions in the column and can take a column down fast.

Here's what I would do:

1. disable your system monitor (dis tty 0)
2. disable all TTY ports on the paddle board, flip the toggle switch, pray you don't shake the card wrong and reseat the card. Any tech on here will tell you that if you rattle that card wrong trying to get it in or out can reset the switch.
3. Reseat the System Monitor card. While it is out make sure you are have the correct dip switch settings. Also to be a real stickler what is the vintage of System Monitor Card you are using? No AA, but AB to AD are supported. Without the AD, just remember your little Nortel blue light won't work.
4. Put the system monitor card back in
5. throw the toggle switch on your paddle board back to enable
6. enable all your tty ports on the card
7. do a stat xsm and see if you get any PWR messages

All PWR messages will be good because at least you are seeing your System Monitor chain.

But....
PWR000 = good message
PWR(anything else) No Bueno, especially the dreaded PWR007.

This should get you there. Now you can do this during business hours because the NTPs tell you how to do this without it being service affecting, BUT I would do it after hours. (I don't trust anyone)

Hope this helps. I just did this process last night for the same IOD message. customer had 2-port paddle board and bad System Monitor.

Also remember to have a secondary access point to the switch (J21 or J25) in case your terminal is on the Paddle board.

Hope this helps.

John

 

[janaya]

And noticing the rest of your capture you have have ask yourself "why is my system monitor disabled?" You may try to enable and re-run the stat xsm to see what it reports back...

but..they could be disabled for a reason.

 

[Firebird Scrambler]

STAT CNI in LD 135. Check if they are both enabled as they will knock out your TTY access.

All the best

Firebird Scrambler
Meridian 1 / Succession and BCM / Norstar Programmer in the UK

http://www.linkedin.com/in/davidbromley

If it's working, then leave it alone!.

 

[mrclean0613]

I appreciate the feed back

I'm familiar with the Paddle Boards on the Legacy Nortel Opt81C - CS1000M (several columns) architecture

However, this issue I'm having is on the Nortel RACK MOUNTED CS1000E - this is the Call Server Architecture

 

[Wane47]

a 1000E has neither system monitors or cni cards.

 

[mrclean0613]

Hey there Wane47 ~

Do you happen to have a CS1000E?

Would you happen to know what the normal lamp conditions are for both Call Server CPU-O & CPU-1 are supposed to be?

Is the inactive side suppose RED on the Power and System LED?

 

[sl1input]

Are you talking about the CPPM call server single slot card or the PP4 type with Utility card housed next to each processor?

If CPPM, then both should have green LED on the far left of card. Now the LED labled CS RED, one should be green (active) and one should be amber (standby).

 

[Wane47]

No, I am not a customer, I work for a vendor. My primary responsibly is installation with a minor in maintenance.

The following is right the manual.


CP PM Call Processor faceplate LEDs
The status LED indications of the NTDW61 CP PM Call Server are as
follows:
• Off: no power
• Red: BIOS self-test running
• Flashing red: bootrom and Operating System (OS) loading
• Yellow: sysload phase 1
• Flashing yellow: sysload phase 2
• Flashing green: SL1 loading on active core
• Green: normal operation
The Active CPU LED indications are as follows:
• Off: no power
• Green: redundant mode, active
• Flashing green: split mode, active
• Yellow: redundant mode, standby
• Flashing yellow: split mode, standby
• Red: single mode
The ELAN LED indications are as follows:
• LED1 Off: 10 Mbps
• LED1 Yellow: 100 Mbps
• LED2 Off: no link, no activity
• LED2 Green: link valid
• LED2 Blink: link valid and activity
The HSP LED indications are as follows:
• LED1 Off: 10 Mbps
• LED1 Yellow: 100 Mbps
• LED1 Green: 1000 Mbps

 

[mrclean0613]

Hey there Wane47 ~

I'm referring to the Alarm/Fan module (NTDU64)

On the inactive CPU Alarm/Fan module (NTDU64)
Here are the lamp conditions on the Alarm/Fan module

System LED = RED
Power LED = RED
The other two LED's = GREEN

On the Active CPU - Alarm/Fan module
All four LED's = GREEN

Does anyone know what the inactive CPU LED’s are supposed to be on the Alarm/Fan module?

 

[sl1input]

sounds like you got the old school cs1000e, Pentium 4.

I would start by connecting pc to com 1 on CPU with red lights and see what is going on in there.