NO System Redundancy Because of HSP Problem

[mszodiac]

Hi Everyone,
I'm hoping someone here as come across this particular system. We have a CS1000 HA system running 7.5.
We haven't had any redundancy for the last few months. The HSP is the issue I think, BUT specifically, it is the LCS. Here's the output.

ld 137

CIOD000
.stat hsp

LCS HSP STATE is DOWN --> I cannot get this to come up.
HSP LINK CARRIER: OK ---> it shows HSP link is up, although stat cpu shows it's down
Auto Negotiation: Enabled
Auto Negotiation Completed: YES
Actual Line Speed: 1000 Mbps
Actual Duplex Mode: Full Duplex

Ethernet (gei unit number 1):
Internet address: 127.2.0.1
Broadcast address: 127.255.255.255
Ethernet address: 00:1b:ba:fd:29:f7
Netmask: 0xff000000; Subnetmask: 0xff000000
874933 packets received; 1300102 packets sent
0 input errors; 0 output errors
0 collisions

===================
.stat cpu

cp 0 1 PASS -- ENBL

SYSTEM STATE = NOT REDUNDANT (HSP DOWN) ---> it shows HSP is down
DISK STATE = NOT REDUNDANT
HEALTH = 14
VERSION = Nov 16 2010, 14:42:29
Side = 0, DRAM SIZE = 1006 MBytes

CP[0] located at IPMG [4 0 1]


cp 1 UNKNOWN PASS -- STDBY

SYSTEM STATE = NOT AVAILABLE
DISK STATE = NOT REDUNDANT
HEALTH = 14
VERSION = Nov 16 2010, 14:42:29
Side = 1, DRAM SIZE = 1006 MBytes

CP[1] located at IPMG [UNKNOWN]

=========================================
Summary of Local System Resource (side 0)
=========================================
File Descriptors
-----------------
alloc 128
free 1920
total 2048

Unprotected Heap (bytes)
------------------------
alloc 83063704
free 438214968
total 521278672

Protected Heap (bytes)
----------------------
alloc 1790484
free 2403820
total 4194304
.

Please note the secondary system is at a different building. They are connected via fibre using fibre to ethernet transceiver.

I've changed the transceivers, and rebuild a new card as a secondary Call Server.

Any ideas? Thank you for your time, and input, as always.

 

[bignose21]

You could try popping the standby CPU out carry it to the same room as the other one put it in a spare slot and join with the direct HSP cable see if it comes up. This would rule in/out any sort on problem with the path.

 

[Tacosoup]

What does the window of the CPU 1 processor look like?

** Proofread carefully to see if you any words out.

 

[mszodiac]

@tacosoup. We have a CPPM card. There's no window on it.

 

[KCFLHRC]

Can you connect to the CPPM card for CPU 1 and talk to the card via the serial cable?

 

[Tacosoup]

If the CPPM card is down, then you may have one of the cards with bad memory.

** Proofread carefully to see if you any words out.

 

[mszodiac]

Yes, I can talk to the secondary CPPM card no problem. It is active. It is the redundant card so all I get is the redundant message on the serial port:
HIRM: Inactive Redundant Mode Performing Background Tests ...

 

[KCFLHRC]

I believe you are going to have to sysload both the active and inactive side to get it to come back. I have seen it before.

 

[mszodiac]

Thank you everyone for your input. I will setup a maintenance window and do a sysload on both the active and inactive side.

 

[KCFLHRC]

You may have to issue the join command after the sysload, but hopefully it will just come up

 

[Wanebo]

We once had an issue where we had the 2 cores seperated on 2 different floors of the building and we saw the problem you describe. We even took the inactive side to the other floor and connected it there and all was fine. Brought it back to it's regular home and patched it back in via patch cords and verified CAT6 cabling and it was down again. We ended up running a cable directly between the cores, no patch panel and patch cords, direct termination. Have not had a problem with it since connecting them that way.