ICP Comms Card Alarm

[Biscuits87]

Hi guys and girls. Having a bit of a headache with a site that I've picked up.
Systems are all MXeIII's running MCD 5 SP1 PR1.
In a 4 server cluster (2 Trunking G/Ws and 2 Agent G/Ws).

It was recently a 5 system cluster but the fifth site was de-clustered.
I can see no reference to the de-clustered system in any of the programming. Remote DN's have been removed, not in the Network or Cluster Elements assignment etc. On three of the 3300's I'm getting an ICP alarm, logs show the following:

[tt]2013/Jul/08 10:04:15 Call Control - Software MSG_DATA: $0 $6 $8 $5A $30 $1 $F4 $8D $44 $5 $C5 $5A $54 $20 $A $76RMSG DL State: 03, LinkH: 118Remote IP: IP ADDRESS Main log_outputs_util_en.c;1205

2013/Jul/08 10:04:15 Call Control - Software UNEXPECTED MESSAGE, OFFSET 2:$1492, PC: $368174LOG_ID: rmsgdlh#51, PROCEDURE: handle_rmsg_dl_release_indicatio, SEVERITY: cp_minorPID: ($6, $5AE)MSG_FUNC_CODE: $D5, MSG_TX_PID: ($0, $1) Main log_outputs_util_en.c;1205
[/tt]

The 4th 3300 is absolutely fine. It has no alarms and the logs are clear.
I've been going through it and trying to figure out what's different with the programming of that one but I've come up short.

As some extra info; There were no alarms when the 5th was originally de-clustered. Only when it was removed from the rack did the alarms start. Any thoughts?


(Sorry to those who thought it would be a simple "It's lost connection with a 3300" )

 

[Lundah]

2 Maintenance Commands to tell you which node the system with the alarm is looking for and not finding: "Message Remote" and "State Xnet All". One of those commands should show the PBX ID of the node that's not able to be reached and you can work from there.

 

[Biscuits87]

Thanks for the reply Lundah, below is a paste of the result:

Signalling | Link | Dest | XNET | State | Owner | ID |Status
Dvc Location |Handle|Switch| Trunk Id | | | |
-------------------------------------------------------------------------------
1 1 11 254 | 97 | 131 | |Established | | |
1 1 11 | | 132 | |Idle | | |
1 1 11 252 |126 | 133 | |Established | | |
1 1 11 2 |104 | 134 | |Established | | |
1 1 11 | | 145 | |Out of Service| | |
-------------------------------------------------------------------------------

145 is the node that's been de-clustered.

Message Remote:

>>>> REMOTE MESSAGE LINK STATUS <<<< Viewed from Plane: A Active
Transport Device | Transport | Related | Link | Link Status | Dest | Applic
Location | Device | Trunk # | Hdle | | PBX |
-----------------------------------------------------------------------------
1 1 11 254 | Ethernet | 0 | 97 | OPEN | 131 | XNet
1 1 11 2 | Ethernet | 0 | 104 | OPEN | 134 | XNet
1 1 11 252 | Ethernet | 0 | 126 | OPEN | 133 | XNet
-----------------------------------------------------------------------------

As you can see it's not there.

I've just checked... All 4 systems still have the ICP/XNET programming for the missing node. Even the system with no alarm has the programming. Possible that the alarm is due to the node being removed without the IP/XNET being deleted first?

 

[Lundah]

That's exactly it. Remove the ICP/XNET programming and the alarm should clear.

 

[Biscuits87]

You mean remove the programming from the elements still in the cluster?

Tried it with one and the alarm has gone to a major. Xnet now shows:

[tt] Signalling | Link | Dest | XNET | State | Owner | ID |Status
Dvc Location |Handle|Switch| Trunk Id | | | |
-------------------------------------------------------------------------------
1 1 11 254 | 15 | 131 | |Established | | |
1 1 11 253 |247 | 132 | |Established | | |
1 1 11 1 |120 | 133 | |Established | | |
1 1 11 | | 134 | |Idle | | |
-------------------------------------------------------------------------------
[/tt]

Suppose if it was on MCD 6 I could try the "Removene" command... but what would it remove? The programming has gone.

 

[Lundah]

Removing the NE from the Cluster programming and removing the IP trunk programming are 2 different things. You might need to reboot the controller after removing the IP trunk programming to the removed node for the alarm to clear. Alternately, you could just use Set Thresholds to squelch the alarm.