3300ICP random reboots

[pakojones]

Hi,

This is my first post on the forum, but I often read it, thanks to you guys for the given help

I'll try to explain the config and what's the problem:

We have two Mitel 3300 with a E2T card running MCD4 in a cluster managed by OPS (in a near future i'll migrate to Common distributed Data), all IP phones (about 630) are using the first ICP, and go to second only when first fails.
The problem is that both ICPs randomly reboots, before the reboot I get a "ICP Comm" alarm. After the reboot I get the "System Diagnostic Report" and in the file L2ResetLog.txt the first lines show this:

!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!

E2t 10.1.0.12 is not responding to PING @ TUE OCT 20 11:54:11 2009



This is not a LX system, so there is no L2 to reset
PING 10.1.0.12: 56 data bytes
ping: timeout
no answer from 10.1.0.12

E2t 10.1.0.12 is not responding to PING

Oddly, the 10.1.0.12 IP address is the one assigned to the E2T card on the same ICP that rebooted.

I also have a doubt, when a E2T card is installed, on the form "License and Option Selection" must appear 1400 IP devices, or 700?, because with the E2T card installed I have 700 IP devices only.

If you need some aditional info, please let me know.

Regards.

Jesus.

 

[Mitelpassion]

what software version are you on?

could be that the E2T is faulty.
can you ping the E2T IP from a workstation or something?

Do you perhaps have duplicate IP addresses assigned.
I've seen this before on a LX system. 3rd line pointed me to investigate the network.

it only happened twice.

Do you have any network scanning or snmp devices probing the controllers?

 

[B0yRac3r]

the 700 option in the licence and options section is configurable, you just need to click on change and then select 1200 radial button. Changing this will require a reboot though.

How have you configured the the E2T card on the system? is it set for DHCP or have you given it a static IP address? as above will be worth checking if theres an IP clash on the network?

What version of ops are you using with MCD 4?

 

[kwbMitel]

My recommendation would be to statically assign the E2T via the method laid out in the Tech Handbook.

The Maintenance logs aren't showing an IP conflict, right?

*******************************************************
Occam's Razor - All things being equal, the simplest solution is the right one.

 

[pakojones]

Hi,

Both systems are MXe controllers using version 10.0.0.10_2

Mitelpassion: Yes, I can ping E2T IP from a workstaion, and no, it's not possible that the E2T IP is duplicated, I use 10.0.0.0/16 for the voice VLAN, and the first half is for static IPs, and the second one is the DHCP IP pool. Also it's impossible to a user to connect to the voice VLAN

B0yRac3r: I don't really installed the E2T cards, it was another guy, but on the main system the IP is assigned via DHCP, and on the secondary one is statically assigned :-(

kwbMitel: No, the maintenance logs are not showing any IP conflict. I'll try to statically assign the IP for the main system.

Is there any way to verify the correct installation/configuration of the E2T cards?

Thanks!

Jesus.

 

[Mitelpassion]

if you get voice when dialling PSTN or voicemail then the card is installed correctly. no alarms right?
E2T is responsible for converting the voice from ISDN to IP and vice versa.

 

[pakojones]

But AFAIK if a MXe server doesn't have a E2T card, then the RTC makes this conversion, so, is there any way to confirm that the E2T card is doing its work?.

I have seen on the Enterprise Manager this error from Icpsys2 (10.1.0.20): Lost connectivity to node: Icpsys2 (10.1.0.20), possibly due to a node failure/network disruption.
How it's possible to lost connectivity with itself?

 

[IrwinMFletcher]

If the E2T card is physically installed, then it will perform the E2T function, so as long as it is detected, the RTC will not do any of the E2T work (apart from T.38 that is). If the E2T card is detected, but can't establish a connection with the RTC card, then the system will reset as you've described.

Just because they are in the same box, doesn't mean they can automatically talk to each other. They are two separate processors. Make sure the IP addresses etc are configured properly.

"Icpsys2 (10.1.0.20): Lost connectivity to node: Icpsys2 (10.1.0.20)"...what are your configured addresses for each RTC/E2T pair on your two controllers?

 

[pakojones]

This is the IP config of both controllers
icpsys1 RTC -> 10.1.0.10/16
icpsys1 E2T -> 10.1.0.12/16
icpsys2 RTC -> 10.1.0.20/16
icpsys2 E2T -> 10.1.0.22/16

From a PC on the same VLAN I can ping to all.

 

[IrwinMFletcher]

How long do they stay up before they reboot? What I'm getting at is: do they both actually come up long enough to make E2T calls and then fail (hours, days later), or does the E2T never connect?

Also, what are the bootlines for the E2T cards?

If they never come up, I would lean towards the bootlines being bad, if they come up and make calls then die, I would lean towards something else in the network, duplicate IP etc.

 

[pakojones]

Hi IrwinMFletcher,

The system is working, users may call outside, but sometimes one of the controllers reboots.

What you mean with E2T bootlines?

 

[IrwinMFletcher]

The E2T has a 'bootline', the same as the RTC cards. The list of parameters it uses to start, you can see this from the 'version' command.

But since it come up and you can make E2T calls, chances are it is ok. I would then look as other suggested at the network.

The RTC and E2T send messages back and forth over IP, these include both control messages as well as heartbeats. If for example another pc comes onto the network with the same IP as the E2T, the RTC will send messages to it, won't get a response and think the E2T is dead and reboot.

There is also the chance that you E2T card is flakey, but it both controllers are doing it, then either you're extremely unlucky or it's something on the network.

 

[Waldosworld]

Is site a new install or recently upgrade?
If you perform maintenance command UpgradeBootrom all does it show the E2T already at correct level?
This can be done and upgraded in low traffic just requires reboot if not at correct level.
If the E2T was incorrectly configured with default alarm thresholds you would have an alarm and would not be visible in hardware config.

 

[pakojones]

I have a doubt, is this IP config correct?

For icpsys1
System IP Address: 10.1.0.10
Subnet Mask: 255.255.0.0
Gateway IP Address: 10.1.0.11
Layer 2 (L2) Switch IP Address: 10.1.0.11
E2T Card IP Address: 10.1.0.12 / 255.255.0.0

For icpsys2
System IP Address: 10.1.0.20
Subnet Mask: 255.255.0.0
Gateway IP Address: 10.1.0.11
Layer 2 (L2) Switch IP Address: 10.1.0.21
E2T Card IP Address: 10.1.0.22 / 255.255.0.0

Thamks.

 

[pakojones]

One question,

If I execute UpgradeBootrom E2T and the card need to be upgraded, when it finishes upgrading, it reboots automaticly, or it offer the option to reboot later?

Thanks.

 

[paterson]

That looks like a conflict to me. change the ICP1 switch IP address to an unused address.

_______________________________________________________________

If you did not take enough time to get it right the first time...

What makes you think that you have time to fix it?

 

[MITELPETE]

Great thread, really useful. thanks all.

Pakojones - The suggestions re ip address conflict/DHCP I think are the most likely. But beware that if you choose to execute the UpgradeBootrom E2T you must follow the advice in the release notes - If it isn't followed your Mitel will die beyond repair:
Mitel help:
Mitel strongly recommends that "Enable Autocomplete" is unchecked and you must use "Submit" button to execute ( not to hit Enter to submit the command)

 

[pakojones]

Thanks to all for your answers, but finally I will install everything on another ICP controllers from scratch, because this system has been upgraded by another guy, and maybe he made a mistake :-(

Thanks again