IP Office 4610SW sudden registration problems

[BillykIII]

In the last few weeks I have suddenly started to have a problem with my IP phones. For no particular reason they all restart and get stuck at Discover:XXX.XXX.XXX.XXX where the xxx is the correct address of the IP500 V2 that they are attached to. Even more bizarrely we think we have it narrowed down to when we connect to the IP500 V2 to look at or modify the config in any way. When we exit the phones fall off the system. This could be a coincidence!
What can I tell you about the system? I have three IP500 V2 in three locations using IP Office R8.1 Manager all conected to a VM Pro voicemail server. I have Digital30s with 5410 and 5420 digital phones at these locations. I have 9 remote locations with 4 to 9 4610SW phones in them, The 4610SW use 2.9.1 files. The whole system has been wonderful for several years. I have not changed any of the basic settings other than the usual admin stuff with users, etc.

It has become a real issue as I recently had to move 50 people and offices to different locations due to a flood in our main office. I had the brilliant idea of using IP phones to accomplish this in the location that I could not wire the digital sets. So I had set up 9 4610SW phones. They work great when they work but they, and all the rest!, go out and seemingly when we open the manager application.

Please let me know your insights and where I might look for an answer. We patch regularly, so the PC with the VM Pro and IP Office manager was recently patched; could it be that? The network is managed by another group and they may have been doing things to the network...
Argh.. I am lost.

Thank in advance.

 

[IPOLackey]

Well when discover comes up it is normally a routing issue on the network.

The manager issue may be a red herring, but how do you connect. Is it installed on a server? Or do you dial into the system via ISDN?

 

[BillykIII]

The manager issue may be a red herring, but how do you connect. Is it installed on a server? Or do you dial into the system via ISDN?

Thanks!
I think routing issue as well but have been assured nothing has been changed. They want an Avaya tech to assure nothing is wrong on the phone system. I am in the process of obtaining some tech time.
Alas it could be, red or otherwise it's beginning to stink! I am going to test it tonight when no one is on the system. The manager and VM Pro are installed on a Win7 Pro box used solely for this. We connect over the LAN and use RDP as with all our servers.

 

[mojoputter]

I'll bet if you statically assign those IP phones they will work, which should tell your network group they have an issue on the network.
I'd also make sure the network group has the option 172 setup on the DHCP server which should give those ip phones the correct information to boot up with and register to the IPO. If you have separate VLANS for voice and data, have them statically assign a pc in the voice VLAN and then see if they can ping the IPO, if they can't there must be some kind of a routing issue.

No matter how hard you try, you can't fix stupid...

 

[BillykIII]

I'll bet if you statically assign those IP phones they will work, which should tell your network group they have an issue on the network.
I'd also make sure the network group has the option 172 setup on the DHCP server which should give those ip phones the correct information to boot up with and register to the IPO. If you have separate VLANS for voice and data, have them statically assign a pc in the voice VLAN and then see if they can ping the IPO, if they can't there must be some kind of a routing issue.

Thanks!

DHCP contains the 176 SSON with a pointer to the IP500 V2 and the TFTP server for the a and b files etc. and that works fine. We do not currently use VLANS, we do have 13 subnets though. I have tried static addressing with no success with this issue, they react as the DHCP phones do in that they work fine when the work.

I can watch the phones start and they load the first file, connect, dhcp and receive their address, connect with the TFTP server and get all of their files .txt, a and b. They then go into discover mode and display the address of the IP500 V2 they are assigned to. ANNNNNDDDD there they stay.

Another tidbit in this roundup is that as I stated above there are three IP500 V2 units in my system and they all share a VM Pro server. The one IP500 V2 that is on a different subnet cannot talk to the voicemail server. I mean with voice stuff because I can see it in any other way, ping, etc.

Thanks again all!!

 

[BillykIII]

Still working on this. Narrowed it down a bit to the fact that we loose connection between the IP500s in the SCN that are on a different subnet. Cannot ping from one to the other. Network guys are setting up wireshark for some investigation. I will keep updating as we go and am open to suggestions from Y'all as well.

 

[BillykIII]

Hey,
If you can believe it this problem is still ongoing!
My vendor has escalated it up to the next tier support. Hope to get this done soon.

 

[joe2938]

sounds like a routing issue between the subnets or no route built between them

 

[amriddle01]

Yep, routing issue/network issue for sure

 

[BillykIII]

Indeed that is what is believed to be the problem.
However, the system has all the routes properly built and delineated. We are using Lanshark on the port the IP 500 unit is on. We do hope to see what is happening here.
What I do know is that when I open the manager and select any or all of the controllers, after a few moments of the configs being opened, all of the IP stuff starts to fall off the system, phone and the connections between the IP 500s. When I make a change, even an artificial one to get it to merge, on ALL three IP 500s and do a merge the IP stuff comes back up. Crazy as this sounds, I can reliably repeat this. The only thing we need to confirm is if it happens to IP stuff on the same subnet. To test that I have put two IP phones on the same subnet that two of the three IP 500s are. I have to do this as all the phones on these are digital, IP phones are used in my smaller branch offices. I will trigger the "fall off" this evening and use the monitor app to see if this happens.
The Lanshark set-up will go on this evening.
I'll let you know!!

 

[amriddle01]

Sounds like an IP address clash, upon sending/receiving data to/from an address that doesnt speak to a certain address very often it will kick of the ARP requests and that's when the clash becomes apparent as you describe

 

[BillykIII]

Hey!
Still working on this!!
My vendor is escalating it with Avaya. Will keep you posted.

 

[janni78]

How can they escalate this with Avaya?
Release 8.1 should be end of support by now.

Last planned service pack for 8.1 was released Dec 1st 2014.

"Trying is the first step to failure..." - Homer

 

[intrigrant]

To give you a hint : the problem is that UDP/TCP ports 1718, 1719 and 1720 are blocked between the sites.
"Nothing changed" is a common reaction from IT engineers but most likely they did a upgrade on their network or some minor changes which breaks the system.
Nevertheless something HAS changed as there was no problem in the past.

 

[BillykIII]

@janni78 That would be up to my vendor. That is what I was told, 10 years they have not caused me to dis believe them.

@intrigrant I have to agree, but the network is not my sandbox...

It is indeed strange that it only happens when I load the configs into the manager. When I save them it all comes back up. Grrrr....