Remote 1616 and 1608 IP sets no dialtone, can't hear people calling in

[gkurcon]

We have a S8300 running CM 5.2. Last night around 8pm we started having some strange network issues, almost like a broadcast storm or possibly some sort of routing loop. The only thing we did yesterday (3 or 4 hours before the problems started) was to add a new SAN device's management interface to our LAN, it has since been removed from the network. We've gotten the data end of things straightened out and stabilized, but now none of our remote sites (WAN) can make calls (no dial tone) and when they receive calls they can't hear anyone on the line, but the caller can hear them. If I connect to Audix from the remote site it works fine (at least I hear the greeting and can log into a mailbox etc-I didn't try to actually send a message). The remote sites are all 1616 and 1608 sets that register back to the S8300 over the WAN. The WAN is a 5MB ring with Cisco routers running RIP at each remote site-they all connect back to a Cisco 3750 core switch the main site. We've had this in place for two or three years without issue. I've rebooted remote site routers and switches, rebooted phones, re-registered phones, rebooted our core switch which connects the LAN that the S8300 is on to our WAN, rebooted the access switch the S8300 is attached to. Nothing has worked so far. Anyone have any sort of recommendations? We have a call into our vendor and I'm still waiting for a response.

Any suggestions are appreciated!

 

[Hudson22]

What alarms are in your pbx?
Can you ping all your equipment at the remote sites?
Hudson22

 

[gkurcon]

Yes we can ping all items at the remote sites-phones included! I can browse network shares at those sites and they can access server resources here at the main site as well-so no problems on the data side, only voice. One thing I just noticed is that I can't ping either of our G350's from nodes that are not on the same VLAN, but it pings if I use a device on the same VLAN. Is that normal? I can ping the S8300 with no problem regardless of what VLAN I'm on.

 

[Hudson22]

If you status your Media Gateways, do the show up or down?
hudson22

 

[gkurcon]

They appear to be up...if I'm reading this correctly...

status media-gateways


ALARM SUMMARY BUSY-OUT SUMMARY H.248 LINK SUMMARY
Major: 0 Trunks: 0 Links Down: 0 # Logins: 01
Minor: 0 Stations: 0 Links Up: 2
Warning: 42

GATEWAY STATUS

Alarms Alarms Alarms Alarms Alarms
MG Mj Mn Wn Lk MG Mj Mn Wn Lk MG Mj Mn Wn Lk MG Mj Mn Wn Lk MG Mj Mn Wn Lk
1 0| 0|20|up
2 0| 0|22|up









Command:

 

[Hudson22]

You could check the G350's arp cache's, just a shot in the dark.

ARP table commands
Use the following commands to configure the ARP table. For more information about these
commands, see Avaya G250 and Avaya G350 CLI Reference, 03-300437.
● Use the arp command to add a permanent entry to the ARP table. Use the no form of this
command to remove either a static entry or a dynamically learned entry from the ARP
table.
● Use the arp timeout command to configure the amount of time, in seconds, that an
entry remains in the ARP table. Entering the arp timeout command without a time
parameter will display the current timeout value. Use the no form of this command to
restore the default value (four hours).
● Use the clear arp-cache command to delete all dynamic entries from the ARP table
and the IP route cache.
● Use the ip max-arp-entries command to specify the maximum number of ARP table
entries allowed in the ARP table. Use the no form of this command to restore the default
value.
● Use the show ip arp command to display a list of the ARP resolved MAC to IP
addresses in the ARP table.
● Use the show ip reverse-arp command to display the IP address of a host, based on
a known MAC address
hudson22

 

[gkurcon]

Yes I saw that elsewhere, problem is I have no idea how to get in. I have a username and password info for the S8300 and ASA but not the G350's apparently. I'm at the mercy of my vendor I guess.

 

[gkurcon]

Resolved.

Well the vendor finally called nearly 5 hours after I placed the service call. They had me reboot all of the remote site routers and our core switch at the main site, power cycle a phone and also clear it...nothing worked. Their final answer was to power cycle the G450's (I thought they were G350's..opps). After that we came back up and now the remote sites are working again. Interestingly, once we power cycled I could ping the G450's from other VLANs. So my guess is it might have very well been an ARP cache issue that was corrected during the restart. Since I could get to the G450's via the same VLAN, couldn't I have just logged in (assuming they gave me the username and password) via telnet and issued a command to clear the cache without taking ALL of my phones down to do so?? Isn't just pulling the power from the G450's frowned upon in general?

Yep, that's my Avaya platinum partner support team....sheesh.

 

[Hudson22]

No, that's how you shut down or power cycle the gateway. You could do a reset from the cli. Also keep in mind to do a copy run start prior to powering it down to back up the configuration.
Hudson22