Avaya 9611 No Dial tone intermittantly

[vulcanccit]

We have an Avaya CM that is our core in Phoenix, and 27LSP gateways around the country. Each market has their own sip trunk that terminates in the market's core cisco router and either an Avaya G430 or G450 as an LSP. One of our sites has been stable for years until we swapped out the Cisco switches to Cisco 2960Gs, and a new Cisco router. The same equipment is in place at all of our sites and all are working well. However, this site, usually in the morning, various phones will not get dialtone. If you actually wait about 6-7 seconds, the phone will eventually get dial tone. In the no dialtone condition, we have done a list trace as they place a call (which completes) and the trace is normal, you just cant hear who you called. We also use a monitoring tool that looks at our entire Avaya environment made by a company called Nectar. When the phone is in the no dialtone condition, Nectar shows the QOS for the phone having a jitter of 500 (normally this is 0 or at least under 20...). Also, Nectar show the typically connections. A good call will show the IP of the phone, and it will show the Phoenix (core) Medpro IP. In the no dialtone condition, it shows the call is terminated on the gateway's DSP card. It is as if it is asking for the dSP resource and gets stuck.

We have also found that this no dial condition usually happens after the phone has been idle for a long time, like over night and happens when they come in the next day, or if they have been away from the phone for hours...

Avaya has this now with a tier 4. They have replaced the gateway completely. They have made sure the phones have the latest firmware as well as the gateway. They have done the Craft Clear on all phones. No errors on the cisco side, gate way is set to auto auto and the switch the same (as in all markets). Avaya also put sniffers on the wire but as usual, when they had that gear online, the phones decided to be perfect lol

So while I wait on Avaya, I thought I would ask you experts if you have ever seen this.

Thank you all,

 

[smokinjoe2938]

Well you stated all was well until the Cisco swap out so this is where I would start looking, at the network layer.

acss sme acis sme acss cm 5.2.1 acss cm and cmm acss aura messaging.

 

[vulcanccit]

well the network boys state all is well but they always say that lol

 

[smokinjoe2938]

Your right they always say that as well as your local telco provider, but all was working well until the swap out, go figure!

acss sme acis sme acss cm 5.2.1 acss cm and cmm acss aura messaging.

 

[Hudson22]

Just a long shot:
Could check this out, had an issue with a customer:
MTU issue:
Maximum Transmission Unit.
MTU is 1500 Bytes default in Cisco equipment between sites. Avaya recommends MIN 300, MAX 550 bytes. At XXXX they lowered it from 1500 to 1400 and we got dial tone.
Hudson22

 

[mattKnight]

I'd be looking at the the ARP cache on the switch.

When you have a phone in this no media state, check that the IP and MAC entry match (or simply clear the arp cache)

Take Care

Matt
I have always wished that my computer would be as easy to use as my telephone.
My wish has come true. I no longer know how to use my telephone.

 

[vulcanccit]

Thank you Matt and Hudson, I am having the network boys look into both of those. They did find a spanning tree issue on the switches which seems to have helped a lot, but still had a phone do it this AM...

 

[csdropout]

Not sure if my experience is related to this, but I had a similar issue when I had an IP address conflict on my G450 gateway. I had dial tone on my phone, but then I would lose dial tone for some time. Back and forth repeatedly until I found out the IP conflict. Hope this helps.

 

[rejackson]

You say that they can still complete a call even when they dont hear dialtone and then they cannot hear the person they called. That tells me the media resource being used cannot reach the phone over the network. It sounds like the network changes have isolated a media resource somewhere in your network that the core can reach to assign it to the call but it (the media resource) cannot reach the phone.

If the medres is at another location it could be a routing issue if the new router is not advertising the voice ip network to the location. If the phones and medres are at the same location it could be a problem between switches. Are the phones that have the problem on a different switch than the MG? That probably means a problem in the trunk (if there is one) between the switches. Get into the local Media Gateway and see if it can ping some of the IP addresses on the phones that are having problems. That will tell you if it can reach them.

 

[vulcanccit]

The MG is local to the site... the sip trunk is plugged into the core router, as well as the data T1. Router, is plugged into the core switch. I have not been personally to this site, but I think there is only one switch...there are only about 15 people in this office. My understanding from the BP is the Dialtone should be coming from the gateway locally at that site...only call setup goes over the sip trunk (routed back to the core). Our network monitoring tool has revealed that even station to station calls have some jitter, and loss at the first part of the call like about 10 seconds into the call. It is almost as if the switch is waking up and going oh wow, I have a call...what do I do? My issue is our network guys are slammed as they have outsourced most of the wan folks... but I have one of the most senior guys looking at it... I might see if we can put the old cisco switches back in... at least pick like 4 or so of the phones that have had issues...put them on the old switch and cross it over to the core and see if they are stable...maybe the new cisco switch has an issue... it is a 2960.

 

[rejackson]

I cannot say where the dial tone comes from but if these are LSP sites then the phones register to the core not the local MG/LSP. The core registers the off hook state and assigns some resource to send dial tone. When Dial tone should be presented the system doesnt know what you are going to dial so the SIP trunk is not involved. They could be checking voicemail or dialing internally. Like you said when they use the core DSPs they work, when they use the local they do not. I still say you should remote to the MG and see if it can ping the IP addresses of the phones.

 

[vulcanccit]

Going off the theory from rejackson, I looked at the DHCP server for that market... I found out no IPs were getting handed out. When they swapped the router, they never put in a DHCP helper pointing to the DHCP server. Once they put that in, all the phones got an ip from DHCP. I suspect the phones were holding on to their leases, and then would try and renew and could not, probably causing the hiccup... I had the market do the CRAFT Clear command last night and so far today all is well. I will see how it goes over the next day or so and update this thread.

 

[mattKnight]

Good catch!

By default, i believe that the handset will continue to use tbe leased address past tje expiry period if it can't renew! This is a break with the dhcp standard...

You can alter the behaviour by changing the DHCPSTD to 1 in the 46xxsettings file or dhcp option 242. All our phones are set like this and we had similar case a few weeks ago that was immwdiately clear what was going on as the phone failed to get an ip address

A sneaky problem you had is now solved!

Take Care

Matt
I have always wished that my computer would be as easy to use as my telephone.
My wish has come true. I no longer know how to use my telephone.

 

[vulcanccit]

Matt what happens when you change that setting? The site reported all was well today. I want to see how they do tomorrow then I will change that setting based on your response, but that setting will affect all of our sites since they register to the core. I suspect that the settings change will may the phone unregister, and sit at the DHCP count down to ip address???

 

[mattKnight]

what happens when you change that setting?

I suspect that the settings change will may the phone unregister, and sit at the DHCP count down to ip address???

Depending on how you deploy it, I believe this is a soft change i.e. it won't require a handset reboot. Obviously a reboot is required to force the handset to retrieve the updated file, but if you use DHCP scope you won't need to.

################# DHCP ADDRESS SETTINGS ##################
##
## DHCPSTD controls whether the phone continues to use an
## expired IP address if the phone received no response to
## its address renewal request. 0 for yes, 1 for no.
##
SET DHCPSTD 1

Shouldn't do, but I can't guarantee it...

Take Care

Matt
I have always wished that my computer would be as easy to use as my telephone.
My wish has come true. I no longer know how to use my telephone.

 

[vulcanccit]

Just letting you all know that we have gone 4 days with perfect performance since I caught the issue with the DHCP Helper not being present on the router. I hope this experience helps others that might have the same issue in the future. Thank you all for helping!