Minet delayed when ringing an IP set

[Telecomp9434]

We have had IP phones deployed through a network of 200 ICP's (5-ICP's) for about 2 years. We were using HP 2620 POE switches with VLAN 180. Everything has been running fine for about two years. The customer replaced all swicthes with Cisco 3750 POE 4 months ago. 2 sites now have constant issues where an extension dials another extension (local ICP) the set rings but they cannot be connected. When the user picks up the handset it keeps ringing. This usually last 10-12 seconds. We have pulled numerous logs with Mitel support, pulled wireshark cpatures, I even hired a Cisco sub contractor to help us trouble shoot. I am pretty sure it is the Cisco switches, I just cannot prove it. The logs indicate IP network congestion on the IP trunks everytime the user has an issue, however the user is not using an IP trunk, these are local ext calls. The customer has a feature call Port security on the switches. Does anyone know what that is and if that may have any effect on the Mitel phones? The phones are all 5212 and 5224.

Thanks-

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[AlphaMann]

You are probably right that it is the Switch. Many manufactures have some sort of "SIP helper" feature. This usually does not work with Mitel and can cause exactly what you are describing to happen. I am not 100% on the Cisco but did a quick search and it may have a feature called NAT SIP ALG. Try to disable it. What or is there something that is doing VLAN translation? (is this device the same?)

 

[slapin]

Looking at the problem description it looks like packet loss.
MiNET is event/action protocol with all the logic driven from the controller. It means when the phone is ringing, it was commanded to do so from the controller. When you pickup the handset, your phone will send an event and the controller will send a command to stop the ringer and setup appropriate audio channel.

In this case the event message is not reaching the controller.

Mentioned port security will not allow the phone to maintain the heartbeat and it will be out of service, so it is not the issue.

I would check speed and duplex. It needs to be set on auto. Also check L2 QoS. I would recommend to disable it for now. Also if the phone is connected via trunk port and using "Voice" VLAN and the computer behind it using a native VLAN, I would try to switch it into access mode and configure ICP IP address as a teleworker. This may give you some ideas what can be wrong.

P.S. I was running 3750 switches for years without any problems... Try to simplify the configuration to the bare minimum and then add features including security and QoS

 

[Telecomp9434]

Thanks slapin, I will reach out to the IT guy and ask him to make those changes. I pull more logs today and wiresahrk captures. The users complained more last friday (5/17). Same problem. (IP network congestion on the IP trunks). When that happens the local IP phones expierence this issue.

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[slapin]

You say IP trunks. Do you refer to 802.1q trunks where ICP is located in the same location as IP phones or IP trunks in terms of ICP or IP trunks as a connection from one location to another?

 

[Telecomp9434]

I should have explained that better...
The issue is happening between two internal extensions (x102 calls x103..etc).
However this is a 5 site network, so usisng the MPLS we can dial across to another site with a code like 81+xxx (xxx being the extension number)
When we pull the wire shark, there seems to be alot of network congestion on the WAN at the same time the end user is compalining. Somehow this congestion is causing the Minet signal to get lost (That is my assumption).

Having congestion on the WAn is a whole separate issue, I know. But what is getting to me is why it is affecting my voice LAN.

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[slapin]

Still not clear where the ICP (The mitel box) is located relatively to IP phones.

Phones will stream voice packets directly, but management and control will go through the ICP. So if the controller is reachable via MPLS, then management traffic will be affected and you may observe all sorts of side effects. As I said MiNET protocol is event/action based, so every key pressed or any character displayed is an event sent to the controller and action sent from the controller. You need to have low latency in order to see good performance. In contrast to SIP protocol where the protocol logic is implemented locally on the end device, you may have some delay, but for example the ringer will stop as soon as you pick up the handset, as nothing needs to be sent from the SIP proxy server for that.

 

[Telecomp9434]

The Mitel is located on the same LAN as the IP sets. The Mitel is 10.185.9.10/ the phone subnet is 10.185.10.20 - 200.

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[Telecomp9434]

Correction the IP phones are 10.185.9.20 - 200...I fat fingered the IP I mentioned above (10.185.10.20)

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[slapin]

Then it becomes interesting. You say when you see a lot of traffic then it is kind of slow. In this case I would look for CPU load of the switch. Could be brodcast storm or loops. Try to establish continuous ping from a PC in the same segment to the controller and capture it in the wireshark on the same PC, so the data volume should be manageable. Capture at least one period when the issue is being observed and see if anything funny was captured and what was the RTT at that time.

 

[SXWizard]

Make sure that the Cisco port that the ICP is connected to is set as an access port for VLAN 180 and not a trunk port.

 

[jrgood0852]

Under tools and features, try disabling CDP (Cisco Discovery Protocol) as I have seen this cause issues before.

 

[Telecomp9434]

I just learned that CDP (Cisco Discovery Protocol) is enabled. IT vendor is stating that it has to be enabled for his Cisco switches. Could this be the issue?

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[slapin]

Actually CDP is used to identify L2 ports for emergency calls location and phones tracking, so it is at least accounted in the design and should not cause any issues in any average network.

I have seen when 3750 will go crazy if portfast is enabled (STP disabled) and someone will plug a simple unmanaged switch into two ports at the same time (attempting to get more bandwidth...), which will create a forwarding loop and drive CPU utilization through the roof.

 

[Telecomp9434]

Thanks Slapin....when I look at the 3750....the lights are flickering so fast (out of the ordinary if you ask me). Would that affect the Minet signaling? If so, should I enable STP?

http://www.Telecomp-inc.com

"Voice and Data Solutions

 

[sarond]

A loop would definitely cause things to go sour, I have had a site brought to its knees because somebody decided to connect the PC port of a phone back in to another network outlet causing a loop. Phones were rebooting and the network basically halted.

When you say the customer replaced their switches do they have sufficient understanding of the network and cabling infrastructure, they may have caused an issue by connecting something wrong or missed something in their switch config.

 

[slapin]

Yep, that is exactly what I have observed. Enabling STP at least for now will help you to find the loop as it will put some of the ports into the blocking state. The trade off is delayed startup until STP will resolve the topology behind the port. Since phones are rarely disconnected/re-connected it should not be a problem.