Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations strongm on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

96xx Voice Quality

Status
Not open for further replies.

bmast

IS-IT--Management
Jun 21, 2010
35
US
Thank you in advance for any help you can offer. I have been troubleshooting voice quality on a new IPO 500V2 8.0.42 installation and am currently stumped.

The installation consists of approximately 58 IP extensions; 9650, 1603, and 3725 DECT phones. The IP Telephones are on a dedicated (physically separate) PoE data infrastructure. LAN 1 of the IP Office serves as DHCP server and is connected to the PoE switch stack, LAN 2 is on the customer data network and connected to the data switch stack. There is no connection between the VoIP and Data network (I have checked the MAC tables in the VoIP switches) except through the IP Office.

I have been getting QoS Voice Quality alarms and degraded voice for several weeks now. A reset of the IP office and/or switch stack seems to improve the situation for a period of time, but the quality soon degrades again. I have tested the following:
1) Verified spanning tree and root bridge configuration is the primary switch, verified that no switching loops were detected. Physically inspected cables to verify no broadcast storm.
2) Viewed statistics in the switches to verify that most traffic is unicast, and that there are not broadcast storms or errored packets.
3) Mirrored the LAN1 port on the IP Office and collected flow statistics using a diagnostic appliance. This revealed some interesting and seemingly inexplicable information:
* Several of the RTP ports 49152, 49154, 49158, etc seemed to remain open and continue to generate traffic roughly equivalent to a g.711 media stream despite all devices in the system being idle (even at 2 am on a sunday).
* This behaviour cleared somewhat when the IPO was rebooted, but quickly re-established itself after a few hours in service.
* I reset the entire switch stack and IP office, and the behaviour seems to have cleared (for the last few hours)
4) Tried to run a VoIP assesment against the installed VoIP terminals. I get a 90% packet loss to ICMP pings to all of the 9650 and 1603 terminals but not to the DECT repeaters or IPO LAN1 port...
* are the phones designed to ignore ICMP? Or is this a hint?
 
No, phones respond to ICMP, you can even ping them via SSA, what does SSA show for the in call stats like jitter packet loss etc ?

 
try to disable the rtcp monitoring under lan1 (or lan2)-> voip
This shows the qos alarms.
Perhaps it is related.


BAZINGA!

I'm not insane, my mother had me tested!

 
The customer has 60 telephones on site and is an active surgery center. the SSA application displays:
* 3 instances of packet loss, all on inbound calls to 3 separate 9650 IP Telephones
* 8 instances of jitter, from 220 ms all the way up to 6,890 ms... (that is what i call jitter) to a mixture of 1603 and 9650 ip phones on outbound calls.

No other alarms are reported.

As for disabling the RTCP monitoring? Do you mean enable or disable... will that just prevent them from being logged?


 
Disable it.


BAZINGA!

I'm not insane, my mother had me tested!

 
I don't mean the alarms, I mean the live stats while watching an extn on call, QoS alarms are just plain bollocks and can be ignored :)

 
OK, RTCP monitor is turned off. i will ignore the QoS alarms and just ask for customer feedback...
 
This will stop giving alarms.
But did they actually had bad calls?


BAZINGA!

I'm not insane, my mother had me tested!

 
YES, they have had a lot of bad 'barely understandable' bad calls. The garbled voice happens on all three handset types and appears random, but the frequency and severity of the issue increases over time (as well as the alarms) until the system is reset. The first time we reset the equipment, the problme cleared for two weeks, the second time for a couple days, the most recent time for only a few hours. Last Friday AM, I reset the switches as well as the IP Office. We have not had complaints since that time, but today will help asses whether the issue is truly improved.
 
So what is the POE switch stack for the phones. (What model/equipment is being used?)
 
Damned good question! When handed the issue by the tech, I failed to ask that questions... which is embarrassing since I know better. I checked with the customer and they said that although it is predominantly on outside calls they have heard 'bad dial tone' when making a call. This customer still has a DSS span with dial tone provided by the carrier, so this doesn't definitively rule out a T1 span problem. The primary reason my thoughts have not wandered in that direction is that this is a re-use of a span in service for 10 years, and there were no T1 alarms in the system monitor. If I get more trouble reports I will get out there with a test set to rule this out...
 
PoE switch stack is:
1st Floor 2 x Dell 3524P configured in a stack w/ multimode fiber to
2nd Floor 1 x Dell 3524P

1st floor stack is STP root

Switches have been given management IPs and passwords, Telnet and Http enabled.

No VLANs configured (except default)

No additional configuration.
 
I would start by buildin a voice vlan!!!
IP phones are great but without a proper network it will fail big time.



BAZINGA!

I'm not insane, my mother had me tested!

 
tlpeter

The only thing on the phone network is VoIP, the data network is both logically and physically separated. We have dedicated switches for the IP Phones.
 
Hi guys,

Same problem here, Seem to be a VCM/IPO Problem

I’m currently working to give appropriate monitor trace and PCAP with problematic calls but it's not done yet

We have delay/no speech path issue were the caller don’t here the called person during 7 second but also echo and packet lost

For anybody who told to check the network, I can reproduce it in my lab with 2 IPO, 2 9650 and 1 switch.... No problem with 1608 or 1616

Jimmy Cormier
Quebec City, Canada
Sorry, I’m french !
20xBCM/SRG,CS1000,7xIP Office,4xVmpro,
3x contactstore,Call Manager,AES,Nice Recording,Callsweet,IEX Totalview,Symposium,Callpilot,Kofax,
 
Hi Jimmy,

Can you try a different lan port on the IPO?
I have had issues with lan ports not being very good and stopped working when there was more then a certain amount of traffic on it.


BAZINGA!

I'm not insane, my mother had me tested!

 
Wow...
I was able to reproduce the problem put since I've hard reset the IPO and re-import the configuration, problem seem gone ... Hard reset has be made with 8.0

If someone can try that and let me know

Jimmy Cormier, Quebec City, Canada
Sorry, I’m french !
15xBCM/SRG,CS1000,23xIP Office,14xVmpro,
10x contactstore,Call Manager,AES,Nice Recording,Callsweet,IEX Totalview,Symposium,Callpilot,Kofax,
 
We hard reset (power down vs. using manager-reboot) and the issue has not re-occurred for the past week. I would really like to answer the 'why' but if the fix holds I will let it rest....
 
Hi bmast,

cool, same situation for me

Just to understand,
«We hard reset (power down vs. using manager-reboot)»
by hard reset you mean push the reset button until the CPU light turn red and start flashing ?

Thank you

Jimmy Cormier, Quebec City, Canada
Sorry, I’m french !
15xBCM/SRG,CS1000,23xIP Office,14xVmpro,
10x contactstore,Call Manager,AES,Nice Recording,Callsweet,IEX Totalview,Symposium,Callpilot,Kofax,
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top