Poor audio?

[fonejack]

I’m have voice quality issues between two ICP3300 running 4.0sp1.
Calls are iptrunked through e2t using g729 over a MPLS WAN

Complaints are random garbled audio, call quality will be good and then one end will get the garbled audio for a few seconds and then clear up. Everything points to a network problem but I can’t find it. Voice is running on its own Vlan and it is QOSed along with priority queuing.

I captured e2t stats real time during a period of bad audio via the shell command E2tshow and noticed that every time I experienced bad audio the jitter underrun counters climb exponentially. Below are the results that show that call with the elevating underruns. Has anyone interpreted these E2t results specifically what the underruns might point to? Any help is greatly appreciated.

C=194 G729 Rx=5192 20ms(cfg=20ms) TxOk=11686 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=70ms JB Ovr=0 Undr=1311 Depth 20ms Hi=270ms] NetDly=6.96ms 10.2.61.5[50092] DTMF_RFC crypted

-> E2tshow
-------------------------------- E2tProxySp Status --------------------
Local E2T has 64 Audio ports, and 128 T38 E2T Listening ports. Is Connected to E2tProxySp. Uses HDLC for transcoding.

E2T configured to use superchannel for transcoding.

MCC Channel Range (Audio) 192 - 255
Virtual MCC Channel Range (T38) 256 - 383
-------------------------------- E2T Stats ---
FRI MAY 07 12:52:41 2010


Mitel Dynamic Jitter Buffer
RTC Local Connected
TDM codec is U_LAW_CODEC -- North America
Total Calls Setup 891 (0 of which were/are T.38)
E2T crypto/clear/T.38 Channels active 7/0/0 (high 21/3/0)
DSP Transcoders currently active 7 (high 19)
Local Transcoders currently active 0 (high 0)
Pass: UDP port not in voice range 31
Info: Ch with multiple src addresses 0
Pass: ICMP_ECHOREQUEST 980
Pass: ICMP_REDIRECT 0
Pass: ICMP_UNREACH 10
Drop: ICMP_UNREACH_PORT UDP 0
Pass: ICMP_ECHOREPLY 878
Pass: ICMP_SRC_QUENCH 0
Pass: icmp_other 0
Drop: Invalid DTMF digit 0
Drop: Rx UDP audio on Closed Ch 0
Drop: wrong g711 codec 880
Drop: codec not g729 0
Drop: codec g729 bad length 0
Drop: codec g711 bad length 1
Drop: Rxed unexpected xcoded packets 0
Calls with Jitter > 300ms 0
Calls with Packet Loss > 2% 1
RTP Sequence errors 650
RTP Seq total missing pkts 817
RTP Seq Packets out of order 0
RTP Seq False Packet Loss 0
RTP Seq False Out of Order 0
Host dst mod by icmp redirect 0
DTMF RTP Pkts Rx 0
DTMF digits received 0
DTMF no inter digit gap 0
DTMF added pkts for min len 0
DTMF overflow 0
DTMF Duplicate E=1 0
DTMF Digits without E=1 flag 0
DTMF pkts digit starve 0
zFree Errors 0
VQ Stats msgs dropped 0
EOS Detection (Active\Req\Reports\Det) 0\0\0\0
EOS event table overflow count 0
aggregate EED counters:
eedInitiated 891
eedSuccessful 872
eedAbandoned 3
eedFailed 16
eedDecryptErrorMustWaitForModify 0
eedUnknownPtWaitForModify 0
eedDataCallWaitForModify 0
eedDtfmWaitForModify 0
eedMustWaitForModify 16
eedControlFsmStateMismatch 0
eedVoiceFsmStateMismatch 0
decryptError 0
Found an E2tProxySp. Will run e2tproxyShow as well
undefined symbol: an
-> -------------------------------- E2tProxySp Status --------------------
undefined symbol: E2tProxySp
-> Local E2T has 64 Audio ports, and 128 T38 E2T Listening ports. Is Connected to E2tProxySp. Uses HDLC for transcoding.
undefined symbol: E2T
->
-> E2T configured to use superchannel for transcoding.
undefined symbol: configured
->
-> MCC Channel Range (Audio) 192 - 255
undefined symbol: Channel
-> Virtual MCC Channel encryptError 0
Range (T38) 256 - 383
undefined symbol: MCC
-> invalidE2tContext 0


Currently Active Channels

C=194 G729 Rx=5192 20ms(cfg=20ms) TxOk=11686 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=70ms JB Ovr=0 Undr=1311 Depth 20ms Hi=270ms] NetDly=6.96ms 10.2.61.5[50092] DTMF_RFC crypted
C=203 uLaw Rx=25941 20ms(cfg=20ms) TxOk=51882 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=9ms JB Ovr=0 Undr=13 Depth 20ms Hi=230ms] NetDly=11.10ms 10.15.32.10[50024] DTMF_RFC crypted
C=205 G729 Rx=1942 20ms(cfg=20ms) TxOk=3879 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=0ms JB Ovr=0 Undr=3 Depth 10ms Hi=230ms] NetDly=6.54ms 10.2.61.5[50104] DTMF_RFC crypted
C=229 G729 Rx=98293 20ms(cfg=20ms) TxOk=196580 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=0ms JB Ovr=0 Undr=120 Depth 20ms Hi=220ms] NetDly=6.43ms 10.2.61.5[50026] DTMF_RFC crypted
C=233 G729 Rx=11013 20ms(cfg=20ms) TxOk=22019 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=0ms JB Ovr=0 Undr=13 Depth 30ms Hi=790ms] NetDly=11.98ms 10.2.61.5[50056] DTMF_RFC crypted
C=242 G729 Rx=9674 20ms(cfg=20ms) TxOk=19346 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=0ms JB Ovr=0 Undr=16 Depth 10ms Hi=340ms] NetDly=6.79ms 10.2.61.5[50068] DTMF_RFC crypted
C=247 G729 Rx=8666 20ms(cfg=20ms) TxOk=17326 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=0ms JB Ovr=0 Undr=15 Depth 30ms Hi=260ms] NetDly=6.88ms 10.2.61.5[50078] DTMF_RFC crypted
C=253 G729 Rx=18835 20ms(cfg=20ms) TxOk=37668 TxErr=0, EOS=0 T38=0 [PktLoss=0 maxBurst=0 OoO=0] [MaxJtr=11ms JB Ovr=0 Undr=11 Depth 20ms Hi=230ms] NetDly=16.16ms 10.29.32.2[50000] DTMF_RFC crypted

-------------------------------- T2E Stats ---
FRI MAY 07 12:52:41 2010


Total T2E Calls Setup 1407 (0 of which were/are T38)
T2E crypto/clear/T.38 Channels active 8/0/0(high 26/3/0)
DSP Transcoders active 8 (high 26)
Local Transcoders active 0 (high 0)
T2E rx inactive ch 359
T2E cleanup counter 1
T2E rx invalid ch 0
T2E rx inactive super ch 0
T2E rx_fail 0
T2E MccAlloc fail 0
T2E MccFree fail 0
T2E Pkts Dropped during JB Tone 0
E2T Pkts Dropped during JB Tone 0
Eth Tx Fail out of resources 0
MccSendHdlcVoiceBuffer fails 0
DspSendVoiceBuffer fails 0
MccAddVoiceBufferToTxQ fails 0
MccAddVoiceBufferToBdRing fails 0
Invalid DTMF Inject state 0
DTMF Inject Start requests 0
DTMF Inject Stop requests 0
loopback channels 0
loopback inactive ch 0
DSP Transcoder Err 0
Last DSP Transcoder Hrd Err 0x0
Cipher Changes 0
Key Changes 1

Currently Active Channels

C=194 116sec rxOk=11666 txOk=5833 20ms txFail=0 udpPort=50004 Phone=10.2.61.5[50092] ip=G729 DTMF_RFC crypted
C=202 40sec rxOk=4002 txOk=2001 20ms txFail=0 udpPort=50020 Phone=10.29.32.2[50050] ip=G729 DTMF_RFC crypted
C=203 519sec rxOk=51960 txOk=25980 20ms txFail=0 udpPort=50022 Phone=10.15.32.10[50024] ip=uLaw DTMF_RFC crypted
C=205 39sec rxOk=3902 txOk=1951 20ms txFail=0 udpPort=50026 Phone=10.2.61.5[50104] ip=G729 DTMF_RFC crypted
C=229 1966sec rxOk=196605 txOk=98303 20ms txFail=0 udpPort=50074 Phone=10.2.61.5[50026] ip=G729 DTMF_RFC crypted
C=233 220sec rxOk=22017 txOk=11009 20ms txFail=0 udpPort=50082 Phone=10.2.61.5[50056] ip=G729 DTMF_RFC crypted
C=242 193sec rxOk=19331 txOk=9665 20ms txFail=0 udpPort=50100 Phone=10.2.61.5[50068] ip=G729 DTMF_RFC crypted
C=247 170sec rxOk=17018 txOk=8509 20ms txFail=0 udpPort=50110 Phone=10.2.61.5[50078] ip=G729 DTMF_RFC crypted
C=253 348sec rxOk=34898 txOk=17449 20ms txFail=0 udpPort=50122 Phone=10.29.32.2[50000] ip=G729 DTMF_RFC crypted

 

[IrwinMFletcher]

Best to have a wireshark trace to compare to. Either there really is jitter, or the E2T thinks there is. You need to know what the stream looks like to know for sure.

 

[fonejack]

My network engineer found the problem. Our headend ICP connected to our core switch was set to 1 Gbps full duplex, he change it to 100 Mbps full duplex and that solved the underrun problem. Audio quality is now clear.