Odd problem with service numbers.

[Telecorn]

It came to my attention recently that service number 811 could not be dialed from our Cisco phones (actually dialing 9-811). I tested it on our CS1K phones and it works fine (9-811). This got me thinking to test all other X11 numbers (except 911 until this is fixed) and the only one that can be called from out Cisco system is 9-411. The route pattern all of these match is the same 9.[2-8]11, so a user dials 9, any number from 2-8, then 11 and Cisco passes the entire dial string over to the PBX where our PSTN access is. When I checked the CS1K the service number programming was all over the place (2 were NXXs, 2 were SPNs, all using different RLIs). I changed them all to be SPNs using the same RLI as 9-411 and it made no difference. When you call 9-311, 9-611, or 9-811 from a Cisco phone it drops the calls almost immediately, 9-411 works fine, and they all work from CS1K phones.

I opened a case with Cisco and they were able to determine that the CS1K is sending incorrect cause codes with the Progress as well as a Disconnect message for the calls. Does anyone know what I can check in the CS1K to find out what is going on, or anything else I can try to see if the behavior changes?

PSTN <====> CS1K <====> Cisco UCM

I turned on logging for the D-channel but the good and bad calls look the same (other than the dialed number of course).

 

[tbonz25]

Can you attach the dchannel traces you pulled as there may be some information that will assist troubleshooting this issue thanks.

 

[Telecorn]

You bet, and they do look a bit different (my brain was getting log overdose that day). I highlighted the good one in green and the 2 bad ones in orange:

[highlight #4E9A06]DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000587 CH 4 19 TOD 16:35:45 CK 03920B36
CALLED #:9411 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_IMSG CC_SETUPCOMP_IND REF 00000587 CH 4 19 TOD 16:35:48 CK 039232E6

DCH 22 UIPE_IMSG CC_DISC_IND REF 00000587 CH 4 19 TOD 16:35:52 CK 03924A46
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_IMSG CC_RELEASE_CONF REF 00000587 CH 4 19 TOD 16:35:52 CK 03924A97[/highlight]

[highlight #F57900]DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000607 CH 4 19 TOD 16:35:54 CK 0392654F
CALLED #:9811 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 00000607 CH 4 19 TOD 16:35:56 CK 039267CD[/highlight]

[highlight #C17D11]DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000687 CH 4 19 TOD 16:36:04 CK 0392A568
CALLED #:9611 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 00000687 CH 4 19 TOD 16:36:04 CK 0392A7FA[/highlight]

Thanks for looking.

 

[tbonz25]

Can you activate both the incoming and outgoing dchannel messages you only have the incoming turned on in these traces. The command is the same just need to activate ENL MSGI XX (XX being the dchannel number) and ENL MSGO XX thanks.

 

[Telecorn]

Sure thing, here they are. There might be a couple of unrelated log messages as this is in production so I think there might be some user calls. I tried to clean them all out as best I could:

DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000336 CH 4 22 TOD 13:31:27 CK 8829E33C
CALLED #:9411 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 00008336 CH 4 22 TOD 13:31:27 CK 8829E33F

DCH 22 UIPE_OMSG CC_PROGRESS_REQ REF 00008336 CH 4 22 TOD 13:31:27 CK 8829E611
CAUSE: #127 - INTERNETWORKING UNSPECIFIED

PROGRESS: CALL IS NOT END TO END ISDN
PROGRESS: INTERWORKING WITH A PUBLIC NETWORK

DCH 22 UIPE_OMSG CC_SETUP_RESP REF 00008336 CH 4 22 TOD 13:31:31 CK 882A0CD2
PROGRESS: CALL IS NOT END TO END ISDN
PROGRESS: INTERWORKING WITH A PUBLIC NETWORK

DCH 22 UIPE_IMSG CC_SETUPCOMP_IND REF 00000336 CH 4 22 TOD 13:31:31 CK 882A0D11

DCH 22 UIPE_IMSG CC_DISC_IND REF 00000336 CH 4 22 TOD 13:31:33 CK 882A1ABF
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_OMSG CC_RELEASE_REQ REF 00008336 CH 4 22 TOD 13:31:33 CK 882A1AC0

DCH 22 UIPE_IMSG CC_RELEASE_CONF REF 00000336 CH 4 22 TOD 13:31:33 CK 882A1B1E

DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000436 CH 4 22 TOD 13:31:37 CK 882A38EE
CALLED #:9811 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 00008436 CH 4 22 TOD 13:31:37 CK 882A38F2

DCH 22 UIPE_OMSG CC_DISC_REQ REF 00008436 CH 4 22 TOD 13:31:37 CK 882A3AC3
CAUSE: #81 - INVALID CALL REF VALUE

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 00000436 CH 4 22 TOD 13:31:37 CK 882A3B30

DCH 22 UIPE_OMSG CC_RELEASE_RESP REF 00008436 CH 4 22 TOD 13:31:37 CK 882A3B31

DCH 22 UIPE_IMSG CC_SETUP_IND REF 000004B6 CH 4 22 TOD 13:31:47 CK 882A894F
CALLED #:9611 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 000084B6 CH 4 22 TOD 13:31:47 CK 882A8953

DCH 22 UIPE_OMSG CC_DISC_REQ REF 000084B6 CH 4 22 TOD 13:31:47 CK 882A8B46
CAUSE: #81 - INVALID CALL REF VALUE

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 000004B6 CH 4 22 TOD 13:31:47 CK 882A8BAA

DCH 22 UIPE_OMSG CC_RELEASE_RESP REF 000084B6 CH 4 22 TOD 13:31:47 CK 882A8BAA

DCH 22 UIPE_IMSG CC_SETUP_IND REF 00000536 CH 4 22 TOD 13:31:57 CK 882AD141
CALLED #:9311 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 00008536 CH 4 22 TOD 13:31:57 CK 882AD145

DCH 22 UIPE_OMSG CC_DISC_REQ REF 00008536 CH 4 22 TOD 13:31:57 CK 882AD937
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 00000536 CH 4 22 TOD 13:31:57 CK 882AD9A0

DCH 22 UIPE_OMSG CC_RELEASE_RESP REF 00008536 CH 4 22 TOD 13:31:57 CK 882AD9A1

DCH 22 UIPE_IMSG CC_DISC_IND REF 00007CB5 CH 4 23 TOD 13:32:21 CK 882B8FFC
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_OMSG CC_RELEASE_REQ REF 0000FCB5 CH 4 23 TOD 13:32:21 CK 882B8FFC

DCH 22 UIPE_IMSG CC_RELEASE_CONF REF 00007CB5 CH 4 23 TOD 13:32:21 CK 882B904C

DCH 22 UIPE_IMSG CC_DISC_IND REF 00007EB5 CH 4 16 TOD 13:32:33 CK 882BEC15
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_OMSG CC_RELEASE_REQ REF 0000FEB5 CH 4 16 TOD 13:32:33 CK 882BEC15

DCH 22 UIPE_IMSG CC_RELEASE_CONF REF 00007EB5 CH 4 16 TOD 13:32:33 CK 882BEC71

 

[tbonz25]

One last request you indicate that this call goes across a T-1 from Cisco to the CS1000 where it then goes to the PSTN (Carrier) can I get the dchannel trace for both the Tie line between the Cisco and CS100 and the carrier T-1 the call goes out on sorry for all the traces but I think this will lead to the issue. I see there is a CAUSE 81 in the trace you sent so the is an invalid call reference somewhere and need to see what the carrier sends back. Also if you could pull the trace for a good call i.e. 411 and a bad one thanks so much.

 

[Telecorn]

OK, I hope this helps you out, again apologies if other calls get mixed in a bit. DCH 22 is the tie between Cisco and CS1K, DCH 10 is between CS1K and PSTN. Good call to 9-411 is first, then 9-811 fails.

.enl msgi 22
OUTPUT TARGETS: TTY
.enl msgo 22
OUTPUT TARGETS: TTY
.enl msgi 10
OUTPUT TARGETS: TTY
.enl msgo 10
OUTPUT TARGETS: TTY
.
DCH 22 UIPE_OMSG CC_SETUP_RESP REF 0000EDBA CH 4 21 TOD 14:23:21 CK 92D5F0A4
PROGRESS: INTERWORKING WITH A PUBLIC NETWORK

DCH 22 UIPE_IMSG CC_SETUPCOMP_IND REF 00006DBA CH 4 21 TOD 14:23:21 CK 92D5F0EF

DCH 22 UIPE_IMSG CC_SETUP_IND REF 00006E3A CH 4 20 TOD 14:23:27 CK 92D62A83
CALLED #:9411 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 10 OMSG SETUP REF 0000005E CH 19 22 TOD 14:23:27 CK 92D62A86
HEADER:01 00 00 2D 08 01 5E 05
BCAP:04 03 80 90 A2
CHID:18 04 E9 83 83 16
DISP:28 0E B1 4D 69 63 68 61 65 6C 20 43 68 75
64 79
CLNG:6C 06 00 80 36 34 34 34
CLED:70 04 C1 34 31 31

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 0000EE3A CH 4 20 TOD 14:23:27 CK 92D62A86

DCH 10 IMSG CALLPROC REF 0000005E CH 19 22 TOD 14:23:27 CK 92D62B65

DCH 10 IMSG PROGRESS REF 0000005E CH 19 22 TOD 14:23:27 CK 92D62C86
CAUSE :INTERNETWORKING UNSPECIFIED
PROGRESS: CALL IS NOT END TO END ISDN

DCH 22 UIPE_OMSG CC_PROGRESS_REQ REF 0000EE3A CH 4 20 TOD 14:23:27 CK 92D62C86
CAUSE: #127 - INTERNETWORKING UNSPECIFIED

PROGRESS: CALL IS NOT END TO END ISDN
PROGRESS: INTERWORKING WITH A PUBLIC NETWORK

DCH 10 IMSG DISC REF 0000964A CH 2 1 TOD 14:23:29 CK 92D631A5
CAUSE :NORMAL CALL CLEARING

DCH 10 OMSG RELEASE REF 0000964A CH 2 1 TOD 14:23:29 CK 92D631A6
HEADER:01 00 00 05 08 02 96 4A 4D

DCH 10 IMSG REL COMP REF 0000964A CH 2 1 TOD 14:23:29 CK 92D631D8

DCH 10 IMSG CONNECT REF 0000005E CH 19 22 TOD 14:23:33 CK 92D65276

DCH 10 OMSG CONN ACK REF 0000005E CH 19 22 TOD 14:23:33 CK 92D65277
HEADER:01 00 00 04 08 01 5E 0F

DCH 22 UIPE_OMSG CC_SETUP_RESP REF 0000EE3A CH 4 20 TOD 14:23:33 CK 92D65278
PROGRESS: CALL IS NOT END TO END ISDN
PROGRESS: INTERWORKING WITH A PUBLIC NETWORK

DCH 22 UIPE_IMSG CC_SETUPCOMP_IND REF 00006E3A CH 4 20 TOD 14:23:33 CK 92D652B8

DCH 10 OMSG CONNECT REF 0000964C CH 2 4 TOD 14:23:35 CK 92D6626C
HEADER:01 00 00 05 08 02 96 4C 07

DCH 10 IMSG CONN ACK REF 0000964C CH 2 4 TOD 14:23:35 CK 92D662BA

DCH 22 UIPE_IMSG CC_DISC_IND REF 00006E3A CH 4 20 TOD 14:23:35 CK 92D664DC
CAUSE: #16 - NORMAL CALL CLEARING

DCH 22 UIPE_OMSG CC_RELEASE_REQ REF 0000EE3A CH 4 20 TOD 14:23:35 CK 92D664DD

DCH 10 OMSG DISC REF 0000005E CH 19 22 TOD 14:23:35 CK 92D664DE
HEADER:01 00 00 08 08 01 5E 45
CSE :08 02 81 90

DCH 10 IMSG RELEASE REF 0000005E CH 19 22 TOD 14:23:35 CK 92D66535

DCH 10 OMSG REL COMP REF 0000005E CH 19 22 TOD 14:23:35 CK 92D66536
HEADER:01 00 00 04 08 01 5E 5A

DCH 22 UIPE_IMSG CC_RELEASE_CONF REF 00006E3A CH 4 20 TOD 14:23:35 CK 92D66537

DCH 22 UIPE_IMSG CC_SETUP_IND REF 00006EBA CH 4 20 TOD 14:23:39 CK 92D6817C

CALLED #:9811 NUM PLAN: UNKNOWN TON: UNKNOWN
CALLING #:6444 NUM PLAN: UNKNOWN TON: NATL

DCH 10 OMSG SETUP REF 0000005E CH 19 22 TOD 14:23:39 CK 92D6817F
10HEADER:01 00 00 2D 08 01 5E 05
BCAP:04 03 80 90 A2
CHID:18 04 E9 83 83 16

DISP:28 0E B1 4D 69 63 68 61 65 6C 20 43 68 75 64 79
CLNG:6C 06 00 80 36 34 34 34
CLED:70 04 A1 38 31 31

DCH 22 UIPE_OMSG CC_PROCEED_REQ REF 0000EEBA CH 4 20 TOD 14:23:39 CK 92D68180

DCH 10 IMSG CALLPROC REF 0000005E CH 19 22 TOD 14:23:39 CK 92D68298

DCH 10 IMSG DISC REF 0000005E CH 19 22 TOD 14:23:39 CK 92D683B7
CAUSE :INVALID CALL REF VALUE

DCH 22 UIPE_OMSG CC_DISC_REQ REF 0000EEBA CH 4 20 TOD 14:23:39 CK 92D683B8
CAUSE: #81 - INVALID CALL REF VALUE

DCH 10 OMSG RELEASE REF 0000005E CH 19 22 TOD 14:23:39 CK 92D683B9
HEADER:01 00 00 04 08 01 5E 4D

DCH 10 IMSG REL COMP REF 0000005E CH 19 22 TOD 14:23:39 CK 92D68412

DCH 22 UIPE_IMSG CC_RELEASE_IND REF 00006EBA CH 4 20 TOD 14:23:39 CK 92D68414

DCH 22 UIPE_OMSG CC_RELEASE_RESP REF 0000EEBA CH 4 20 TOD 14:23:39 CK 92D68414

 

[Telecorn]

I'm going to be away for a couple of weeks now so I'll check back after that.

 

[Telecorn]

I don't know if this helps but it appears the issue also presents itself when an Avaya phone is call forwarded to an external local area number and someone calls that Avaya phone from a Cisco phone, but again the working and failing numbers are all over the board.

Works
95872223456
94032233333
94035555555
94036666666
94037777777

Fails
94032333333
94032849831
94036452222
94036453285
94033333333
94034444444
94038888888
94039999999