ISDN PRI Calls periodically drop - Cause=41

[adc110]

I have an IP Office V2 running 11.0.4. It has (2) full PRI trunks (23-channel) in it with 2 separate PRI cards. Over the past few months, users mentioned that they were dropping calles (very infrequently as it appears from the log files that they maybe get 8-10 drop outs with an average call volume of 500 calls a day (total of inbound/outbound). Users never reported it as it wasn't that frequent and nobody seemed to know what the issue is. I setup logging and determined that when these events happen (logged within Chronicall with call recording) the monitor logs show a Cause=41 code. I do have Trunk 5 PRI card set as Network for time and Trunk 9 PRI card set as Fallback (just as an FYI). Here is a quick snapshot of the logs showing this event but have 2 questions:

1. The log snapshot below shows that this dropped call is Line 5, Channel 3 but later shows the BCahnnel as Slot 21 and Channel 43 which baffles me and not quite sure what slot and channel this is referring to?? How do you get a slot 21 and Channel 43?

2. I firmly believe this to be a Carrier issue but system monitor and logs do not show any errors or warnings at all (no clock clips, alarms, etc.) and only the sysmon logs actuall show the Cause=41. Users report a brief white noise and then call drops (we can actually hear this noise within Chronicall when playing back the recorded conversation). This even happens on both outbound and inbound calls. Carrier was involved but says they see not errors and circuits running clean. Problem is that out of 500 calls a day and only 8-10 times is making issue tough to troubleshoot. Any thoughts on pressing back on carrier?

Thanks in advance for any advice on this one!

10:05:22 10843882mS CMLineRx: v=5
CMReleaseComp
Line: type=Q931Line 5 Call: lid=5 id=3 in=1
BChan: slot=0 chan=3
Cause=41, Temporary failure
10:05:22 10843882mS CMCallEvt: 0ac00e1000000600 5.3.1 149 Q931 Trunk:5 CHAN=3: StateChange: END=A CMCSConnected->CMCSCompleted
10:05:22 10843884mS CMLOGGING: CALL:2020/01/1010:04,00:01:18,000,9293359000,I,CALLROUTEVM-AA,6100,SRSMITH,b,,0,,"",0,n/a
10:05:22 10843884mS CD: CALL: 5.3.1 BState=Connected Cut=2 Music=0.0 Aend="Line 5" (3.4) Bend="CALLROUTEVM-AA(CALLROUTEVM-AA)" [VoiceMail] (21.43) CalledNum=CALLROUTEVM-AA () CallingNum=8665551212 (SRSMITH) Internal=0 Time=78141 AState=Idle
10:05:22 10843885mS CD: CALL: 5.3.1 Deleted
10:05:22 10843886mS CMCallEvt: 0ac00e1000000600 5.3.1 -1 Q931 Trunk:5 CHAN=3: StateChange: END=X CMCSCompleted->CMCSDelete
10:05:22 10843886mS CMExtnEvt: RAS: CALL LOST (CMCauseTemporaryFailure)
10:05:22 10843887mS CMCallEvt: 0ac00e1000000602 0.1538.0 -1 RAS.0: StateChange: END=X CMCSConnected->CMCSCompleted
10:05:22 10843887mS CMExtnEvt: v=1013 State, new=PortRecoverDelay old=Connected,0,0,RAS
10:05:22 10843887mS CMExtnTx: v=RAS, p1=0
CMReleaseComp
Line: type=RAS 1 Call: lid=0 id=1013 in=0
Called[CALLROUTEVM-AA] Type=Voicemail (102) Reason=CMDRX_Attendant Calling[8665551212] Type=Unknown Plan=ISDN Pres=Allowed (0)
BChan: slot=21 chan=43
Cause=41, Temporary failure
10:05:22 10843888mS CMMap: a=21.43 b=0.0 T0
10:05:22 10843895mS CMExtnEvt: RAS: CMExtnHandler::SetCurrent( id: 1538->0 )
10:05:22 10843902mS CMCallEvt: 0ac00e1000000602 0.1538.0 -1 RAS.-1: StateChange: END=X CMCSCompleted->CMCSDelete
10:05:22 10843902mS CMExtnEvt: v=1013 State, new=Idle old=PortRecoverDelay,0,0,RAS
10:05:22 10843903mS CMExtnTx: v=RAS, p1=0
CMReleaseComp
Line: type=RAS 1 Call: lid=0 id=1013 in=0
BChan: slot=21 chan=43
Cause=41, Temporary failure

 

[kyle555]

Who sent the release? I don't read monitor traces very often.

And is there any SIP on your end of things like SIP sets or something?

I figure if you sent the release, it's on you to explore a bit earlier in that monitor trace to get an idea why and if the carrier sent it to you, you should push them as to why.

I see your call dropped at 10h05.22. Any idea when that call started, or how long they are usually up before they disconnect? Is there anything consistent around how long the calls were up at the time they dropped?

 

[meyer1y2k]

Unfortunately, the description for a cause 41 is vague:
Cause No. 41 - temporary failure [Q.850]
This cause indicates that the network is not functioning correctly and that the condition is no likely to last a long period of time; e.g., the user may wish to try another call attempt almost immediately.

What is interesting is that you report hearing white noise occasionally (static) on calls. On a clean running PRI, you shouldn't hear static.

How are the PRIs delivered? Is there an IAD between the PBX and the carrier? Are there T1 circuits or is this some type of fiber delivery with SIP to PRI conversion going on? To best answer this, a better understanding of the physical setup would be needed. This sounds to me like it is a physical issue or a clocking problem of some sort.

If the PRIs are delivered via T1 and connected directly to the IP Office, you can use the System Status App to see if there is a history of alarms and/or errors on the PRI connections.

I would also insist upon the carrier running a 24 hour monitor. Many times, carriers will just monitor for 15 minutes which may not capture any intermittent physical problems on the "last mile" (if there is a last mile).

 

[rdoubrava]

There is a temporary failure at the physical layer on the ISDN network

The only thing you can do is replace the cable between the IP Office PRI.

This is a carrier issue