SIP registration has failed, setting backoff retry timer to 30/60/120 secs

[trinetintl]

I have a customer with an IPO 500 V2 11.1.2.1.0 Build 3 that has been in production for over a year now working beautifully. The system started having intermittent issues with the SIP trunk randomly losing registration and going out of service. The provider can be pinged OK and so far the only option to allow the system to register is to reboot the controller unit. This is blowing my mind and the fact that it is random is making the issue almost impossible to troubleshoot. I tried upgrading to the latest firmware release to see if by chance and a hopeful miracle the problem got fixed but I was not lucky. I am seriously contemplating downgrading to whatever as long as it works. Does anybody out there have any ideas I can try to help me figure out what is going on with this system? Any help will be greatly appreciated!!

Here is a small snip of the log:

15:16:56 522682933mS PRN: SIP Line (17) parameter userid0001 registration has failed, setting backoff retry timer to 30 secs
15:16:56 522682941mS PRN: Begin Stack Trace, Task=CM taskaddr=f53ea6c4 stkbot=f53deb30 stackptr=f53ea430(640) stacksize=48000
15:16:56 522682941mS PRN: findfunc f026420c f0c02f50 f0c17194 f0c14470 f0c144ac f0c27ef0 f0c6d910 f0c9fdd8
15:16:56 522682941mS PRN: findfunc f027125c f0343704 f0265a54 f0265a08
15:16:56 522682941mS PRN: offsets f53ea430(640) f53ea460 f53ea460(592) f53ea480 f53ea480(560) f53ea4c0 f53ea4c0(496) f53ea4f0 f53ea4f0(448) f53ea500 f53ea500(432) f53ea580 f53ea580(304) f53ea5b0 f53ea5b0(256) f53ea5c0
15:16:56 522682941mS PRN: offsets f53ea5c0(240) f53ea600 f53ea600(176) f53ea650 f53ea650(96) f53ea660 f53ea660(80) aaaaaaaa
15:16:56 522682941mS PRN: End Stack Trace

RE

http://www.trinet-intl.com

APSS, AIPS

 

[Okkie26]

Enable sip verbose logging in sys monitor.

We need more info about why the registration fails.

 

[trinetintl]

Thank you, I turned it on, now the only thing I can do is wait until it happens again.

RE

http://www.trinet-intl.com

APSS, AIPS

 

[teletechman]

Is this Comcast in the US? I just had a similiar issue and come to find out Comcast had an issue and they needed to fix it. It was a known issue that they did not let us know about while I tried to trace it.
Mike

 

[trinetintl]

No, it is with Intermedia in the US. The customer called me a few minutes ago to tell me that the SIP trunk registration apparently went down but it came back up on its own. Unfortunately, I turned on the verbose after that incident.

RE

http://www.trinet-intl.com

APSS, AIPS

 

[trinetintl]

Trunk just crashed about 30 minutes ago, this is the initial registration failure, I read it, and it seems the IPO loses contact with the SIP provider, but besides that, I cannot see a good reason as to why or what's causing the failure. When we went in when it was first reported, we ran tests and while the system was down, system status could successfully ping the IP of the SIP provider.

168872225mS Sip: sip_indicateTimeOut Timer 6
168872225mS Sip: Timer 6 callback found dialog f18bf378 367be64faf929672fced78874faf052a REGISTER SIPDialog::INITIAL
168872225mS Sip: Completed ... (todelete) removing/preserving Dialog f18bf378 of CallId: 367be64faf929672fced78874faf052a and State: SIPDialog::INITIAL(0)
168872226mS Sip: SIP Line (17) RegistrationResult result s=2 interval=0
168872226mS PRN: SIP Line (17) parameter userid0001 registration has failed, setting backoff retry timer to 30 secs
168872226mS Sip: SIP Line (17) ActionOnTemporaryFailure 65.29.114.11 retry after 30000 flag 2 number of proxies 1
168872226mS Sip: SIP Line (17) ActionOnTemporaryFailure found index 0
168872226mS Sip: SIP Line (17) ActionOnTemporaryFailure marked a proxy as failed, there are available proxies 0
168872226mS Sip: SIP Line (17) check whether to put the trunk oos..., IsRegisteredIfRequired 0 active 1
168872226mS Sip: SIP Line (17) SetTrunkInService 0 (previous: 1, from_reg: 1, response code: 0 from_stop_trunk 0) begin checks...
168872226mS Sip: SIP Line (17) SetTrunkInServiceVariable to false
168872232mS PRN: Begin Stack Trace, Task=CM taskaddr=f53ec134 stkbot=f53e05a0 stackptr=f53ebea0(640) stacksize=48000
168872232mS PRN: findfunc f026420c f0c02f50 f0c17194 f0c14470 f0c144ac f0c27ef0 f0c6d910 f0c9fdd8
168872232mS PRN: findfunc f027125c f0343704 f0265a54 f0265a08
168872232mS PRN: offsets f53ebea0(640) f53ebed0 f53ebed0(592) f53ebef0 f53ebef0(560) f53ebf30 f53ebf30(496) f53ebf60 f53ebf60(448) f53ebf70 f53ebf70(432) f53ebff0 f53ebff0(304) f53ec020 f53ec020(256) f53ec030
168872232mS PRN: offsets f53ec030(240) f53ec070 f53ec070(176) f53ec0c0 f53ec0c0(96) f53ec0d0 f53ec0d0(80) aaaaaaaa
168872232mS PRN: End Stack Trace
168872232mS Sip: SIP Line (17): SetTrunkInService to OOSERVICE
168872232mS Sip: SIP Line (17) RestartTrunkComm now 0 dns 1 from_trunk_in_service 1
168872232mS Sip: SIP Line (17): StartTrunkComm enabled 1 in_service 0 from_startup 0
168872233mS Sip: SIP Line (17): StartTrunkComm dns_ok 1 OptionsNeededForKeepAlive 0 RequiresRegistration 1 check_oos 1


RE

http://www.trinet-intl.com

APSS, AIPS

 

[sizbut]

You could try turning off the SIP lines "Check OOS" settings and see what happens.

The risk with that is the system will still try to use the line when its connection to the ISP really is out of service. Also, normally the regular Options messages sent by the "Check OOS" function help keep connections through firewalls, etc open. However it looks like that's not helping in this case so no lose to try switching it off.

Stuck in a never ending cycle of file copying.

 

[critchey]

Do you have the binding refresh set in network topology? Do you have keepalives setup under the VOIP tab for the LAN1/LAN2 interface?

Something else Ive seen in a somewhat similar situation is under the credentials reduce the time in between registrations. It starts at 60 minutes, if it typically craps out after 30 minutes, try say 20 minutes on the registration.

The truth is just an excuse for lack of imagination.

 

[trinetintl]

No on binding refresh, No under Keepalive, registration is indeed at 60 minutes. The reason I have not dived deeper into the configuration is that we have another customer running almost an identical setup both running one of the newest models of an IPO 500 V2 on one of the flavors of 11.1, both are running identical configurations and Intermedia SIP trunks. The only difference is that the one having the problem is using Spectrum fiber and has a CISCO ASA firewall managed by Spectrum, we cannot go in to check anything, and the second customer is using fiber from a local carrier and we manage their Meraki firewall.

Besides the recommendations that you guys provided, will it make any difference if I use a different port instead of the 5060, what do you think? This is something that is recommended on several Avaya documents and application notes for configuring Intermedia SIP Trunk. Honestly, I have never followed them, I keep using the same configuration that has always worked for me for years. Maybe it is time to take a serious look at doing it as they recommend. My hesitation and frustration is that this site had been running trouble-free for over a year now, my hunch is still strong that something inside their network changed but I do not have the access or the means to prove it and I could be wrong.

Thanks, everybody for the help, I will try the ideas and post any news when it happens and if it happens.

RE

http://www.trinet-intl.com

APSS, AIPS