406V2 4.0.14 Reboots same time every morning

[telecomgear]

I'm re-visiting an old problem with a 404V2 running 4.0.14 that was rebooting and changing V.M. passcodes.
The V.M. problem has cleared...for now, but my reboot problem persists.
I've attached a monitor trace of the reboot. I see an alarm full message right before the reboot. Any ideas are always appreciated. Thanks.

176066841mS CMExtnTx: v=127, p1=0
CMFacility
Line: type=DigitalExtn 4 Call: lid=0 id=14230 in=0
Called[] Type=Default (100) Reason=CMDRdirect Calling[6153492754] Type=Unknown Plan=ISDN Pres=Allowed (0)
IE CMIESupplementaryService (3)
Interpretation APDU
rejectAnyUnrecognisedInvokePdu
CallingName.Invoke.CodePageISO8859-1
invokeId 39
user 'GOBEAM - CHICAG' presentation Allowed
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6153492754
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=18 gain=32
IE CMIECallingPartyName (110) GOBEAM - CHICAG
IE CMIERespondingPartyName (228) GOBEAM - CHICAG
Timed: 06/03/08 07:20
176067135mS RES: Thu 6/3/2008 07:20:10 FreeMem=46648384(41) CMMsg=6 (7) Buff=100 624 500 1045 5 Links=8712
176072826mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: TimerExpired cause=CMTCNoAnswerTimeout
176072826mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: ActionNoAnswerTimeout()
176072827mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: Retarget NOANSWER EXCEPTED=00000000 INCLUDED=00000000 valid=0
176072827mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: RetargetNoAnswer on HUNTGROUP=MainNightAA. Ring Attempt 0 All Extensions Attempted
176072827mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: AddHGTarget MainNightAA (depth=1) allowq=0 type=CMNTypeUnknown
176072828mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: OV visable. VM visable.
176072828mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: FindNextInServiceHgInHgIncludedList: Resolved No Alternative!
176072828mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: SELECT: TRY VOICEMAIL orig_hg(200) orig_user()
176072828mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: ADD VOICEMAIL TARGET
176072829mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: ADD VOICEMAIL TARGET: SUCCEEDED
176072831mS CMExtnTx: v=109, p1=0
CMReleaseComp
Line: type=DigitalExtn 5 Call: lid=0 id=14220 in=0
Cause=129,
Timed: 06/03/08 07:20
176072832mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: AddHGTargetRingGroupNewIdleExtn 109
176072832mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: ADD USER: ReceptionCordle depth=0 disallow_cw=0 dnd=0 real_call=0 type(CMNTypeVoicemail) incl(0x0) excpt(0x3c), allow_redir(1) remote=00000000
176072834mS CMTARGET: 5.15.1 2841 Q931 Trunk:5 CHAN=19: SelectTargetAvailable 1 targets
176072835mS CMExtnTx: v=RAS, p1=0
CMSetup
Line: type=RAS 1 Call: lid=0 id=1008 in=0
Called[#Main] Type=Voicemail (102) Reason=CMDRX_MailboxAccess SndComp Calling[6154648806] Type=Unknown Plan=ISDN Pres=Allowed (0)
BC: CMTC=Speech CMTM=Circuit CMTR=64 CMST=Default CMU1=ULaw
BChan: slot=6 chan=8
IE CMIECalledPartyName (224) MainNightAA
IE CMIEOriginalCalledPartyName (236) Main
IE CMIEOriginalCalledPartyNumber (238)(P:100 S:100 T:101 N:100 R:3) number=200
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:3) number=200
IE CMIEIcrPriorityDetail (239) Priority = 1
IE CMIERespondingPartyName (228) SMITHVILLE, TN
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6154648806
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=19 gain=32
UUI type=Local [oos] [0x6f 0x6f 0x73 ]
Display [6154648806>Main]
Timed: 06/03/08 07:20
Locale: enu
176072836mS CMMap: a=6.8 b=1.254 T
176072837mS CMExtnRx: v=RAS, p1=0
CMConnect
Line: type=NoLine 0 Call: lid=0 id=-1 in=0
IE CMIERespondingPartyName (228) #Main
IE CMIERespondingPartyNumber (230)(P:100 S:100 T:102 N:100 R:4) number=
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceVoicemail type=0 number=0 channel=8 gain=32
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:3) number=200
176072838mS CMExtnTx: v=109, p1=0
CMFacility
Line: type=DigitalExtn 5 Call: lid=0 id=14227 in=0
Called[109] Type=Unknown (0) Reason=CMDRdirect SndComp Calling[6153492754] Type=Unknown Plan=ISDN Pres=Allowed (0)
IE CMIECalledPartyName (224) MainNightAA
IE CMIEOriginalCalledPartyName (236) Main
IE CMIEOriginalCalledPartyNumber (238)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIEIcrPriorityDetail (239) Priority = 1
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6153492754
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=18 gain=32
UUI type=User2User [.] [0xff ]
Display [HeadOfQueueChanged]
Cause=124, Answered By Other(IPO)
Timed: 06/03/08 07:20
Locale: enu
176072838mS CMExtnTx: v=110, p1=0
CMFacility
Line: type=DigitalExtn 3 Call: lid=0 id=14228 in=0
Called[110] Type=Unknown (0) Reason=CMDRdirect SndComp Calling[6153492754] Type=Unknown Plan=ISDN Pres=Allowed (0)
IE CMIECalledPartyName (224) MainNightAA
IE CMIEOriginalCalledPartyName (236) Main
IE CMIEOriginalCalledPartyNumber (238)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIEIcrPriorityDetail (239) Priority = 1
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6153492754
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=18 gain=32
UUI type=User2User [.] [0xff ]
Display [HeadOfQueueChanged]
Cause=124, Answered By Other(IPO)
Timed: 06/03/08 07:20
Locale: enu
176072839mS CMExtnTx: v=132, p1=0
CMFacility
Line: type=DigitalExtn 4 Call: lid=0 id=14229 in=0
Called[132] Type=Unknown (0) Reason=CMDRdirect SndComp Calling[6153492754] Type=Unknown Plan=ISDN Pres=Allowed (0)
IE CMIECalledPartyName (224) MainNightAA
IE CMIEOriginalCalledPartyName (236) Main
IE CMIEOriginalCalledPartyNumber (238)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIEIcrPriorityDetail (239) Priority = 1
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6153492754
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=18 gain=32
UUI type=User2User [.] [0xff ]
Display [HeadOfQueueChanged]
Cause=124, Answered By Other(IPO)
Timed: 06/03/08 07:20
Locale: enu
176072840mS CMExtnTx: v=127, p1=0
CMFacility
Line: type=DigitalExtn 4 Call: lid=0 id=14230 in=0
Called[127] Type=Unknown (0) Reason=CMDRdirect SndComp Calling[6153492754] Type=Unknown Plan=ISDN Pres=Allowed (0)
IE CMIECalledPartyName (224) MainNightAA
IE CMIEOriginalCalledPartyName (236) Main
IE CMIEOriginalCalledPartyNumber (238)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIERedirectingPartyName (232) Main
IE CMIERedirectingPartyNumber (234)(P:100 S:100 T:101 N:100 R:4) number=200
IE CMIEIcrPriorityDetail (239) Priority = 1
IE CMIERespondingPartyNumber (230)(P:0 S:3 T:0 N:1 R:4) number=6153492754
IE CMIEDeviceDetail (231) LOCALE=enu HW=8 VER=4 class=CMDeviceISDNTrunk type=2 number=5 channel=18 gain=32
UUI type=User2User [.] [0xff ]
Display [HeadOfQueueChanged]
Cause=124, Answered By Other(IPO)
Timed: 06/03/08 07:20
Locale: enu
176072843mS CMExtnTx: v=110, p1=0
CMReleaseComp
Line: type=DigitalExtn 3 Call: lid=0 id=14221 in=0
Cause=123, Forward To Voicemail(IPO)
Timed: 06/03/08 07:20
176072844mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: AddHGTargetRingGroupNewIdleExtn 110
176072844mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: ADD USER: JackieCharleboi depth=0 disallow_cw=0 dnd=0 real_call=1 type(CMNTypeUnknown) incl(0x0) excpt(0x3c), allow_redir(1) remote=00000000
176072847mS CMExtnTx: v=132, p1=0
CMReleaseComp
Line: type=DigitalExtn 4 Call: lid=0 id=14222 in=0
Cause=123, Forward To Voicemail(IPO)
Timed: 06/03/08 07:20
176072848mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: AddHGTargetRingGroupNewIdleExtn 132
176072848mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: ADD USER: ChuckBiter depth=0 disallow_cw=0 dnd=0 real_call=1 type(CMNTypeUnknown) incl(0x40) excpt(0x3c), allow_redir(1) remote=00000000
176072851mS CMExtnTx: v=127, p1=0
CMReleaseComp
Line: type=DigitalExtn 4 Call: lid=0 id=14223 in=0
Cause=123, Forward To Voicemail(IPO)
Timed: 06/03/08 07:20
176072852mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: AddHGTargetRingGroupNewIdleExtn 127
176072852mS CMTARGET: 5.14.1 2842 Q931 Trunk:5 CHAN=18: ADD USER: MarkBallard depth=0 disallow_cw=0 dnd=0 real_call=1 type(CMNTypeUnknown) incl(0x40) excpt(0x3c), allow_redir(1) remote=00000000
176072854mS CMExtnTx: v=109, p1=0
CMReleaseComp
Line: type=DigitalExtn 5 Call: lid=0 id=14232 in=0
Cause=123, Forward To Voicemail(IPO)
Timed: 06/03/08 07:20
176072855mS CMLineTx: v=5
CMConnect
Line: type=Q931Line 5 Call: lid=0 id=15 in=1
BChan: slot=0 chan=19
Display [++f,]
176072857mS PRN: Alarm log full
176072857mS PRN: Begin Stack Trace
176072857mS PRN: pc=0a7ea894
176072857mS PRN: lr=ff6af614
176072857mS PRN: findfunc ff6af5f4 ff6afb4c ff6afe40 ff6afec4 ff54f2c4 ff5e0158 ff54abc0 ff54ab84 00000000 00000000
176072857mS PRN: End Stack Trace
176072857mS PRN: .FATAL TLB Instr address=ff27fc7c d=5 pc=0a7ea894 ff6af5f4 ff6afb4c ff6afe40 ff6afec4 ff54f2c4 IP 406 DS 4.0(14)

********** contact lost with 192.168.111.211 at 07:20:31 6/3/2008 - reselect = 2 **********
******************************************************************


********** SysMonitor v6.0 (14) **********

********** contact made with 192.168.111.211 at 07:20:56 6/3/2008 **********

********** System (192.168.111.211) has been up and running for 28secs(28685mS) **********


********** Warning: TEXT File Logging selected **********



********** Warning: TEXT Logging to File STARTED on 6/3/2008 07:20:56 **********

28686mS PRN: Monitor Started IP=192.168.111.210 IP 406 DS 4.0(14) Girtman
(IP Office: Supports Unicode, System Locale is enu)
28686mS PRN: LAW=U PRI=1, BRI=0, ALOG=4, ADSL=0 VCOMP=0, MDM=0, WAN=0, MODU=3 LANM=0 CkSRC=5 VMAIL=0(VER=0 TYP=1) CALLS=0(TOT=0)
28833mS LIST2: IFACE(4) "LAN1","DialIn","RemoteManager","DialIn"
29136mS RES: Thu 6/3/2008 07:20:56 FreeMem=47605860(1) CMMsg=2 (3) Buff=100 632 500 1052 5 Links=1438
29682mS PRN: Sorting User List...
29685mS PRN: User List Sorted
29688mS PRN: Sorting Hunt Group List...
29688mS PRN: Hunt Group List Sorted
29690mS PRN: Sorting Directory List...
29691mS PRN: Sorted Directory List.
29693mS PRN: Config Write Completed
31686mS PRN: +++ START OF ALARM LOG DUMP +++
31686mS PRN: ALARM: 25/01/2008 07:39:58 IP 406 DS 4.0(7) <TLB Instr> CRIT RAISED addr=ff27fb98 d=5 pc=2d3048f8 ff6af0a8 ff6af600 ff6af8e4 ff6af968 ff54ec44
31686mS PRN: ALARM: 31/01/2008 07:29:47 IP 406 DS 4.0(7) <TLB Instr> CRIT RAISED addr=ff27fb98 d=5 pc=1ee05a38 ff6af0a8 ff6af600 ff6af8e4 ff6af968 ff54ec44
31687mS PRN: ALARM: 01/02/2008 07:08:23 IP 406 DS 4.0(7) <Impl.Dep.> CRIT RAISED addr=ff27fb98 d=5 pc=fee48efc ff6af0a8 ff6af600 ff6af8e4 ff6af968 ff54ec44
31687mS PRN: ALARM: 04/02/2008 07:49:29 IP 406 DS 4.0(7) <TLB Instr> CRIT RAISED addr=ff27fc70 d=5 pc=0f988c88 ff6af0a8 ff6af600 ff6af8e4 ff6af968 ff54ec44
31688mS PRN: ALARM: 05/02/2008 07:55:50 IP 406 DS 4.0(7) <TLB Instr> CRIT RAISED addr=ff27fc67 d=5 pc=052bfe70 ff6af0c8 ff6af600 ff6af8e4 ff6af968 ff54ec44
31688mS PRN: ALARM: 06/02/2008 07:44:42 IP 406 DS 4.0(14) <TLB Instr> CRIT RAISED addr=ff27fe74 d=5 pc=04378538 ff6af5f4 ff6afb4c ff6afe40 ff6afec4 ff54f2c4
31688mS PRN: ALARM: 06/02/2008 07:47:35 IP 406 DS 4.0(14) <TLB Instr> CRIT RAISED addr=ff624e08 d=5 pc=00027120 ff6af614 ff6afb4c ff6afe40 ff6afec4 ff54f2c4
31689mS PRN: ALARM: 06/02/2008 08:07:50 IP 406 DS 4.0(14) <Impl.Dep.> CRIT RAISED addr=ff27fb98 d=5 pc=fed3bc14 ff6af614 ff6afb4c ff6afe40 ff6afec4 ff54f2c4
31689mS PRN: +++ END OF ALARM LOG DUMP +++
33638mS CMLineRx: v=5
CMLineDown


To Us and Those Like Us, Damn Few Left

 

[kholladay]

Where is this piece of IE comming from "GOBEAM - CHICAG"?

Kyle Holladay
ACA-I, ACA Call Center, ACS-I, ACS-M, TIA-CTP, MCP/MCTS Exchange 2007
ACE Implement: IP Office

"If it worked the way it should you wouldn't need me

 

[Bas1234]

I had this problem with a customer, every morning the Night message was turned off. Turned out to be the power socket. Every morning when he came in the shop he turned on the light but also the IPOffice...

I saw it in the Manager under audit trail, every morning it had a cold start at almost the same time.

Greetzzz...Bas

___________________________________________
It works! Now if only I could remember what I did...
___________________________________________

 

[telecomgear]

I'm not sure what the GOBEAM-CHICAG is. It wasn't present on the other monitor files I have showing a reboot.

To Us and Those Like Us, Damn Few Left

 

[Escorthosis]

Save this file and clear the alarm log.
Only 8 entries are logged and the latest arent logged any more. So clear the alarmlog using the monitor application to see the latest cause.
If you reboot your system while connected to the DTE port you'll see that the maximum number of entries in the alarmlog is eight.
Why not upgrading to 4.1.9. It has solved some intermittend reboot problems for my customers.
If you're running 4.0.14 you could also be running 4.1.09.
(This would also be the standard answer from Avaya when you'll ask them for help so be one step ahead and see if the problem is resolved in 4.1.9)

 

[tlpeter]

I see a phew things

IE CMIECallingPartyName (110) GOBEAM - CHICAG
If that name is in the directory then remove the spaces and the -

IE CMIERespondingPartyName (228) SMITHVILLE, TN
Remove the , and the spaces

Also i see this

Display [++f,]

IS there anything called ++f

I would start looking in voicemail

Never ever use ,-&^%$ or any other strange characters in an ipoffice !!!
ALso do not use spaces in the directory
This has been changed in 4.0 and causes trouble



ACA - Implement IP Office
ACS - Implement IP Office
ACA - Voice Services Management
______________
Women and cats can do as they please and men and dogs should relax and get used to the idea!

 

[telecomgear]

IE CMIECallingPartyName (110) GOBEAM - CHICAG

IE CMIERespondingPartyName (228) SMITHVILLE, TN"

The above was generated from incoming calls to the IPO, not outbound.

There are no @#$%^ type characters in the user names or hunt groups.

I also have no Directory intries in the config.

I have cleared the alarm log in Monitor, and then made a test call to the IPO from my cell phone, and got a similar entry in monitor of "rejectAnyUnrecognisedInvokePdu" from my cell phone call. My caller id was present in the entry, followed by the "rejectAnyUnrecognisedInvokePdu". IPO then replaced mu caller id with "Lafayette, LA".
I guess I need to ask the customer what they are seeing for caller id.
Can it be that the service provider is sending characters or other info in the caller id field that the IPO is not expecting or liking? This still does not explain the reboots at the same time in the morning.
I think and hope a upgrade or downgrade will fix the problem.





To Us and Those Like Us, Damn Few Left

 

[tlpeter]

I never have seen name coming in instead of numbers
That does not happen in europe so i don't know anything about it really


ACA - Implement IP Office
ACS - Implement IP Office
ACA - Voice Services Management
______________
Women and cats can do as they please and men and dogs should relax and get used to the idea!

 

[kwing112000]

I would say to rebuild it. You can spend a month trying to figure this out or spend 3 or 4 hours rebuilding it. Blow it away. downgrade and upgrade thru all the right steps to whatever version you need it at. Then retype the whole config in and i bet you never have another issue.

 

[digital12188]

You said this is am old problem.. How long has it been happening?? was it prior to 4.0 or did u upgrade unit and it started??

Is there any power issues in area where system located.. as wierd as it is.It could be dumd as a generator test starting up same time everyday cause reboot.

Also It looks like there is no VCM card in unti is that correct? I have had a problem with old VMs going bad and causing reboots..

Just a few area to look at