Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations John Tel on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

watchdog timeout IPO 9.0.1.0 845

Status
Not open for further replies.

ipotech808

Vendor
Feb 9, 2012
92
US
I have a IPO 9.0.1.0 845 IP500 V2 with 4 remote sites connected to it. The main site houses 4 PRIs and 2 VCM64 cards. They are using VMPro and One X on this with almost 340 IP extensions and 700+users (its high because of fax finder users added in).

They have 9608 and 9641G IP phones connected to it. Everything is connected with the Avaya POE switches gigabit.

The issue I am having is after doing couple merges and then a reboot, I get this and I would have to DTE AT-X3, 2 to erase everything and start from scratch. I tried to reload a few .bak files but they were also corrupted. The system has been doing this ever since last year Nov. Avaya R&D did not get back to me on this but I am hoping someone else out here got this error and knew how to fix it. Any takers? Thanks in advance..


P14 Loader v1.33
CPU Board Version is: 0x04
CPU PLD Version is: 0x25
CPUID: 0x12 Present: 0xA0 Present2: 0x02
Using FPGA build 0x00000827
Programming FPGA ...
FPGA ID 0x01 Version 0x00 Build 0x00000827
Initialising NAND device.. registering..
Added device: ST Micro 64MB NAND (small block) - 0x0000

SMXFS: This is a SD 2.0 card
SMXFS: This is SDHC card
SMXFS: IT'S SD/SDHC CARD
SMXFS: Sector Number is 76A000
SMXFS: Sector Size is 200
OPTIONAL SD Card not present/recognised
Initialising FileSys device.. registering..
Added FileSys device
Running:
A:\system\primary\ip500v2.bin
Reading main image file..
Read complete
Expanding MPPC image ...
Expansion complete

FecRtpFilter::FecRtpFilter starting
FecRtpFilter::FecRtpFilter after OSBuf:Alloc
FecRtpFilter::FecRtpFilter complete

Alarm Logging enabled (max entries=32, in use=32)
Audit Logging enabled (max entries=111, in use=63)
NoCacheStaticHeap=29274320 NoCacheDynamicHeap=3231536 Overflow=0
Constructor StaticHeap=45608 DynamicHeap=73187124
Factory Test Status 00000001
Product Variation Status 00000003
Boot Time: 2014/2/25 07:39:11
Boot Header, Current Image: 0xffffffff, 0xfffffffe
Boot Image Counts: 0xfffffffe, 0xffffffff, 0xffffffff, 0xffffffff
FPGA: ID=0x01 Issue=0x00 BuildNum=0x00000827
Conf DSP: loading image to addr 0090, size 32480
Conf DSP: checking image
Conf DSP: image load check completed
Conf DSP: image loaded and running
Confer DSP - image version 02.07 (release 00)
Broadcom Switch Mode was 0x05 now 0x06
EthChipRevId: 0x01 EthPhyId: 0x0143bc30
+++++++++++++++ CardType = 12 +++++++++++
+++++++++++++++ CardType = 12 +++++++++++
+++++++++++++++ CardType = 18 +++++++++++
+++++++++++++++ CardType = 18 +++++++++++
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
+ loader: 1.33
+ cpu: id 12 board 4 pld 25 type c10 options a002
+ cpu: sn 13WZ226005ZF
+ fpga: id 1 issue 0 build 827
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
++++++++++++++++++++ LIST OF MODULES ++++++++++++++++++++
+------------------------------------------------------------
+ Slot 1: Base PHONE8 Board=0x05 PLD=0x03
+ Mezzanine PRID U Board=0x03 PLD=0x0a
+------------------------------------------------------------
+ Slot 2: Base PHONE8 Board=0x05 PLD=0x03
+ Mezzanine PRID U Board=0x03 PLD=0x0a
+------------------------------------------------------------
+ Slot 3: Base VCM64 Board=0x05 PLD=0x11
+ Mezzanine ATM4 V2 Board=0x01 PLD=0x04
+------------------------------------------------------------
+ Slot 4: Base VCM64 Board=0x05 PLD=0x11
+ Mezzanine ATM4 V2 Board=0x01 PLD=0x04
+------------------------------------------------------------
+++++++++++++++++ END OF LIST OF MODULES +++++++++++++++
PRI: configurePriPluginCard slot=1 universal=1 single=0
PRI: Identified: mod_card_type=9, ds1=1, r2=0
PRI: Bases= 45009000 4500a000 45008000 4500b000
Falc56 detected, VSTR 0x05, WID 0x50
Falc56 detected, VSTR 0x05, WID 0x50
PRIU DSP: loading image to addr 0090, size 32480
PRIU DSP: checking image
PRIU DSP: image load check completed
PRIU DSP: image loaded and running
PRIU DSP 1: image version 02.02 (release 00)
SLOT 1: Dual PRI24 Module Added
PRI: configurePriPluginCard slot=2 universal=1 single=0
PRI: Identified: mod_card_type=9, ds1=1, r2=0
PRI: Bases= 45019000 4501a000 45018000 4501b000
Falc56 detected, VSTR 0x05, WID 0x4e
Falc56 detected, VSTR 0x05, WID 0x4e
PRIU DSP: loading image to addr 0090, size 32480
PRIU DSP: checking image
PRIU DSP: image load check completed
PRIU DSP: image loaded and running
PRIU DSP 2: image version 02.02 (release 00)
SLOT 2: Dual PRI24 Module Added
TI VCM: New PCB, slot 2, TDM stream 4
Broadcom Switch Mode was 0x05 now 0x06
TI VCM: Created core 0, devId 9, channels 16
TI VCM: Created core 1, devId 9, channels 16
TI VCM: Created core 2, devId 10, channels 16
TI VCM: Created core 3, devId 10, channels 16
VCM configured in slot 2
SLOT 2: Analogue Trunk Mezzanine 4 version 2 Added
ATM4Uv2 DSP: loading image to addr 0090, size 57056
ATM4Uv2 DSP: checking image
ATM4Uv2 DSP: image load check completed
ATM4Uv2 DSP: image loaded and running
ATM4Uv2 DSP 2: image system identifier, 0x05
ATM4Uv2 DSP 2: image version 02.02 (release 00)
Host->DSP: Base 0x009c, Size 32, Read-ptr 0, Write-ptr 0
DSP->Host: Base 0x00bc, Size 64, Read-ptr 0, Write-ptr 0
+++DSP Clipping Value is set to 6000
TI VCM: New PCB, slot 3, TDM stream 6
Broadcom Switch Mode was 0x05 now 0x06
TI VCM: Created core 4, devId 13, channels 16
TI VCM: Created core 5, devId 13, channels 16
TI VCM: Created core 6, devId 14, channels 16
TI VCM: Created core 7, devId 14, channels 16
VCM configured in slot 3
SLOT 3: Analogue Trunk Mezzanine 4 version 2 Added
ATM4Uv2 DSP: loading image to addr 0090, size 57056
ATM4Uv2 DSP: checking image
ATM4Uv2 DSP: image load check completed
ATM4Uv2 DSP: image loaded and running
ATM4Uv2 DSP 3: image system identifier, 0x05
ATM4Uv2 DSP 3: image version 02.02 (release 00)
Host->DSP: Base 0x009c, Size 32, Read-ptr 0, Write-ptr 0
DSP->Host: Base 0x00bc, Size 64, Read-ptr 0, Write-ptr 0
+++DSP Clipping Value is set to 6000
Dummy allocate 5280 to align QmcBds f7beeb60
IP408_QmcBds f7bf0000 Tx 16 Rx 8 Size 6144
IP408_QmcBds f7bf1908 Tx 16 Rx 8 Size 6144
Configuration of Slot modules environment
*** configureSlotModulesEnvironment type =12 slot = 0
*** configureSlotModulesEnvironment type =12 slot = 1
*** configureSlotModulesEnvironment type =18 slot = 2
*** configureSlotModulesEnvironment type =18 slot = 3
Registering Volume 0, Drive 0 (A:)
Registering Volume 1, Drive 1 (B:)
SEC: Initialisation OK. Base = 0f040000. Type = SEC1P0. Coherent Cache = 1.
Unit warm start = 1
RTC BRAM = A5FFC300
Configuration of interrupts
DVP 0: Constructor - 8 lines firstPort 0 - base TS 0 - deviceType 2
DVP 0: VpBootLoad successful (65 ms)
DVP Product VP_DVP_PRODUCT_VE_880 found
0000009925 - XDPHW: Enet Port ID: 0 IP Address: 1.1.1.3
0000009990 - XDPHW: Enet Port ID: 1 IP Address: 1.1.1.4
DVP 0: Boot complete event
DVP 0: RevCode=0xB ProdId=0x80 SwVer=1.02
DVP 0: CreateLines entry adding start 0 end = 8 before lines = 8 (firstPort 0)
Setting default options for DVP(All Lines)...
0000010032 - XDPHW: Enet Port ID: 2 IP Address: 1.1.1.5
DVP 0: Initialisation complete event 0xff
DVP 0: L00 P00: Ring - t/slot( 0: 0) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L01 P01: Ring - t/slot( 1: 1) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L02 P02: Ring - t/slot( 2: 2) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L03 P03: Ring - t/slot( 3: 3) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L04 P04: Ring - t/slot( 4: 4) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L05 P05: Ring - t/slot( 5: 5) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L06 P06: Ring - t/slot( 6: 6) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L07 P07: Ring - t/slot( 7: 7) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: Initialisation complete.
DVP 0: CreateLines entry adding start 8 end = 16 before lines = 8 (firstPort 0)
DVP 0: L08 P08: Ring - t/slot( 8: 8) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L09 P09: Ring - t/slot( 9: 9) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L10 P10: Ring - t/slot(10:10) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L11 P11: Ring - t/slot(11:11) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L12 P12: Ring - t/slot(12:12) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L13 P13: Ring - t/slot(13:13) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L14 P14: Ring - t/slot(14:14) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
DVP 0: L15 P15: Ring - t/slot(15:15) - law(A) - DTMF(1 1) - pulse OFF - Neutral - MW
0000010034 - XDPHW: Enet Port ID: 3 IP Address: 1.1.1.6
0000010036 - XDPHW: Enet Port ID: 4 IP Address: 1.1.1.7
0000010038 - XDPHW: Enet Port ID: 5 IP Address: 1.1.1.8
0000010040 - XDPHW: Enet Port ID: 6 IP Address: 1.1.1.9
0000010042 - XDPHW: Enet Port ID: 7 IP Address: 1.1.1.10
0000010044 - XDPHW: Enet Port ID: 8 IP Address: 1.1.1.11
0000010046 - XDPHW: Enet Port ID: 9 IP Address: 1.1.1.12
0000010048 - XDPHW: Enet Port ID: 10 IP Address: 1.1.1.13
0000010050 - XDPHW: Enet Port ID: 11 IP Address: 1.1.1.14
0000010075 - XDP: Initializing DSP instances
0000010075 - XDP: Initializing AGGR instances
0000010075 - XDP: Registering with XDPHW5
SMXFS: This is a SD 2.0 card,0
SMXFS: This is SDHC card
SMXFS: IT'S SD/SDHC CARD
SMXFS: Sector Number is 76A000
SMXFS: Sector Size is 200
0000 27 50 48 53 44 30 34 47 30 6d 42 1a 60 00 d7 0d 'PHSD04G0mB.`.▒.
Manu:39 Oem:pH Prod:SD04G rev:48 sernum:6d421a60 date:7/2013
Card Inserted into System SD Slot: 2894266368 bytes free of 3975184384 bytes capacity
found FLASH file: ..\vcomp\48105ak.310, len=e2
found FLASH file: ..\vcomp\48105ae3.312.03, len=1ee4e
found FLASH file: ..\platform\onehz.bin, len=3e80
found FLASH file: ding.pcm, len=40a
found FLASH file: ..\vcomp\482ak.311.09, len=10e
found FLASH file: ..\vcomp\48204ae6.311.09, len=1da6a
found FLASH file: descriptor, len=42
Attempting Copy from MemoryCard to NAND
No Copy Required from MemoryCard to NAND
Attempting to copy SMDR and DHCP files to NAND
File created A:\system\primary\dhcp.dat
SD file A:\system\dynamic\smdr\smdrbuf.txt not found
Failed to copy SMDR & DHCP files to NAND
RouteSystem::LAN1 Complete...
RouteSystem::LAN2 Complete...
RouteSystem:: LAN1 ipaddr=C0A83D0A ipmask=FFFFFF00
RouteSystem:: LAN2 ipaddr=C0A82B01 ipmask=FFFFFF00
ASBC. Second Watermark is Invalid
Valid Second Watermark
Software Licence Grace Period Day=200 Grace allowed=2
Software Licence Grace Version 7
Allow licence handler to run early...
Attempting Config load from FILE...
TDMLink 1 Online
Config File: Header Info: crc32=3664439585, length=877000, ver(9.0.1.0 build 845), unit_type=0
Config File: CheckFileIntegrity PASSED. (crc32=3664439585, file length = 877000)
SetSystemLocale: enu, 2
SetSystemLocale: enu, 2
Control Unit type IP 500 V2 last version 9.0.1.0 build 845
FILE Config loaded successfully
Attempting Security Config load from FILE...
Security config File: Verify File Integrity: PASSED
Security Config loaded successfully from FILE
Attempting call log load from FILE...
Call log File: Header Info: crc32=1070098769, length=2545, ver(9.0.1.0 build 845), unit_type=9
Call log File: CheckFileIntegrity PASSED. (crc32=1070098769, file length = 2545)
Call Log FILE loaded successfully
Diffserve settings - VAL=b8 VIDEO=b8 MASK=fc
SetSystemLocale: enu, 2
Configuration::Complete
keys.txt could not be opened
RouteSystem::Starting DHCP Server (LAN1)
Reading DHCP data from SD card
Initialising SMDR
smdr header check returned 15
Load SMDR from file returned 0
Load SMDR from NAND...
.WATCHDOG TIMEOUT wdogmax 249990000l StartUp stack f14b8ea4 f1483904 f14ffab4 f14fec98 f14fe5dc f1513590 f1511bb8 f1511c68 f14ab834 f14ab8a0 f14a5fd8 f14a6108 f017cbf0 f000a24c f017929c f0329f64 f0329f10 IP 500 V2 9.0.1.0 build 845

 
With that amount of user i would put a server edition in.
350 extensions is within the limits but 700 users is not.
Do all these users have One-X Portal?
How many groups do you have?
How many shortcodes?


BAZINGA!

I'm not insane, my mother had me tested!

 
80% of them use One X.

There is 29 HG

I only added about 30 more short codes to the default.

We were told using IP500 V2 was okay for this build by Avaya... I don't know what to do I am trying to load any backups i have and they fail...
 
Are these 350 extension and 700 users on one system or in a SCN?
But i think your config is indeed corrupt.
I would start over with an offline config and then load that one in to see what happens.

BAZINGA!

I'm not insane, my mother had me tested!

 
Yes after much anticipation with authorization codes not working over the SCN and issues with other things, we had to migrate everyone to the main unit. It makes no sense that Avaya tells us 384 max extensions, 1000users max when i am not even close to that... grrrr.
 
Check in SSA how much memory the config uses.
Try a rebuild first because it is corrupted now for sure.

BAZINGA!

I'm not insane, my mother had me tested!

 
SSA says about 870k.

I realllly cant rebuild, so much data and i have to do 911 test calls in 4hrs... well at least i can say that i maxed this thing out and killed it.

do you think repeated bootp requests could cause it to time out?
 
I see over 200+ requests come in for bootp and i am wondering if that could be the cause? the IPO is the DHCP server for the phones so the phones point to the IPO. What you think Peter?
 
I would try to use a dedicated DHCP server.
But you should make a monitor trace and let it log until there is a reboot.
Then analyze the trace and see what happens before the reboot.
It could be faulty programming and i really think that because of the corrupted config.


BAZINGA!

I'm not insane, my mother had me tested!

 
well im kinda stuck cause none of the saves (even the good saves) are working now...
 
That is why you have to rebuild.

BAZINGA!

I'm not insane, my mother had me tested!

 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top