Hey folks. Looking for some assistance in troubleshooting an issue with an old BCM450 still in production at one of our sites. This BCM450 suddenly rebooted itself one morning and got stuck in a boot loop (initial observation).
At about 9AM into the workday, I got the notification that it had gone down. I checked the alarms, which are emailed, and there was no warning. Last issue was a bad password signing into the system via BEM weeks ago.
History: This unit had issues randomly rebooting about four years ago. Turned out the PSU was the culprit. We pulled it, tested it, and it failed. We ordered a replacement PSU and it's been solid since.
Considering the last issue, I immediately pulled the PSU and tested it with an ATX PSU tester on a load, it tested fine.
Since the PSU doesn't seem to be the issue, I put it back in and booted up the system with access to the console. I'm seeing a good POST and it's reading the HDD. However, when it gets to the FPGA configuration, it fails. I have a multi-image HDD, a replacement HDD, but it's R6, and a clone of this drive. So far, replacing the drive with its clone has the same result. Trying the R6 disk also has the same result. We have not tested with the multi-image HDD yet.
EDIT - Boot Loop Video:
We have current weekly backups up to 8/12/2024. The system is set to reboot itself every week in the evening (1AM) after a backup (11PM).
BFT Tage: BCM450 BFT NTC01050
Original System HDD: LBNNTMEND8N37G NTC03101SXE6 01 N0202621 BCM450 R5 Programmed Hard Drive (Seagate Barracuda 7200RPM SATA 80GB - OEM - came with the system).
We have an BCM450 R6 Replacement HDD (160GB) and a Multi-Image BCM450 HDD (250GB).
I have ordered a replacement PSU just in case.
Any assistance or nudge in the right direction for next steps would be appreciated. Below is what I'm seeing in the console and what BEM has recorded for our system (BEM About):
For reference, here's what I'm seeing in the console:
BCM U-Boot 1.2.2 (Jul 8 2010 - 14:37:07), Build: hudson-BCM_5_Sustaining_M450-2 9
CPU: 8567_E, Version: 1.1, (0x807d0111)
Core: E500, Version: 2.2, (0x80210022)
Clock Configuration:
CPU:1333 MHz, CCB: 533 MHz,
DDR: 266 MHz, LBC: 33 MHz
QE: 533 MHz
L1: D-cache 32 kB enabled
I-cache 32 kB enabled
Board: BMB450 R1.0
I2C: ready
DRAM: Initializing
DDR: 512 MB
POST: running...
POST memory PASSED
L2 cache 512KB: enabled
Extended Memory test: Passed
FLASH: 8 MB
Reboot Reason: Power Recycle
Configuring Mac address from NsSecSi PROM
PCI: first=0 last=0
PCIe: LTTSM=0x16
PCIe: first=1 last=1
In: serial
Out: serial
Err: serial
Net: FSL UEC1, FSL UEC0, FSL UEC2, FSL UEC3, FSL UEC4, FSL UEC5, FSL UEC6, FSL UEC7
IDE: Bus 0: OK
Device 0: Model: ST31000524NS Firm: 130C Ser#: 9WK54CTC
Type: Hard Disk
Capacity: 131071.9 MB = 127.9 GB (268435455 x 512)
Device 1: not available
start USB...
USB: EHCI pci controller (1131, 1562) found @(0:1:2)
RTRY/TRDY timeout set to 00000000 from 00008000
EHCI regs address 0x80002020
OHCI pci controller (1131, 1561) found @(0:1:0)
OHCI regs address 0x80000000
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices...
probing USB device 0...
probing USB device 1...
1 Storage Device(s) found
reading m450_fpga.image
124284 bytes read from ide ,finished....
Image Name: M450 MAIN: rel 91
Created: 2008-09-05 14:15:40 UTC
Image Type: PowerPC Unknown OS Firmware (gzip compressed)
Data Size: 124220 Bytes = 121.3 kB
Load Address: 00010000
Entry Point: 00000000
Verifying Checksum ... OK
Uncompressing fpga Firmware ... OK
Configuring The FPGA Device .................................................... ................................................................................ ................................................................................ ................................................................................ ............................
Test failed: FPGA Done not clear clocking 16 more times
Test failed: FPGA Done still not clear
Tried to write 480131 bytes...
ERROR: WriteConfigDataToFPGA()
Failed at download fpga image
Fatal Error Occurred
Hit any key to stop autoboot: 16
=>
For reference, the BEM About:
Nortel Business Element Manager Version 60.3.0
View [URL=']README[/URL]
View [URL=']License Agreement[/URL]
Copyright (c) 2005-2024 [URL=']Nortel[/URL] All rights reserved.
Installed Cartridges:
BCM 50R5 Element Manager Cartridge 5.0.0 (BCM50R5.aa94-0)
0.1.0 (JDM version:603b02_01 supports BES1xx/2xx and BES1xxx/2xxx devices)
BCM 450R5 Element Manager Cartridge 5.0.0 (BCM450R5.ab62-0)
BCM 50R6 Element Manager Cartridge 6.0.0 (BCM50R6.ab47-0)
Technical Support: http://www.nortel.com/support
Java Version: 1.6.0_07
Java Vendor: Sun Microsystems Inc.
Java Home: C:\Program Files\Nortel\Business Element Manager\jre
Java Class Version: 50.0
Java Class Path: C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\lib\patch.zp;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin\run.jar;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\server\default\conf;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin\lax.jar;
Operating System: Windows Vista
System Architecture: x86
Operating System Version: 6.2
Build ID: bcmsusemfw60aa07
At about 9AM into the workday, I got the notification that it had gone down. I checked the alarms, which are emailed, and there was no warning. Last issue was a bad password signing into the system via BEM weeks ago.
History: This unit had issues randomly rebooting about four years ago. Turned out the PSU was the culprit. We pulled it, tested it, and it failed. We ordered a replacement PSU and it's been solid since.
Considering the last issue, I immediately pulled the PSU and tested it with an ATX PSU tester on a load, it tested fine.
Since the PSU doesn't seem to be the issue, I put it back in and booted up the system with access to the console. I'm seeing a good POST and it's reading the HDD. However, when it gets to the FPGA configuration, it fails. I have a multi-image HDD, a replacement HDD, but it's R6, and a clone of this drive. So far, replacing the drive with its clone has the same result. Trying the R6 disk also has the same result. We have not tested with the multi-image HDD yet.
EDIT - Boot Loop Video:
We have current weekly backups up to 8/12/2024. The system is set to reboot itself every week in the evening (1AM) after a backup (11PM).
BFT Tage: BCM450 BFT NTC01050
Original System HDD: LBNNTMEND8N37G NTC03101SXE6 01 N0202621 BCM450 R5 Programmed Hard Drive (Seagate Barracuda 7200RPM SATA 80GB - OEM - came with the system).
We have an BCM450 R6 Replacement HDD (160GB) and a Multi-Image BCM450 HDD (250GB).
I have ordered a replacement PSU just in case.
Any assistance or nudge in the right direction for next steps would be appreciated. Below is what I'm seeing in the console and what BEM has recorded for our system (BEM About):
For reference, here's what I'm seeing in the console:
BCM U-Boot 1.2.2 (Jul 8 2010 - 14:37:07), Build: hudson-BCM_5_Sustaining_M450-2 9
CPU: 8567_E, Version: 1.1, (0x807d0111)
Core: E500, Version: 2.2, (0x80210022)
Clock Configuration:
CPU:1333 MHz, CCB: 533 MHz,
DDR: 266 MHz, LBC: 33 MHz
QE: 533 MHz
L1: D-cache 32 kB enabled
I-cache 32 kB enabled
Board: BMB450 R1.0
I2C: ready
DRAM: Initializing
DDR: 512 MB
POST: running...
POST memory PASSED
L2 cache 512KB: enabled
Extended Memory test: Passed
FLASH: 8 MB
Reboot Reason: Power Recycle
Configuring Mac address from NsSecSi PROM
PCI: first=0 last=0
PCIe: LTTSM=0x16
PCIe: first=1 last=1
In: serial
Out: serial
Err: serial
Net: FSL UEC1, FSL UEC0, FSL UEC2, FSL UEC3, FSL UEC4, FSL UEC5, FSL UEC6, FSL UEC7
IDE: Bus 0: OK
Device 0: Model: ST31000524NS Firm: 130C Ser#: 9WK54CTC
Type: Hard Disk
Capacity: 131071.9 MB = 127.9 GB (268435455 x 512)
Device 1: not available
start USB...
USB: EHCI pci controller (1131, 1562) found @(0:1:2)
RTRY/TRDY timeout set to 00000000 from 00008000
EHCI regs address 0x80002020
OHCI pci controller (1131, 1561) found @(0:1:0)
OHCI regs address 0x80000000
scanning bus for devices... 2 USB Device(s) found
scanning bus for storage devices...
probing USB device 0...
probing USB device 1...
1 Storage Device(s) found
reading m450_fpga.image
124284 bytes read from ide ,finished....
Image Name: M450 MAIN: rel 91
Created: 2008-09-05 14:15:40 UTC
Image Type: PowerPC Unknown OS Firmware (gzip compressed)
Data Size: 124220 Bytes = 121.3 kB
Load Address: 00010000
Entry Point: 00000000
Verifying Checksum ... OK
Uncompressing fpga Firmware ... OK
Configuring The FPGA Device .................................................... ................................................................................ ................................................................................ ................................................................................ ............................
Test failed: FPGA Done not clear clocking 16 more times
Test failed: FPGA Done still not clear
Tried to write 480131 bytes...
ERROR: WriteConfigDataToFPGA()
Failed at download fpga image
Fatal Error Occurred
Hit any key to stop autoboot: 16
=>
For reference, the BEM About:
Nortel Business Element Manager Version 60.3.0
View [URL=']README[/URL]
View [URL=']License Agreement[/URL]
Copyright (c) 2005-2024 [URL=']Nortel[/URL] All rights reserved.
Installed Cartridges:
BCM 50R5 Element Manager Cartridge 5.0.0 (BCM50R5.aa94-0)
0.1.0 (JDM version:603b02_01 supports BES1xx/2xx and BES1xxx/2xxx devices)
BCM 450R5 Element Manager Cartridge 5.0.0 (BCM450R5.ab62-0)
BCM 50R6 Element Manager Cartridge 6.0.0 (BCM50R6.ab47-0)
Technical Support: http://www.nortel.com/support
Java Version: 1.6.0_07
Java Vendor: Sun Microsystems Inc.
Java Home: C:\Program Files\Nortel\Business Element Manager\jre
Java Class Version: 50.0
Java Class Path: C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\lib\patch.zp;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin\run.jar;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\server\default\conf;C:\Program Files\Nortel\Business Element Manager\BusinessElementManager\bin\lax.jar;
Operating System: Windows Vista
System Architecture: x86
Operating System Version: 6.2
Build ID: bcmsusemfw60aa07
Last edited: