NBX V3000 suddenlny is frozen

[iso30]

My NBX V3000 suddenlny is frozen.
the NBX hang, no dial tone on the phones, not accessible from the network, can't receive outside calls.
The access to the NBX via Hyper terminal is possible during the problem.I am turning off the NBX and on in order to bypass the issue.
This is causing a check disk that is taking from 1 to 2 hours to finish, during this period we are not able to receive or dial calls.

Any ideas??
How to reboot NBX from hyperterminal?

 

[cknipe]

Press CTRL+R to reset the nbx from hyperterm that will soft reset it and keep it from the disk check

 

[iso30]

Hi

Any idea on how to check logs on NBX?
what to search for to diagnose the problem encoutered?

Thanks

 

[petenutz]

Hi, I'm having the same kind of problems with a V3000. No web access, no answering, presumably no dialtone. Is there anything special way of setting up Hyperterminal to accept the ctrl-r? I am communicating with the console fine it just won't reboot with ctrl-r. Only suspended tasks are imap141 and TransportRe. The name gets cut off so I'm not sure what that is. Can't get the ti command to work with TID.

Thanks in advance for advice!

NAME ENTRY TID PRI STATUS PC SP ERRNO DELAY
---------- ------------ -------- --- ---------- -------- -------- ------- -----
tExcTask 10b189c 6fb3fc8 0 PEND 10e579a 6fb3f38 3006b 0
tLogTask 1047474 6fb16c4 0 PEND 10e579a 6fb1630 3d0001 0
tShell 10c28e4 6a2e4bc 1 READY 1059db8 6a2e190 0 0
LedDisplay 9c7508 6f70738 2 DELAY 1053f1e 6f706b8 0 6
tTelnetd 1073e04 6a6cc5c 2 PEND+S 1051fcc 6a6cb88 0 0
tWdbTask 107cc70 6a2f674 3 PEND 1051fcc 6a2f328 0 0
Disk 0 9c4258 6a2aa64 3 PEND 1051fcc 6a2a9dc 0 0
itimerd 92e498 6485e6c 3 PEND 1051fcc 6485de0 0 0
t1 92e498 6483cd8 3 PEND 1051fcc 6483c5c 0 0
cleanupd 92e498 6481b44 3 PEND 1051fcc 6481ad4 0 0
kmemd 92e498 647f9b0 3 PEND 1051fcc 647f918 0 0
fsflushd 92e498 647d81c 3 PEND 1051fcc 647d798 3d0002 0
pageoutd 92e498 647b688 3 PEND 1051fcc 647b60c 0 0
ElvisCodecTd24784 49f03ac 38 PEND 1051fcc 49f0350 0 0
nbxElvis d245e8 5c9dd4c 39 PEND 1051fcc 5c9dcec 0 0
ElvisAudioRd281f0 49f26f8 39 PEND 10e579a 49f2628 0 0
ElvisReceivd283b0 49f1564 39 PEND 10e579a 49f1498 0 0
nbxH3errorHcf80d0 5c98d98 40 PEND 1051fcc 5c98cf8 0 0
tAioIoTask110b8818 6f9b718 50 PEND 1051fcc 6f9b6b8 0 0
tAioIoTask010b8818 6f94584 50 PEND 1051fcc 6f94524 0 0
tNetTask 106cf14 6a8ce6c 50 PEND 10e4d01 6a8ce0c 0 0
nbxH3AudioRcfd95c 5c93de4 50 PEND 1051fcc 5c93d8c 0 0
nbxH3readercfddac 5c45988 50 PEND 1051fcc 5c45930 0 0
tH3Input df41c8 5a82158 50 PEND 1051fcc 5a82114 0 0
tH3LoSnd df2138 5a80fc4 50 PEND+T 1051fcc 5a80f78 3d0004 2
tH3UDPRx df79d4 5a7fe30 50 PEND 1051fcc 5a7fc7c 0 0
TransportRe6121c4 594552c 50 SUSPEND 615900 5945254 0 0
imap141 d402a0 45bba10 50 SUSPEND+I 30eaa43 82fff104 0 0
tAioWait 10b8d58 6fa28ac 51 PEND 1051fcc 6fa25a8 0 0
tFtpdTask 10c398 6a6b4a0 55 PEND 1051fcc 6a6b3cc 0 0
time 6121c4 592a060 55 PEND 1051fcc 5929fe8 3d0004 0
nbxCallProc111d14 594d38c 60 PEND 10e4d01 594cfb0 d40010 0
VoiceMailRe6121c4 5940578 75 PEND 1051fcc 593ff94 0 0
nbxCCMDBserd08c38 5a79ce8 99 PEND 10e579a 5a79c40 0 0
nbxDownloadd1ffb0 5ccc6ec 100 PEND 10e4d01 5ccbf84 d40010 0
nbxCCMserved08648 5a7ec9c 100 PEND 1051fcc 5a7ebb8 0 0
nbxCCMVTLsed08648 5a76bbc 100 PEND 1051fcc 5a76ad8 3d0001 0
nbxVAppIL eaa6c0 5a6c498 100 PEND 1051fcc 5a6c06c 0 0
nbxMediaT00ddf6b0 5a65ef8 100 PEND 10e579a 5a65e30 d40010 0
nbxMediaT01ddf6b0 5a51614 100 PEND 10e579a 5a5154c d40010 0
nbxMediaT02ddf6b0 5a3cd30 100 PEND 10e579a 5a3cc68 d40010 0
nbxMediaT03ddf6b0 5a2844c 100 PEND 10e579a 5a28384 0 0
nbxInetD d3e3a0 4767110 100 PEND 1051fcc 4766d40 2b0001 0
nbxVMAA0 d402a0 49f6ea0 100 PEND 1051fcc 4760b20 2d 0
nbxVMAA1 d402a0 469dfc8 100 PEND 10e579a 469dd7c d40005 0
nbxVMAA2 d402a0 46693f8 100 PEND 10e579a 46691ac d40005 0
nbxVMAA3 d402a0 46341f0 100 PEND 10e579a 4633fa4 0 0
nbxBitSended402a0 45a2994 100 PEND+T 10e579a 45a28d4 d40010 32553
nbxNotMgr a61a60 4592100 100 PEND 10e579a 4592064 0 0
nbxYAVA a55130 458f7dc 100 PEND 10e579a 458f630 0 0
OutDialT0 a656b4 457ba2c 100 PEND 10e579a 457b990 0 0
OutDialT1 a656b4 45670c8 100 PEND 10e579a 456702c 0 0
OutDialT2 a656b4 4552710 100 PEND 10e579a 4552674 0 0
OutDialT3 a656b4 453dd84 100 PEND 10e579a 453dce8 0 0
INetDlvry0 a71390 452936c 100 PEND 10e579a 45292b8 0 0
TCPServerT a75958 451edd0 100 PEND 1051fcc 451eb88 2b0001 0
DiskLogger 9c7508 6a281bc 128 PEND 1051fcc 6a28148 0 0
Performance9c7508 6a050ec 128 PEND 1051fcc 6a05078 0 0
nbxLogger dd4814 692abd4 200 PEND+T 1051fcc 692a65c 3d0004 29786
nbxLogRemotdd96b4 688b45c 200 PEND 10e579a 688b388 0 0
rdcTask dfbdb4 49b0970 200 PEND 10e579a 49b08b0 0 0
nbxDiagCLISdfa010 5c3f9ac 200 PEND 1051fcc 5c3f8a4 0 0
tEmWeb 625310 59e43f4 240 PEND+T 1051fcc 59e3f94 0 1683
cpuUsageTas9c2c4c 6ffee20 255 PEND 1051fcc 6ffedd4 0 0
nbxACDRTS c85750 5a71470 255 PEND+T 1051fcc 5a70cf0 3d0004 224
value = 0 = 0x0

 

[cknipe]

First off when troubleshooting system "lock ups" it can be tedious, tricky and exhausting. When the system freezes first find out
1. Do any phones have dialtone (no)
2. Can you access the v3000 admin Gui (no)
3. Can you ping the v3000? (no)
4. Check the v3000 ethernet cable, is it connected? does it have a link light with activity?
5. Move the v3000 to a different ethernet switch port, does that help?
6. do an "i" command from the console and take note of suspended tasks
7. See my notes on running nbxSaveCore command, if 1-5 does not help 3com support will need to be involved to decipher some of the logs. Also, start keeping good records of the time (exact) that the system fails.
8. Sometimes its a good idea to plug the nbx into a hub and plug that hub into the network switch. This will allow you to connect your pc to that hub and use a program like wireshark, to sniff the line to see whats coming thorugh. sometimes there can be other apps on the network using certain ports like 1025 and 1026 that can interfere with the nbx...not as likely though



when the system is locked up, before doing a ctrl+R, type nbxSaveCore and rpess enter. this will take a few minutes but it will dump the current logs into a certain directory

nbxSaveCore will do asynchronous writes to the disk and does not tar up all the files, saving precious minutes when you need to get the system operational. This new command means less downtime for your customers and we even created two different ways to enter the command so a novice technician or customer that might be taking instructions on the phone (cell phone if they are entering this command...) will not be tripped up by the traditional bumpy case commands of a CLI.

A typical NBX100 with 64mb of memory can run a savecore in less than three minutes !!!

Savecore writes a new directory to the disk based on the time of your savecore. For example, you would see a new directory in /htfs0/Core_Jan09_03_35pm. In this directory, you will need to copy all the files for development to analyze.

Files in this directory are:

nboss.inf
nboss.cor
adminlog.txt
NBossLog.txt
upglog.txt
All OldNBossLog's

 

[petenutz]

Brilliant! Thank you. Do you know how to enter the Nboss> mode on the v3000? I'd like to do a memtest.

 

[cknipe]

I dont know off the top of my head...but I've seen memory issues in the past and kinda know what to watch for. If you are running code ver 6.0 or higher, the main page (system identity) shows in real time how much free memory you have in bytes. After a reboot, take a look at that number....it should be pretty far up there (i think at least 1,000,000) that number should eventually balance out somewhere....If you notice that it keeps ticking back and back all the way to 0, there may be a memory issue (i fixed this last time with a simple memory upgrade).

 

[petenutz]

Thanks for the advice. I've been keeping an eye on the memory usage and indeed it loses about 512kB a day. I've been monitoring for about a week. That's interesting that you say you attribute this to bad memory. In my experience I'd call this a memory leak attributed to a software problem and usually bad memory manifests in a hard crash out of nowhere. Does the 512kB loss look like what you saw before?

Thanks again!