2400IMS MMG having troubles

[phadobas]

Ok, old system I know. It has 2 stacks. One night, none of the phones would work any more. You pick up your phone, get dial tone and try to dial, but any button you press on your phone, would not break dial-tone. On the alarm display, the system had a red light on "NMC Error".

Big system, immediate action was needed, so the on-site admin put the active PC07 card into make-busy, which activated the standby PC07 on the next shelf. Apparently he found some manual that said PC07 was the Network Management Controller card or something like that. The system came back to life and started working normally. EXCEPT!!!

They have an attendant console (one of those big old ones with a 25-pair cable), which now can't receive calls. It can make outgoing calls, but no calls would terminate on it.

What do I have them look at?
Some more data that could be irrelevant, but I put this down here anyways:
The attendant console is controlled by a CS07 card, which is in the 2nd stack, on shelf 2 (if you start counting shelves with 0).
The PC07 cards are in the 1st stack, shelf 1 and 2 (again counting starts at 0).

 

[belevedere]

Has the PC07 card been reset? I seem to remember that the console can go into night mode in some instances. Try to toggle the night switch. Make sure a headset or handset is plugged in. It will go into nights when removed.

 

[OzzieGeorge]

And when it recovers into night mode, it doesn't light the night mode lamp so it looks as though it is in day mode but isn't, Cycle night mode a couple of times to get things back to normal!

 

[phadobas]

Well, looks like the console problem is solved.

last night the whole system went down. No phones had dial tones, and nothing worked. The on-site guy called me and we were troubleshooting this thing for 3 hours to get it back up.

I'm still trying to understand a few things about it.
It has 4 CP34 cards, which per his manual, is a processor card. 2 of these exist on the second shelf and 2 on the 3rd shelf. Why do you need 4???
When the system was down, only CP34 on the 2nd shelf is slot 6 had a green light, the other 3 cards had no lights on them. I had him push the reset button on this, and the system came up after about 10 minutes of boot time. All appeared to work, but only for ten minutes. Then it went down again.
Next I had him re-seat all 4 of these cards, and they all lost their green light and pushing the reset button on any of them had no effect.

I then had him push the START button on the display panel, but first had him flip the "LOAD" switch to down I think, so it didn't load the system data from the "bubble memory". I think this is the term he found in his manual. Anyways, this time the system went through its whole bootup process again, this time 3 of the 4 CP34 turned green. The one that remained off was the one that was the only green at the time of the crash.

Since then, the system is up and the console can be called. This system is being replaced soon, but I still need it to work till then.

So what's up with 4 CP34 cards? Did we do all-right or we only got lucky?

 

[belevedere]

From what I can remember, that is the proper way to reboot the MMG. A reboot will take that long. The CP-34 is a local processor for that stack. The 4 cards are most likely for redundancy. You might have one bad card.

 

[phadobas]

Ok, so after a few days of running, the system went down again. It needed a reboot again. It is being replaced in the near future but that's still a couple of weeks out and I need to stabilize it somehow until then. I have several hundreds of users on it. There must be a card or part that's causing all this.

I attach the shot of the display panel, and here is the description of what cards are in what alarm states:

The following are the cards with error lights:

LP 0
FIRST SHELF
slot 12 BS06 BUS1 MB ERROR LIGHT
slot 13 BS06 BUS1 MB ERROR LIGHT
slot 16 BS08 MB ERROR LIGHT
SECOND SHELF
slot 8 BS 06 BUS1 MB ERROR LIGHT
slot 14 BS 06 BUS1 MB ERROR LIGHT
THIRD SHELF
slot 00 PA PW 21 CPU ALARM (REST OF LIGHTS ON THIS ARE GREEN)
slot 8 BS 06 BUS1 MB ERROR LIGHT
slot 14 BS 06 BUS1 MB ERROR LIGHT
slot 24 SC06 MB LIGHT (THIS ONE IS MANUAL BECAUSE I DID IT—I TOOK THIS OFF MANUAL MAKE-BUSY AND IT DID NOT CHANGE THE CONDITION OF ANY OF THE OTHER CARDS)

LP 1
FIRST SHELF
slot 24 BS 06 BUS1 MB ERROR LIGHT
slot 25 BS 06 BUS1 MB ERROR LIGHT

LP 2
FIRST SHELF
slot 24 BS 06 BUS1 MB ERROR LIGHT
slot 25 BS 06 BUS1 MB ERROR LIGHT
THIRD SHELF
slot 14 SW 05 MB

The person on site, changed out the CPU card (CP34) on shelf 1, slot 6, that had no light last night. The light did not come on with the new card, and it did not change the conditions of any of the above cards. I did hit the reset button on the card and that did nothing. I did not however reset the whole system.

On the BS 08 card, I do not have a spare of this card to switch it out. I am guessing, but it looks like this might be causing a bunch of the errors. The one with the errors is on bottom shelf of LP 0 in slot 16. There is another one in slot 14 with no errors.


So anybody has a clue what's going on with this system?

 

[belevedere]

I can think of only one person who may remember this old system and may be able to help. I cannot post his information here. How can I get you a private message?

 

[phadobas]

For now you can send a text to area code 727 and then 423 and 0064

 

[belevedere]

Done.

 

[phadobas]

Received and talked to him. I'll have him also talk to the on-site tech. Hopefully he can help. Thank you.

 

[belevedere]

You're welcome. Please post the results. I'd like to know how it turns out.

 

[bengi1234]

Well here's how it turned out:

Found a mis-set BS-08 card that handled the errors on the PBUS path. Found a mis-set SC-06 which handled the error on MP0. I think we had a bad tech from a local company who messed with those settings but that's another story. Also found and replaced a bad power card on the third shelf of LP0.

Everything running smoothly now according to my display board ... except every few days getting a c-level infinite loop that splatters my TDSW and HSW cards and does weird things to my attendant console (you ring and ring but the board never receives the calls although it looks like it's on). Trying to figure out which card this is that causes the C-Level infinite loop but the manual is not much help. I tried replacing my NMC card (SC-17) but that didn't fix it. Any ideas?

 

[OzzieGeorge]

Are you getting fault reports and if you are what are they? I have an old program that apparently interprets the alarms and I could dig it out if it would help!

 

[bengi1234]

Hi Ozzie, thanks I started a new thread on this as I am having a different set of issues and seems like I handled part of these ones.

 

[belevedere]

C-Level infinite loop - Periodically the system send a "keep alive" message to each circuit card and port. A C-Level infinite loop means that the message timed out and a proper response was not received. This could be a bad port or card or microcomputer chip anywhere. The error message should have the location of the faulty unit on it. I believe Ozzie is referring to an old DOS program that interpreted the alarm for you so you did not have to try and figure it out with the manual.