No Reason for 8710 or MCC Crash and Burn?!?

[Stinney]

While out of state, of course, our phone system had a major failure. There was no dial tone on any set. No incoming or outgoing calls. Could not access 8710 servers using virtual IP, or direct IP addresses. Could not dial into maintenance board.

Both servers were on, MCC had power but was showing major alarm(s).

Our BP wasn't able to get in and it was going to take over an hour for a tech to get on site. I had our IT dept shut the MCC off, wait and then repower the MCC. The system came back up.

Server B is now the active server and appears to have switched at the exact time the power cycle was done.

The regular disp alarm command in ASA doesn't show any alarms until the power was cycled.

I can access server B remotely but can't access server A remotely. IT can from internal connection to network.

Server B logs don't seem to say much, but not sure I'm looking at the right thing.

Server B Restart Log shows restart at the time power was cycled:
Interchange 4 (RELOAD) yes Active 05/01 9:14

Interchange Server report shows Server A's hardware and processes OK.

BP said they can't tell us what happened. Concerned it can happen again and still not know why.

ANY thoughts would be appreciated.

 

[nohuhu]

Stinney,

list hist
disp err
disp event

 

[klmccoy]

Saw something similar recently. Check your IPSI firmware version. There is a known problem with HW2 FW12. The only difference was the system reset itself and did a server interchange.

 

[Spookytooth]

Had a couple of ipsis with FW12. They will reset exactly every 248 days dropping everything. Need to upgrade FW on the ipsis to the latest.

 

[Stinney]

System is almost 248 days from the date of installation.

Is there a report that will show if the IPSIs reset? I've done a disp hist, disp err, disp event. Unfortunately, there is nothing reporting prior to 5/1 when the system was restarted, only when the system was restarted in August when it was installed.

I did a li con all and the system reports the IPSIs as HW12 FW030.

Thanks for all the suggestions.

 

[Spookytooth]

your OK its not the hw number its the fw number, your OK with 30