No inside or outside calls after reboot - HELP!

[vyrys]

rebooted the Merlin Magix after needing to replace batteries in UPS.

Now I cannot make or receive any outside calls... just get a fast busy.
Could it be the 100DCD card?

Any assistance would be greatly appreciated.

 

[vyrys]

Here is error log

A Last 99 System Errors:

A Message ss/pp Cnt First Last Code
A POOL BUSY &/OR OOS 00/01 - - 05/07 14:59:17 4C03
A PRI SVC STATE INCONSIST 12/01 - - 05/07 14:59:20 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 14:59:27 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 14:59:36 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 14:59:43 7002
A POOL BUSY &/OR OOS 00/01 - - 05/07 14:59:56 4C03
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:00:00 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:00:01 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:00:18 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:01:36 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:08 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:17 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:25 7002
A PRI SVC AUDIT TIMEOUT 12/02 - - 05/07 15:02:27 7001
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:33 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:41 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:50 7002
A PRI SVC AUDIT TIMEOUT 12/03 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/04 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/05 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/06 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/07 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/08 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/09 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/10 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/11 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/12 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/13 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/14 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/15 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/16 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/17 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/18 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/19 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/20 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/21 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/22 - - 05/07 15:02:52 7001
A PRI SVC AUDIT TIMEOUT 12/23 - - 05/07 15:02:52 7001
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:02:58 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:03:23 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:03:35 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:03:46 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:04:06 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:04:26 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:04:35 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:05:14 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:06:50 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:07:28 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:07:52 7002
A PRI SVC STATE INCONSIST 12/01 - - 05/07 15:08:03 7002
A PRI SVC STATE INCONSIST 12/02 - - 05/07 15:08:03 7002
A POOL BUSY &/OR OOS 00/01 - - 05/07 15:08:33 4C03

 

[Telecomboy]

Your PRI is down. Possibly you turned up the cabinets in the wrong order. You need to power them on right to left giving 1-2 seconds each before powering the next one on.

 

[vyrys]

I did try the following:

Reboot (in the order you specified) 4 times.
Checked with our PRI service provider (Time warner) they state all is fine on their end.
Board renumber

so far nothing is working.
All was fine till the reboot.

could the 100DCD card be bad?

 

[Telecomboy]

Do you have a red light on the 100DCD card? Those errors above show the Magix is not seeing the PRI signal properly from the carrier. Also check your permanent and transient errors to see if you see any other errors that point to the issue. Try unplugging the T1 cable from the 100DCD and plugging it back in too. That can help resynch the circuit.

 

[vyrys]

No red light
If I unplug the T1 cable (RJ45) from the 100DCD card a red light comes on. It will disappear shortly after replugging it.
I also resocketed the card just for giggles.


below are the additional errors
Permanent Errors:

A Message ss/pp Cnt First Last Code
A E911 OVERFLOW 00/00 039 12/14 02:33:58 05/07 13:46:51 0802
A ABK EXTENSION BUSY 00/00 002 05/12 02:00:00 04/28 02:00:00 0404
A PRI SVC AUDIT TIMEOUT 12/01 228 02/10 15:50:39 05/07 13:15:27 7001
A PRI SVC AUDIT TIMEOUT 12/02 234 02/10 15:50:39 05/07 15:15:18 7001
A ERROR LOG


A PRI SVC AUDIT TIMEOUT 12/03 234 02/10 15:50:39 05/07 15:18:42 7001
A PRI SVC AUDIT TIMEOUT 12/04 234 02/10 15:50:39 05/07 15:18:43 7001
A PRI SVC AUDIT TIMEOUT 12/05 234 02/10 15:50:39 05/07 15:18:44 7001
A PRI SVC AUDIT TIMEOUT 12/06 234 02/10 15:50:39 05/07 15:18:45 7001
A PRI SVC AUDIT TIMEOUT 12/07 234 02/10 15:50:39 05/07 15:18:46 7001
A PRI SVC AUDIT TIMEOUT 12/08 234 02/10 15:50:39 05/07 15:18:47 7001
A PRI SVC AUDIT TIMEOUT 12/09 234 02/10 15:50:39 05/07 15:18:48 7001
A PRI SVC AUDIT TIMEOUT 12/10 234 02/10 15:50:39 05/07 15:18:49 7001
A PRI SVC AUDIT TIMEOUT 12/11 234 02/10 15:50:39 05/07 15:18:50 7001
A PRI SVC AUDIT TIMEOUT 12/12 234 02/10 15:50:39 05/07 15:18:51 7001
A PRI SVC AUDIT TIMEOUT 12/13 234 02/10 15:50:39 05/07 15:18:52 7001
A PRI SVC AUDIT TIMEOUT 12/23 239 05/13 15:18:35 05/07 15:19:02 7001
A PRI SVC AUDIT TIMEOUT 12/22 239 05/13 15:19:01 05/07 15:19:01 7001
A PRI SVC AUDIT TIMEOUT 12/21 239 05/13 15:19:04 05/07 15:19:00 7001
A PRI SVC AUDIT TIMEOUT 12/20 239 05/13 15:19:07 05/07 15:18:59 7001
A PRI SVC AUDIT TIMEOUT 12/19 239 05/13 15:19:12 05/07 15:18:58 7001
A PRI SVC AUDIT TIMEOUT 12/18 239 05/13 15:19:19 05/07 15:18:57 7001
A PRI SVC AUDIT TIMEOUT 12/17 239 05/13 15:19:21 05/07 15:18:56 7001
A PRI SVC AUDIT TIMEOUT 12/16 239 05/13 15:19:24 05/07 15:18:55 7001
A PRI SVC AUDIT TIMEOUT 12/15 239 05/13 15:19:26 05/07 15:18:54 7001
A PRI SVC AUDIT TIMEOUT 12/14 239 05/13 15:19:29 05/07 15:18:53 7001

A Transient Errors:

A Message ss/pp Cnt First Last Code
A POWER UP COLD START 01/00 003 05/07 12:08:42 05/07 13:38:59 0016
A COMMAND BUFFER FULL 02/00 009 05/07 12:09:02 05/07 14:08:26 000E
A COMMAND BUFFER FULL 03/00 011 05/07 12:09:02 05/07 14:12:40 000E
A POOL BUSY &/OR OOS 00/02 006 05/07 12:09:47 05/07 14:11:38 4C03
A NO I-VMS PORT IN SERV 00/00 004 05/07 12:10:28 05/07 14:14:00 0C01
A POOL BUSY &/OR OOS 00/01 153 05/07 12:11:04 05/07 15:18:19 4C03
A PRI SVC STATE INCONSIST 12/01 159 05/07 12:11:47 05/07 15:19:30 7002
A INVALID SLOT INTERRUPT 00/00 168 05/07 12:42:13 05/07 14:12:03 0010
A SLOT STREAM CNT EXCEEDED 01/00 105 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 03/00 004 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 07/00 039 05/07 12:42:13 05/07 14:10:16 000C
A SLOT STREAM CNT EXCEEDED 16/00 217 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 04/00 024 05/07 12:42:13 05/07 14:10:02 000C
A SLOT STREAM CNT EXCEEDED 14/00 009 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 02/00 082 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 10/00 165 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 15/00 204 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 12/00 030 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 13/00 160 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 09/00 160 05/07 12:42:13 05/07 14:10:02 000C
A SLOT STREAM CNT EXCEEDED 08/00 094 05/07 12:42:13 05/07 14:10:02 000C
A SLOT STREAM CNT EXCEEDED 11/00 116 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 05/00 245 05/07 12:42:13 05/07 14:10:03 000C
A SLOT STREAM CNT EXCEEDED 06/00 101 05/07 12:42:13 05/07 14:10:02 000C
A TASK RUNNING TOO LONG 00/00 003 05/07 12:43:53 05/07 14:11:30 000F
A SOFTWARE WARM START 00/00 003 05/07 12:43:53 05/07 14:11:30 0004
A ALL TTRs UNAVAILABLE 00/00 091 05/07 12:43:54 05/07 14:11:48 0C03
A COMMAND BUFFER FULL 01/00 004 05/07 12:47:36 05/07 13:39:54 000E
A PRI SVC STATE INCONSIST 12/02 004 05/07 12:57:32 05/07 15:09:18 7002
A SANITY INT NOT GENERATED 01/00 003 05/07 13:39:53 05/07 13:39:55 7802
A ERROR LOG


A INVALID SANITY RESPONSE 01/00 003 05/07 13:39:53 05/07 13:39:55 7804
A CARD INSERTED/REMOVED 06/00 002 05/07 14:07:14 05/07 14:12:03 000B
A SOFTWARE COLD START 00/00 002 05/07 14:07:15 05/07 14:12:04 0003
A POWER UP COLD START 00/00 002 05/07 14:07:29 05/07 14:12:14 000A
A SANITY INT NOT GENERATED 02/00 003 05/07 14:08:25 05/07 14:08:27 7802
A INVALID SANITY RESPONSE 02/00 003 05/07 14:08:25 05/07 14:08:27 7804
A SANITY INT NOT GENERATED 03/00 003 05/07 14:12:39 05/07 14:12:41 7802
A INVALID SANITY RESPONSE 03/00 003 05/07 14:12:39 05/07 14:12:41 7804

 

[Telecomboy]

It sees the PRI which is good. I was looking for red/yellow/blue alarms related to the PRI which would point to a carrier issue. It is possible some of the programming was lost during the power down. Do you have a backup saved on the pcmcia card that you can use to restore the configuration? If you have auto backups turned on it will save one every Sunday night. I would do a restore and see if that takes care of it.

 

[vyrys]

welllll.... (and this is true)
I had just set it to auto backup every sunday at 2am

when I went in today to check, it was turned off.

how do I check my backups to see if they are current?

 

[Telecomboy]

It will save the last 3 you have saved. If one of those is recent enough you can use it to restore. Sys prog-start-system-backup/restore-restore and check to see which files are there to use for restoring.

 

[TouchToneTommy]

Go into Maintenance, Slot, and the slot for the 100D card.
Busy it out, Reset the slot, and Restore it.
See if that will make it start working again.

 

[vyrys]

Can i reset the slot without a restore?

I really can't make heads or tails out of what the backups are. They are named odd things, and have no recent dates on them.
There is an Auto.back1 and Auto.back2 but I think they are older.

I do have recent .ams files that I have backed up to a computer. Are they the same type of system data as the the autobackups?

 

[Telecomboy]

Those are the defaults so you don't have a backup on the system. You can restore from the ams file. It is the same info.

To reset a module, first busy it out. Then reset it, then restore it. These are all very quick procedures.

 

[vyrys]

could you give me step by step?
I poked around and just couldn't tell if it was exactly what I needed to do...and I don't want to guess. I'll probably screw it up.
sorry

 

[vyrys]

Here's what I've got:
My 100DCD is in slot 12

Maintenance > Slot > 12 > busy-out (completed)
Maintenance > slot > 12 > reset > Data Port 1, Data Port 2, slot? I'm guessing slot ?
Maintenance > slot > 12 > restore > do you want to continue?

if I say yes, where does it restore from?

 

[vyrys]

ok.. managed to complete all those tasks.

still no dial out or in.

Should I consider the card bad at this point?

 

[Telecomboy]

I would do a print of your PRI and post it here. We can take a look and see if programming looks ok. Not that it is impossible for the card to go bad on a power cycle, but I doubt that is what is going on here.

 

[vyrys]

I've done two restores. One from 3 days ago and one from a month ago.
below is the PRI print out.

A PRI INFORMATION



A Slot 12 Switch: 5ESS

A System: By line

A BchnlGrp #: Slot: TestTelNum: NtwkServ: Incoming Routing:
A 1 12 CallbyCall By Dial Plan

A Channel ID: 7 6 5 4 3 2 1 8 9 10
A 11 12 13 23 22 21 20 19 18 17
A 16 15 14

A Line PhoneNumber NumberToSend
A 813
A 814
A 815
A 816
A 817
A 818
A 819
A 820
A 821
A 822
A 823
A 824
A 825
A 826
A 827
A 828
A 829
A 830
A 831
A 832
A 833
A 834
A 835

A Network Selection Table

A Entry Number: 0 1 2 3
A Pattern to Match: 101**** 10***

A Special Service Table

A Entry Number: 0 1 2 3 4 5 6 7
A Pattern to Match: 011 010 01 00 0 1
A Operator: none none none none none none none none
A Type of Number: I I I N N N N N
A Digits to Delete: 3 3 2 2 1 1 0 0

A Call-By-Call Service Table

A Entry Number: 0 1 2 3 4
A Pattern 0: 0
A Pattern 1: 1
A PRI INFORMATION


A Pattern 2: 2
A Pattern 3: 3
A Pattern 4: 4
A Pattern 5: 5
A Pattern 6: 6
A Pattern 7: 7
A Pattern 8: 8
A Pattern 9: 9
A Call Type: BOTH BOTH BOTH BOTH BOTH
A NtwkServ: No Service No Service No Service No Service No Service
A DeleteDigits: 0 0 0 0 0

A Entry Number: 5 6 7 8 9
A Call Type: BOTH BOTH BOTH BOTH BOTH
A NtwkServ: No Service No Service No Service No Service No Service
A DeleteDigits: 0 0 0 0 0

A Dial Plan Routing Table

A Entry Number: 0 1 2 3
A NtwkServ: Any service Any service Any service
A Expected Digits: 0 3 3 3
A Pattern to Match: 517 738
A Digits to Delete: 0 3 3 3
A Digits to Add: 200 253

A Entry Number: 4 5 6 7
A NtwkServ: Any service Any service Any service Any service
A Expected Digits: 3 3 0 0
A Pattern to Match: 159
A Digits to Delete: 3 3 0 0
A Digits to Add: 443

A Entry Number: 8 9 10 11
A NtwkServ: Any service Any service Any service Any service
A Expected Digits: 0 0 0 0
A Pattern to Match:
A Digits to Delete: 0 0 0 0
A Digits to Add:

A Entry Number: 12 13 14 15
A NtwkServ: Any service Any service Any service Any service
A Expected Digits: 0 0 0 0
A Pattern to Match:
A Digits to Delete: 0 0 0 0
A Digits to Add:
P3F1FIM Tone N  $!Print !(Engli!sh )!: >N  $!Make a! selec!tion ! N  $!All ! !Trunk !Info N  $!SysSet!-up !T1 Inf!o N  $!Dial P!lan !PRI In!fo N  $!Labels! !RmoteA!ccess N  $!Back ! !Oper I!nfo S 

 

[Telecomboy]

So, lines 13-35 are in your pool 70? I would also verify that your service provider is set to Custom 5ESS for switch type and NOT NI2 which is common/default for most.

 

[vyrys]

yes.. 13-35 are in pool 70.

I can have them check. but would they just change all of a sudden? It was working right before the reboot.

 

[Telecomboy]

I have had an instance where the PRI was working fine for years (unknown to me the provider was programmed as NI2) and then I rebooted the system and the PRI would not sych back up. After trying everything with the system I had ATT check that they were programmed Legacy 5ESS and that is when we found out they were programmed as NI2. They made the change to Legacy 5ESS (also can be called Custom 5ESS) and the PRI came back up and started working. They haven't had an issue since.