MICS keeps going down 1

[cpds]

This is our 2nd MICS at this location; it has XC7.1 SW, PRI, DTI, 6 port combo, 6 port expansion with 14 ASM's. This unit will freeze up sometimes once a week. Time and date will be on the phones but no intercom dial tone and none of the buttons will work. Here are the EVT and ALM codes we are getting. The provider says they see no errors on their end on the circuit. ANY help on this issue would be greatly appreciated. We are at a loss.
SYS TEST LOG
10/25/22
11:30 SYS STARTUP
11:30 EVT 299
11:30 EVT 462-01610101
11:32 EVT 367-0001 S5
11:33 ALM 372-0
11:34 ALM 50-09 3
11:36 EVT 680-1 S9
12:10 EVT 310-3C0005E5
12:10 EVT 799-000301S7
14:11 EVT 904-0108003C
14:13 EVT 310-36000200
15:16 EVT 310-3C0005E5
10/26/22
09:50 EVT 310-63000200
10:22 EVT 310-3C0005E5
10:22 EVT 310-36005A00
12:09 EVT 310-3C0005E5
12:10 EVT 799-000301S7
12:10 EVT 799-000401S7
12:10 EVT 310-3C0005E5
12:11 EVT 799-000401S7
14:20 EVT 904-0108003C
14:21 EVT 799-001703S7
10/27/2022
04:20 EVT 462-01610101
04:22 EVT 367-0001 S5
04:25 ALM 50-09
04:27 EVT 680-01 S9
04:27 UTAM CODE REQ'D
05:34 EVT 299
05:34 EVT 462-01610101
05:36 EVT 367-0001 S5
05:37 ALM 372-0
05:38 ALM 50-09
05:40 EVT 680-01 S9
05:40 UTAM CODE REQ'D

NETWORK EVT LOG
10/25/2022
11:33 EVT 372-0
10/27/2022
05:37 EVT 372-0 6 TIMES
Thanks for your help.....

 

[curlycord]

Tough one but there seems to be more DTI card alarms overall, and a Clocking alarm.

I would start with replacing the DTI card and Combo Card.

Did you install an NVRAM R2 when it was upgraded to 7.1? if not then just do that first.

Maybe double check programming for PRI but I thinks it's more hardware related.
Did you restore programming when you swapped KSU's or program from scratch?
If it were a bug then then maybe the bug was copied over to the new KSU.

Normally you would replace the parts before the KSU.

=----(((((((((()----=

http://www.curlycord.com

Toronto, Canada

Add me to LinkedIN

 

[curlycord]

libellis (TechnicalUser)28 Oct 22 20:38
One can hope - maybe the solution is as simple as it was in this thread (see last post in thread).

[URL unfurl="true"]https://www.tek-tips.com/viewthread.cfm?qid=167112[/url]

-------

Funny, I say tough one in that post too, lol

cpds, do you have the full alarm pdf?

Here is some of your alarms related to the DTI...

310: Dti_cart_event P5 'B' TestLog
310: | CAUSE:
310: | 1)Bad Stimulus message received by dti from KSU
310: | Parameters are: Source file (eCpQinpEh {value 30})
310: | error number (0)
310: | 2)Bad encapsulated Stimulus 2 byte message received by dti
310: | from KSU. Parameters are: Source file (eCpQinpEh {value 30})
310: | error number (4)
310: | 3)Bad encapsulated Stimulus multibyte message received by
310: | dti from KSU. Parameters are: Source file
310: | (eCpQinpEh {value 30}) error number (4)
310: | ENVIRONMENT:
310: | 1)Occurs on DTI cartridge only.
310: | 2,3)Occurs on DTI cartridge only.
310: | HUMAN ACTION:
310: | Check DTI connections. If problem persists call
310: | operating company.


372: CC_MSTR_CHG P2 'E' NetLog TestLog
372: | CAUSE:
372: | This logevent is generated whenever the clock
372: | control server switches the clock master, i.e.
372: | from primary to secondary and vice versa or
372: | tracking is re-established on one of the clock
372: | sources.
372: | The parameter indicates the new clock source :
372: | 0 - Tracking on primary
372: | 1 - Tracking on secondary
372: | ENVIRONMENT:
372: | This event can occur for a variety of reasons.
372: | The intent is to provide indication that tracking
372: | has changed or re-established. This is important
372: | for data applications.
372: | HUMAN ACTION:
372: | Unless the problem is due to the customer, e.g.
372: | unplugging the digital link, the customer should
372: | contact the installer.

799: ISDN_CP_Error P7 'B' Trace TestLog
799: | CAUSE:
799: | A call processing error has occured on an ISDN line.
799: | ENVIRONMENT:
799: | The first parameter identifies the location (line number
799: | or loop number). The second parameter is an error code,
799: | defined as follows:
799: | 1: Can't acquire the B-channel from the B-channel Arbitrator.
799: | This indicates an internal software error.
799: | 2: There is no free line available for the call (internal error).
799: | 3: A non B-channel call has been attempted. Such calls are
799: | not supported in this version of software.
799: | 4: Failed to instantiate on the chain (internal error).
799: | 5: Activation procedure failed (internal error).
799: | 6: Index conversion failed (internal error).
799: | 7: Unexpected digits on Manual Answer line. Possible
799: | Network / Norstar configuration mis-match.
799: | 8: Can't seize CO line on a BRI (internal error at the
799: | driver or card level).
799: | HUMAN ACTION:
799: | None.


904: qERR_In_tLD P3 'B' Trace TestLog
904: | CAUSE:
904: | A logevent has been caused in the ISDN loop driver or broadcast
904: | call handler by the receipt of unexpected input.
904: | 1st parameter is an error sub-code (see source code).
904: | Remaining bytes are optional parameters passed to the error call.
904: | See the source code at the location of the error. call.
904: | ENVIRONMENT:
904: | ISDN loop driver or broadcast call control receiving input.
904: | HUMAN ACTION:
904: | Determine input that caused event and state of the ISDN
904: | driver when the call occured. Take steps to fix
904: | incorrect event or have driver accept it.

=----(((((((((()----=

http://www.curlycord.com

Toronto, Canada

Add me to LinkedIN

 

[cpds]

Thanks Curlycord, we did replace the complete system and programmed from scratch. We had the old system at our shop plugged in for months and it never went down but do not have a circuit connected. I did get an email from the provider late yesterday saying they are investigating some issues on their circuit. Is there a possibility that could be the issue?

 

[jsaad]

Are you using a CSU/DSU?

 

[curlycord]

When it freezes you would think software or hardware.

If software (KSU) is talking to software (carrier), and the carrier software is disturbed, then it might cause Nurse Ratched to perhaps administer a dose of unknown liquid in a tiny cup.

If hardware related then it would be:
-Carriers Pair Gain box
-DTI patch cable
-DTI card
-Combo or Services card
-NVRAM cartridge
-Software

If the ALL new hardware acts the same way then you would lean to the battle of the softwares.

Compare PRI settings with KSU and Carrier [URL unfurl="true"]https://www.tek-tips.com/faqs.cfm?fid=1646[/url]

=----(((((((((()----=

http://www.curlycord.com

Toronto, Canada

Add me to LinkedIN

 

[cpds]

jsaad I'll check but normally leave internal CSU: on. Sys is down again as of 9:00pm spectrum is saying Please check power to pbx and or cabling we are seeing d channel down on the audiocode. I can't login so I will have to contact someone on site after 8:00 this morning.

 

[cpds]

curlycord I have ordered a DTI card and I will be on site when it comes in to replace, I have another combo card.

 

[cpds]

Update: I replaced the DTI card and the T1 cable on 11/4/22. I have been monitoring the system and so far everything looks good no alarm codes. Thanks for the help curlycord I'll keep monitoring the system for another week.

 

[curlycord]

Nice, thanks for the update and looking forward to the next one.

=----(((((((((()----=

http://www.curlycord.com

Toronto, Canada

Add me to LinkedIN

 

[cpds]

System went down again yesterday and again this morning. I called Spectrum and they had no alarms but did see in their log where the system had disconnected from the circuit. I guess I'll try replacing the Combo card and see what happens.

Sys Test Log
11/8 12:45 EVT 221-7642 I remoted into system
11/8 13:08 EVT 310-36000200
11/8 15:08 System Startup
11/8 15:08 EVT 299 S1
11/8 15:08 EVT 462-01610101
11/8 15:10 EVT367-0001 S5
11/8 15:11 ALM 322-0
11/8 15:13 ALM 50-90
11/8 15:15 EVT 680-01 S9
11/8 15:15 UTAM CODE REQ'D
11/9 04:08 EVT 310-36000200
11/9 04:39 EVT 247-01370263

SYS ADMIN LOG
11/8 12:45 EVT 220-7642 S4

NETWORK EVT LOG
11/8 15:11 ALM 372-0

 

[cpds]

System went down twice again today, what happens is they have time and date on the phones no intercom dial tone and none of the buttons work. I'm going to replace the system again and see what happens. Thinking it may be a power issue I have plugged the main cabinet into an outlet by itself and all the asm's work off of power strips. It's a tough one but any suggestions would be a greatly appreciated.

 

[cpds]

I replaced the MICS and all cards and it went down within 2 hours. If anyone knows someone or company I could call to have them check the programing, I have installed many of the MICS's but maybe I'm missing something. WE don't have anyone in are area that works on Nortel anymore.

 

[curlycord]

See my link to the FAQ about comparing.

Post here what the MICS has and what the Carrier has, ask them, go through that list with them.

Maybe check there is proper ground at the receptacle.

Did you change the software card too?
If so, did you do a restore from a backup or program from scratch?

Did you change the NVRAM cartridge? is it Rls 2?

Has the carrier bothered to replace their pair gain box (or it's card) at site? or just blame the system?

=----(((((((((()----=

http://www.curlycord.com

Toronto, Canada

Add me to LinkedIN

 

[cpds]

Complete system replacement programed from scratch, this system keep going down so much I put the other one back in. Spectrum is set at Per our engineer;
We are set to

NI2
framing: T1_FRAMING_ESF_CRC6
line-code: B8ZS
clock-master: CLOCK_MASTER_ON

My settings match with clock set to primary, all 23 channels are working and provisioned.
I will check the grounding and check all extensions for external voltage.
Thanks for the input..

 

[libellis]

In case it's some sort of intermittent or noisy power problem, can you put the system on a UPS to see if that resolves the issue? Even before that, checking the mains voltage and also checking with one of those small plug in testers (with the three lights) that the outlets are wired properly will at least eliminate two power related factors. I'm having some difficulty understanding how a faulty link to the Carrier would be the cause of all the symptoms you're describing, such as buttons and intercom not working so even internal calls can't be made.

 

[cpds]

libellis thanks for the response, I have checked all the grounding and voltage today they tested good. I also tested all extension jacks for foreign voltage coming into the system there was none. My next step is to eliminate all the peripherals, station module and 15 ASM's to see if that's the culprit. One side note Spectrum did come in today and replace cable modem and add a splitter and now the system is going down more often.

 

[cpds]

I hope everyone has a happy new year. Looks like I'll be starting the new year out with the same ongoing MICS issue. Here's some of the troubleshooting we have done the past two months.
Replaced DTI and Comb card in sys #1
Replaced T1 cable
Replaced Station mod
Replaced 25 pair amphenol cable to station mod (found two bad pairs going to PP).
Have 15 ASM's but only 7 working so was able to move the ASM's around to eliminate that being the issue.
Tested all extensions for foreign voltage found none.
Had system on UPS moved to regular outlet then to backup generator outlet made no difference.
Tested outlets for grounding and voltage test good.
Replaced complete sys #2 MICS, DTI, Cartridge R2, with XC7.0, 6 port comb ( sys went down while installing so put sys #1 back in.

Spectrum has replaced the cable modem and the gateway we are still testing with them.

I have 9 Nursing Homes all with Nortel MICS's with the same programming setup. I have been monitoring some of the other systems. They are getting the same event code's 310-xxxxx and more but are not shutting down like this one. I am working on getting a system that has been working in a nursing home and was replaced.

 

[libellis]

Out of curiosity, is the Spectrum equipment located in the same area as the MICS, or is there a significant distance between them? What type of T1 cable is being used (shielded or unshielded)? Is signal ground carried in the cable pairs?

 

[cpds]

libellis thanks for the response, the gateway and the MICS are only about 6' apart. I used CAT6 cable unshielded but Spectrum replaced mine and I don't know what they used. The on site contact that is rebooting the system for me is saying that the DTI card is green and the gateway has a green light. I think the circuit maybe up but still could be causing an issue with the system. However it could be something with the system that's my dilemma..... Thanks