Cisco WIC-1DSU-T1 line code question.

[Ironman2401]

I have a really wierd problem happening with one of my T1 lines and I have no idea why it is happening. We recently had another point to point T1 installed by Verizon and it was plagued with problems from the start, but Verizon finally cleared up all the problems, except for one which they insist that is my problem. The line will go down every so often without explination, and when it is up the amount of CRC and framing errors generated is quite large. The main location is fine, it is the remote side that is having the problem. Both routers are Cisco2621XM routers with IOS 12.2T and a WIC-1DSU-T1 module. I've swaped the routers around and the problem still exists at the main site. The only thing I havn't changed is the encapsulation from HDLC to PPP, but as far as I know that shouldn't effect the line code. The Verizon techs are saying that the CSU is switching for B8ZS to AMI linecoding during idle times. I've never heard of this before, but it would explain the errors that I'm getting. Any help is greatly appreciated, and if you need any more info I can get it off the routers easy.

edit: and yes, the linecode showing on the smart jack is B8ZS

Main site diagnostics:
show int S0/0

Serial0/0 is up, line protocol is up
  Hardware is PQUICC with Fractional T1 CSU/DSU
  Description: connected to hpmcore3
  Internet address is 10.21.1.1/30
  MTU 1500 bytes, BW 1544 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:02, output 00:00:00, output hang never
  Last clearing of "show interface" counters 2d09h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/300/0 (size/max total/threshold/drops)
     Conversations  0/23/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1158 kilobits/sec
  5 minute input rate 0 bits/sec, 0 packets/sec
  5 minute output rate 1000 bits/sec, 1 packets/sec
     915236 packets input, 224116948 bytes, 0 no buffer
     Received 24341 broadcasts, 0 runts, 0 giants, 0 throttles
     0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort
     1113478 packets output, 324302933 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up

show service-module

Module type is T1/fractional
    Hardware revision is 0.104, Software revision is 0.2,
    Image checksum is 0x70F47262, Protocol revision is 0.1
Receiver has no alarms.
Framing is ESF, Line Code is B8ZS, Current clock source is line,
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.
Last module self-test (done at startup): Passed
Last clearing of alarm counters 2d10h
    loss of signal        :    0,
    loss of frame         :    0,
    AIS alarm             :    0,
    Remote alarm          :    0,
    Module access errors  :    0,
Total Data (last 96 15 minute intervals):
    0 Line Code Violations, 0 Path Code Violations
    0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
    0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs
Data in current interval (666 seconds elapsed):
    0 Line Code Violations, 0 Path Code Violations
    0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
    0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

Remote End Diagnostics

show int S0/0

Serial0/0 is up, line protocol is up
  Hardware is PQUICC with Fractional T1 CSU/DSU
  Internet address is 10.21.1.2/30
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 1/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 2d10h
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/300/0 (size/max total/threshold/drops)
     Conversations  0/9/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1152 kilobits/sec
  5 minute input rate 1000 bits/sec, 1 packets/sec
  5 minute output rate 0 bits/sec, 0 packets/sec
     1115153 packets input, 324570262 bytes, 0 no buffer
     Received 24400 broadcasts, 1 runts, 166 giants, 0 throttles
     17411 input errors, 1442 CRC, 15963 frame, 0 overrun, 0 ignored, 6 abort
     916480 packets output, 224385248 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up

show service-module

Module type is T1/fractional
    Hardware revision is 0.104, Software revision is 0.2,
    Image checksum is 0x70F47262, Protocol revision is 0.1
Receiver has no alarms.
Framing is ESF, Line Code is B8ZS, Current clock source is line,
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.
Last module self-test (done at startup): Passed
Last clearing of alarm counters 2d10h
    loss of signal        :    0,
    loss of frame         :    0,
    AIS alarm             :    0,
    Remote alarm          :    0,
    Module access errors  :    0,
Total Data (last 96 15 minute intervals):
    3657 Line Code Violations, 1965 Path Code Violations
    0 Slip Secs, 170 Fr Loss Secs, 376 Line Err Secs, 199 Degraded Mins
    199 Errored Secs, 109 Bursty Err Secs, 178 Severely Err Secs, 0 Unavail Secs
Data in current interval (198 seconds elapsed):
    0 Line Code Violations, 0 Path Code Violations
    0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
    0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

 

[dx1]

Clock source should be line on one side and internal on the other.

 

[BuckWeet]

the CSU switching from B8ZS to AMI is a load of crap.. I'd check the timing thing like dx1 mentioned. If you're still getting those errors, the problem is definately a telco issue, especially if its at one side.

I've had several T1's that have one leg bad.


BuckWEet

 

[Ironman2401]

Well I changed to clock source to internal on the remote side as you can see below, and it worked fine for about 12 hours and then started picking up errors as you can see below. The link never went down, but I shouldn't be getting all these errors. I agree that the explination that they gave me was crap, but they refuse to come on site anymore because they say the problem is deffinatly on my end. Any more ideas, and specific test I can tell the telco to run to make them realize its their problem? I'm desperate for help right now.

Remote side: show service-module

Module type is T1/fractional
    Hardware revision is 0.104, Software revision is 0.2,
    Image checksum is 0x70F47262, Protocol revision is 0.1
Receiver has no alarms.
Framing is ESF, Line Code is B8ZS, Current clock source is internal,
Fraction has 24 timeslots (64 Kbits/sec each), Net bandwidth is 1536 Kbits/sec.
Last module self-test (done at startup): Passed
Last clearing of alarm counters 13:28:50
    loss of signal        :    0,
    loss of frame         :    0,
    AIS alarm             :    0,
    Remote alarm          :    0,
    Module access errors  :    0,
Total Data (last 54 15 minute intervals):
    1080 Line Code Violations, 597 Path Code Violations
    0 Slip Secs, 40 Fr Loss Secs, 129 Line Err Secs, 14 Degraded Mins
    85 Errored Secs, 57 Bursty Err Secs, 45 Severely Err Secs, 0 Unavail Secs
Data in current interval (21 seconds elapsed):
    0 Line Code Violations, 0 Path Code Violations
    0 Slip Secs, 0 Fr Loss Secs, 0 Line Err Secs, 0 Degraded Mins
    0 Errored Secs, 0 Bursty Err Secs, 0 Severely Err Secs, 0 Unavail Secs

Remote side: show int S0/0

Serial0/0 is up, line protocol is up
  Hardware is PQUICC with Fractional T1 CSU/DSU
  Internet address is 10.21.1.2/30
  MTU 1500 bytes, BW 1536 Kbit, DLY 20000 usec,
     reliability 255/255, txload 1/255, rxload 4/255
  Encapsulation HDLC, loopback not set
  Keepalive set (10 sec)
  Last input 00:00:00, output 00:00:00, output hang never
  Last clearing of "show interface" counters 13:32:48
  Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0
  Queueing strategy: weighted fair
  Output queue: 0/1000/300/0 (size/max total/threshold/drops)
     Conversations  0/9/256 (active/max active/max total)
     Reserved Conversations 0/0 (allocated/max allocated)
     Available Bandwidth 1152 kilobits/sec
  5 minute input rate 26000 bits/sec, 3 packets/sec
  5 minute output rate 3000 bits/sec, 3 packets/sec
     552545 packets input, 329946551 bytes, 0 no buffer
     Received 5687 broadcasts, 0 runts, 27 giants, 0 throttles
     4236 input errors, 449 CRC, 3709 frame, 0 overrun, 0 ignored, 78 abort
     442428 packets output, 96505419 bytes, 0 underruns
     0 output errors, 0 collisions, 0 interface resets
     0 output buffer failures, 0 output buffers swapped out
     0 carrier transitions
     DCD=up  DSR=up  DTR=up  RTS=up  CTS=up

 

[Ironman2401]

if this helps at all this is the configuration of the Serial interfaces on both routers:

Main site

interface Serial0/0
 description connected to hpmcore3
 ip address 10.21.1.1 255.255.255.252
 ip pim dense-mode
 fair-queue 300 256 0
 service-module t1 remote-alarm-enable

Remote site

interface Serial0/0
 ip address 10.21.1.2 255.255.255.252
 ip pim dense-mode
 fair-queue 300 256 0
 service-module t1 clock source internal
 service-module t1 timeslots 1-24
 service-module t1 remote-alarm-enable

 

[Ciscon]

Make your self a hard loop, and plug it right off the smart jack. The other side should come up as up/up looped. If you continue to see errors, do the same on the opposite side. If you continue to see errors, then the problem is on the telco's network.

If the errors stop then the problem could be on your end. If they do ask them to give you a loop from the A end CO to the A end. Then also check the Z end CO to the Z end. The A CO to Z end. And last but not least Z end to A CO.

You'll drive them nuts, but you'll find where the problem is.

 

[Ironman2401]

Someone recommended using the cablelength long command. How exactly do i use the cablelength command? All I have is a Serial0/0 interface, no controller to configure.

config t
int S0/0
service-module t1 cablelength long
% Invalid input detected at '^' marker.
service-module t1 ?
  clock                Service module clock source
  data-coding          Service module DCE line coding
  fdl                  Specify the FDL standard for a DS1 data link
  framing              Framing format
  lbo                  Line Build Out format
  linecode             Line coding format
  remote-alarm-enable  Remote/Yellow alarm enable
  remote-loopback      Remote Loopback/RL disable
  timeslots            Fraction timeslots

does my version of IOS not have the cablelength command?

 

[commsguy]

if the smart jack is on premise and close by, this is not necessary

commsguy

http://www.convetech.com

 

[IllegalOperation]

Those errors could result from a physical media problem.

Stupid question, but did you try replacing the jumper from the serial interface to the demarc? If its too long of a run to replace - did you run a test to check the SNR, attenuation, continuity, etc on the cable?

Ive had a similar scenario, where the problem ended up being a link with too much noise. It was ran across a water heater, and along some high powered lights. Problem went away after re-running the line...

 

[tmt7734]

I've seen similar problem due to wiring problems. How far is the router from the demarc? Are there and transformers or flourescent lights near the router?

 

[idontread]

Ironman2401>

The errors shown under 's serv' and 's int s0' etc are 'generally' associated to trouble on the facilities (the entire T1 span). There are 3 basic things to narrow down the trouble to. (1) facilities (or the span), (2) demarc/extended demarc (IW) and (3) serial port/wic on the router/CSU

Place a trouble call to verizon (or whoever you service provider is) and have them do an 'extended' stress (or intrusive) test. That test will take one hour atleast (with 4 patterns to the NIU).

The T1 cct is terminating onto an NIU from where its extended to the router/csu location. By having ILEC stress to NIU, you 'll know if the trouble is on the facilities or if its clean upto NIU. if test is clean to NIU, then problem lies within your prem. Next thing to check is the IW and if possible, request your ILEC to stress all the way to the CSU. neways, after the facilities, next thing to check is IW (inside wiring). A tech with a t-bert at the NIU and demarc would be able to isolate the trouble.

3rdly, it could be the router/wic/s-port but I doubt it. also ask your ILEC to check and veriy and report back to you with what is the PEAK-2-Peak voltage level on the T1. ideally, it should be 5.0 v but 4.x is fine. anything less, I raise hell with my ILEC till they bumb it up to a comfy level.

Hope all this helps