Cisco 7200VXR - Serial interfaces drop

[rainman]

Hello,

I am troubleshooting a similar situation on two separate C7206 routers where ALL serial interfaces on a channelized DS3 drop (UP/DOWN) and it takes a reload to bring the interfaces back up.

Both routers have been online for years without problems such as this. We recently upgraded the routers from NPE 300's to NPE-G2's. The routers are also running BGP and part of an ISP network that passes public internet traffic.

Has anybody else run into a situation like this on their network(s)? These 7206's just started acting up about 2 months ago and it happens about twice a week. It takes a reload to cause the interfaces to go to an UP status and stay UP/UP.

I even went as far as to perform this test while the router was experiencing the problem:

Create a T1 on the channelized DS3 then set it up for "local loopback", which would normally force the Serial interface to be UP/UP (Looped). The interface goes UP/UP for 5 secs, then immediately drops. After I reloaded the router, all the serial interfaces went UP/UP, including the one that I created and the loop was still there and now active.

We have alot of 7200's and only 2 of them are doing this, but coincidentally both are running BGP. Are there any BGP exploits that I need to be concerned about that can cause Serial interfaces to drop?


Thanks,

Rainman

 

[vipergg]

I would go to cisco CCO and look for IOS bugs relating to your problem , this sounds like a code issue .

 

[rainman]

Sadly enough, I have already worked with TAC on it for about 2 weeks on and off, and they can't come up with anything. Due to the repeated issues with it and sensitivity, I pursued an RMA for the entire chassis/port adapters. I will know if it's a hw/sw prob in about 2 days....


-Rainman

 

[castlesmadeofsand]

Sounds like the T1 card to me...

Tim

 

[rainman]

Well, I worked with Cisco TAC and they have not come up with any resolutions. I persued an RMA w/ them and I am swapping the entire box/cards out tonite with a new one.

Half of me hopes this works and I find out it's a hardware issue but I have a feeling that Cisco has an un-identified bug with their new NPE-G2 module. This 7206 had an NPE-G1 in it and it had no problems.

Again, I have ANOTHER 7206 on my network with an NPE-G2 and it also acting funky. I bet it's bug related...

Also when the router boots up if I watch on the console port I get an error before the IOS loads. This isn't the exact error but you guys get the idea:

Unable to determine ifindex for T1 controller 2/1

I get this for every T1 i've created on the channelized DS3... Including on the NEW chassis they shipped me. I really think it's a 12.4 bug w/ the NPE-G2...

Just wondering if anyone else is running 7206's with NPE-G2's and a channelized DS3 that has seen this. Cisco hasn't been much help...


-Rainman

 

[rainman]

The issue has been identified:

Bug id: CSCsd44501

The line protocol flaps for PA CT3,CE3, MC-8TE1 with any serial encaps configuration. This condition occurs only with following midplane revision chassis:

-CISCO7206VXR (midplane) HW version less than 2.8 - 800-04667-11

-CISCO7204VXR (midplane) HW version less than 2.8 - 800-04766-11.

There are no known workarounds.

So everybody - if you have a C7206VXR and plan on upgrading to NPE-G2's you might want to hold off unless your router chassis is a ver 2.1 with "show version".

Now I have to downgrade to the old NPE300/400's that were in it. Cisco says that they've shipped RMA routers w/ a hw ver of 2.1 to their customers and it's fixed the issue so this is the route i'm going down.


-Rainman

 

[castlesmadeofsand]

Good post. Thhanks for the info.

Tim