Problems with Cisco SFPs

[CiscoHelpForMe]

Hello All,

I'm having one hell of a time with this problem and it's driving me crazy..

Hardware: 4 Cisco 3750x switches in 2 stacks of 2. Each switch has a 10GB (SFP-10GB-SR)
which is connected to its oppisite stack mate for a total of 2 10gb links. Theses two links
are configured in a port channel (20GB total).

Layout:
-------------
SWITCH1A--1ft/.5m_Patch_Cord--patchpanel--460ft/140m_Verticle_Cable_Run--patchpanel--3ft/.5m_Patch_Cord--SWITCH2A
SWITCH1B--1ft/.5m_Patch_Cord--patchpanel--460ft/140m_Verticle_Cable_Run--patchpanel--3ft/.5m_Patch_Cord--SWITCH2B

Problem:
-------------
The 10GB interfaces will not come active across the 140m cable. If I connect Switch1B to 2B
Directly via the patch cables, the ports come active and work as expected. If I install a 1GB
SFP across the 140m link it also works. The 10GB Interfaces however never come up, are not errordisabled
or have any other abnormality.

What I have done to fix:
--------------------------
Reboot all Switches
Replace All Patch cables
Move SFP to another port
Shut/Noshut Actual interfaces and Portchannel
Remove interfaces from portchannel, then above again.

All Cables are OM3 Spec, are brand new and have been tested with a OTDR, fluke meter
and have very minimal DB loss (1.1db+/-). All cable connections pass with excellent results.
But the 10GB SFPs still fail to come active across the 140m cable.

The OM3 spec and these SFP's (Orig. Cisco) are rated from 300m which is double the
length of the installed cable.


Can anybody help me with this before I have to have myself committed???

 

[ADB100]

I'd suspect faulty SFP+'s - Concentrate on a single link and move the 4 SFP+'s around.
I had an issue recently where one 1000BaseSX SFP was faulty - it initially looked like a fibre cabling issue as the link would come up when the cable was partially removed (i.e. not 'clicked in'). This was obviously a manufacturing defect with either the RX or TX optic inside the SFP. This SFP also worked on a short connection.
It sounds unlikely but I get the impression you have tried lots already...

Andy

 

[CiscoHelpForMe]

Thank you for the reply.

The SFP is not faulty, as I can connect the 2 switches 10gb ports back to
back with a patch cable and it works just fine with no problems. I have tried
to get a single link working across the 140m cable but it does not work,
it does work via a 1m patch cable.

All cables are firmly connected into their ports (1st thing i checked )
I have 5 10GB SFPs The same happens with all 5.

 

[VinceWhirlwind]

Are all the patch cables using LC connectors?

I've had words with cable installers who fail to put a X in LC - LC structured cabling. Very annoyed I was.

On one of the non-working links, remove one end of the patch cable, split the LC connector into its two halves and then patch them back in opposite to how they were when joined.

 

[chieftan]

With the 3750, is it a 48 port or 24?

Okay, lets assume it is a 48 port.... that means, from a fiber perspective, the SFP's are being installed into either 45,46,47 or 48 on each switch. When you say you utilised a patch cable (Cat5e or Cat6) it worked. So, this tells me that you either installed an RJ45 module into the slot where the Fiber module is located or you used one of the other ports of the switch.

If you used one of the other ports of the switch then the config could be wrong on the Module ports.

Have you tried changing the actual Module?

Can you post the configs of the ports you are conencting the SFPs to?

I am not sure about the 3750's as I have not had to configure one (although we are using 3750's it is a managed service and therefore cannot access the switches), but I do know on some switches you have to tell the port to accept the 10gb link as they are originally configured for 1gb - Might be worth a check.

 

[CiscoHelpForMe]

Thanks for the replies.

@Vince
Yes, all connections are LC.
I have tried to swap the cables around but it doesn't work.

@chief
Switches are 24 port 3750x.
All SFPs are 10GB Fiber 10gb ports are Ten1/1/1 1/1/2 or Ten2/1/1 Ten2/1/2.
There is nothing special about the port configs. When the switchs are connected back to back with a
patch cable the SFPs work and connection comes up. When the link is placed on the longer cable, it
does not. The 10GB SFPs, have been correctly installed into 10GB SFP ports on the switches.
With 10GB, there is no speed setting possible, its fixed at 10GB Full Duplex.

 

[ADB100]

The distances for the different fibre specifications are listed here:

http://www.cisco.com/en/US/prod/collateral/modules/ps5455/data_sheet_c78-455693.html

I assume you have already seen this 300M is the max distance (SFP-10G-SR & SFP-10G-SR-X) over your cabling (OM3). The jumps between fibre specifications are quite dramatic - i.e. OM2 82M, OM3 300M - Are you absolutely sure the fibre and patch cables are OM3 spec?

I'd get a couple of SFP-10G-LRM modules and see if these work?

 

[CiscoHelpForMe]

Hello All,

Thanks for the Input.
I managed to get it working, not only were the fibers x'ed incorrectly
and I needed a newer IOS.. Thanks to all that helped..

@ADB100
Yes, I saw/know the distance specs for OM3/OM4. Yes 100% sure.
All cables are brand new OM3. All cables/connections were already
tested with 2 separate fiber testing tools and the cables have only
a .99-1.03 signal loss.

 

[ADB100]

Ahh... Didn't think of the really obvious... X-cables. Typically fibre patch cables are crossed (its easy to work out by looking at them anyway) - so its the infrastructure cabling that isn't crossed - i.e. patch cable X, infrastructure straight, patch cable X, so putting these all together you actually get a straight cable and you need to flip the TX & RX legs over.

As for the IOS upgrade, I doubt this would have prevented the link coming up?

Andy

 

[CiscoHelpForMe]

I would tend to agree, but when I changed the x-over before it didn't work..
I've been doing networks for 25+ years, This is the 1st time I've seen something
like this..