Switch Trunks: What the heck do they do??? 6

[Bubbalouie]

Hi,

I am not really a switch person so I hope I am asking the right questions the right way here. I have 'inherited' a network setup in the following manner:

WS-C2924-XL <- WS-C3548-XL <- WS-C2924-XL -> WS-C3548-XL -> WS-C2924-XL
Switch5         Switch3        Switch1         Switch2        Switch4

the two switches on the left are in one bldg
Switch5 Port1 is connected to Switch3 Port2 (via ethernet cable)
Switch3 Port1 is connected to Switch1 Port1 (via a fiber optic media converter)

the switch in the middle is in one bldg
in addition to switches connected to Port1 and Port2:
Port3 is connected to a PIX 506
Port4 is connected to a 1750 router

the two switches on the right are in one bldg
Switch4 Port1 is connected to Switch2 Port2 (via ethernet cable)
Switch2 Port1 is connected to Switch1 Port2 (via a fiber optic media converter)

The only changes I've made so far are:
I moved the physical links between the switches as indicated above (they were all over the place and I mainly did it so I could try and make sense of it!).
I removed spanning-tree portfast off the linked ports and also off the last 6 ports off Switch2 and Switch3 for some workgroup switches which I had moved from various ports (all ports had it on and I'd read that you should not have other switches hooked up like that?)

It seems to be running fine now or at least a lot better than when I started it (loops???)

My main question concerns the links between the switches. I am poring through stuff I find on the Internet about Cisco switches and I keep reading about TRUNKS. Should I have trunks setup between my switches? Are there gains I could accrue by doing so? If so, a gentle nudge in the right direction would be most appreciated!

My second question is on the spanning-tree portfast. Every port had it on it. I understand no spanning-tree portfast on ports connected to switches and hubs. What about Port3 and Port4 on Switch1 where I have the PIX and Router hooked up? Are there times when you should have just spanning-tree or portfast enabled on a port?

My third question is what is that router doing? Can the PIX do it?

My fourth and final question concerns the daisy chaining of these switches. I've heard I shouldn't have more than 3 switches daisy chained. I moved some workgroup switches off of Switch4 to Switch2 and Switch5 to Switch3. I have some sites that connect thru site-to-site VPN's with the PIX. They pass through a switch at their location. If I have a resource on Switch4 or Switch5 are they passing through 4 switches and thus violating the 3 daisy chained switches rule?

I'm obviously a little out of my element here and hope I've explained it well. I'm looking for any tips and pointers anyone has to offer.

Thanks in Advance!

 

[unclerico]

My main question concerns the links between the switches. I am poring through stuff I find on the Internet about Cisco switches and I keep reading about TRUNKS. Should I have trunks setup between my switches? Are there gains I could accrue by doing so? If so, a gentle nudge in the right direction would be most appreciated!

A short and concise answer is that you only need trunks if you are running multiple VLAN's.

My second question is on the spanning-tree portfast. Every port had it on it. I understand no spanning-tree portfast on ports connected to switches and hubs. What about Port3 and Port4 on Switch1 where I have the PIX and Router hooked up? Are there times when you should have just spanning-tree or portfast enabled on a port?

You you can leave port-fast enabled on the interfaces connected to the PIX and the Router. You should not have port-fast enabled on ports connected to other switches/hubs. If you need a rapid uplink to your downlevel switches/hubs you can enable port-fast trunk or configure Rapid Spanning-Tree

My third question is what is that router doing? Can the PIX do it?

Good question. Unfortunately we have no idea what the router's purpose is. I'm assuming that it is for your Internet connection?? It could also be configured as a router-on-a-stick to facilitate inter-VLAN communication if you are running multple VLANs.

My fourth and final question concerns the daisy chaining of these switches. I've heard I shouldn't have more than 3 switches daisy chained.

Most good network designs will have a couple of core switches with redundant connections. Hangining off of each redundant core switch will be a couple of redundantly connected distribution switches. From the distribution switches will be redundantly connected access layer switches. So as you can see, "daisy chaining" is almost always prevalent in a topology. With that said, there is a right way and a wrong way to daisy chain and depending on how you do it will determine how many problems that you have.

Your question about having no more than three devices daisy chained is referring to your network diameter. Spanning-tree calculations are based in large part off of how large your diameter is. By default it is set to 7 which means if you hang an 8th switch off the end you have just had yourself a resume building event. The diameter is calculated from your root bridge outwards. So in your diagram if Switch1 is the root of all of your VLANs then you're fine. Also note that if you don't have redundant connections between your switches (i.e. no loops) then you don't really have to worry so much; however, the potential is there if you're not careful. Check out this article here:

http://www.cio.com.au/article/65115/all_systems_down

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

Thanks for the response! I swear I'm not trying to be dense but...

"A short and concise answer is that you only need trunks if you are running multiple VLAN's."
each switch has a navtive VLAN1. i'm guessing that those don't really count as seperate VLANS since they are local to each machine?


"You you can leave port-fast enabled on the interfaces connected to the PIX and the Router. You should not have port-fast enabled on ports connected to other switches/hubs. If you need a rapid uplink to your downlevel switches/hubs you can enable port-fast trunk or configure Rapid Spanning-Tree"
i'm gonna check them out.


"Good question. Unfortunately we have no idea what the router's purpose is. I'm assuming that it is for your Internet connection?? It could also be configured as a router-on-a-stick to facilitate inter-VLAN communication if you are running multple VLANs."
hmmm. that was a pretty vague question on my part. PIX is connected to the Internet and the router is just plugged into that switch. maybe it provides routing for the network.


"Your question about having no more than three devices daisy..."
i like that answer and the article. i hope you don't mind, but i'm gonna start using that '...you have just had yourself a resume building event' too. that's a great line!

i would like to follow up with a couple of things though and it has to do with the '...if you don't have redundant connections' statement. i read somewhere that you can setup multiple ports between switches and have them act as one load-balanced connection (or something like that. can't remember where i saw it now). i have some application servers in the bldg that has Switch1 in it. I was thinking that if I dedicated two ports to that on Switch1 and Switch2 that my users might get a boost in performance to those apps. would that be the 'redundant connections' you say i need to be careful about?

 

[Bubbalouie]

etherchannel! that's what i saw about bringing two ports 'together'.

 

[stubnski]

each switch has a navtive VLAN1. i'm guessing that those don't really count as seperate VLANS since they are local to each machine?

Each switch has it's own VLAN database that stores it's own VLAN information. If you want to learn more about VLANs I would encourage you to look at Cisco's website and also to look into VTP.

VLANS
More info


Stubnski

 

[unclerico]

This is a long one so hold on

each switch has a navtive VLAN1. i'm guessing that those don't really count as seperate VLANS since they are local to each machine?

Well, not exactly. I don't want to get too technical here, but once you connect two switches together and form a trunk there isn't really much that is considered "local" anymore.

hmmm. that was a pretty vague question on my part. PIX is connected to the Internet and the router is just plugged into that switch. maybe it provides routing for the network.

If you post the router configuration along with you addressing scheme(s) used internally we can probably help you determine what it is for.

i like that answer and the article. i hope you don't mind, but i'm gonna start using that

Yeah, that article just reaffirms that if you don't know exactly what you're doing especially at layer 2 you could be in a world of hurt. Always, always, always keep detailed documentation of your toplogy. And of course you can use that line, I tell it to my junior admins all the time

i read somewhere that you can setup multiple ports between switches and have them act as one load-balanced connection (or something like that. can't remember where i saw it now). i have some application servers in the bldg that has Switch1 in it. I was thinking that if I dedicated two ports to that on Switch1 and Switch2 that my users might get a boost in performance to those apps.

You're referring to Etherchannel/Link Aggregation. Absolutely you should make use of that between your switches. It isn't necessarily a true load-balancing tool just because none of the algorithms that it uses are a true round-robin and links are not chosen based on congestion. Frames are forwarded based on a hashing algorithm. You can load-balance based on MAC Address, IP Address, TCP/UDP ports or a combination. Again, I don't want to get too technical here so if you want a further explanation I'd be happy to give it to you.

would that be the 'redundant connections' you say i need to be careful about?

Yes and no. If you don't configure the Etherchannel correctly there is a possiblity of a loop. You should always have the ports that you want to bundle together administratively shutdown until you have the configuration completed on both switches. Once the configuration is complete re-enable them. Even though an Etherchannel contains more than one link, Spanning-Tree looks at it as a single link. A lot of times admins will build redundant etherchannel links between switches.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

OK, here is what I think I want to do:

1. Leave vlan's and trunks alone for now. I'll tackle that after I read up some more on it..

2. Setup Etherlink channels 2 ports on Switch1 and linking them to 2 ports on Switch2 AND 2 ports on Switch1 to 2 ports on Switch3.

3. I read something about UDLD should be used on ports connected with fiber but I don't see that currently on the ports connected via fiber. Anything to worry about there?

Would that be a decent start of a plan?

I'll post the router config seperately if you would like to comment.

 

[unclerico]

1. Leave vlan's and trunks alone for now. I'll tackle that after I read up some more on it..

If it's working currently then definitely leave it alone. Good choice to leave it alone until you fully understand what the consequences will be from any changes that you make.

2. Setup Etherlink channels 2 ports on Switch1 and linking them to 2 ports on Switch2 AND 2 ports on Switch1 to 2 ports on Switch3.

Go for it

3. I read something about UDLD should be used on ports connected with fiber but I don't see that currently on the ports connected via fiber. Anything to worry about there?

Yes, it would be a wise idea to enable that. You have two choices; agressive or normal. If you choose agressive mode the interface will be shut down after, I believe, 8 missed echo's/echo-replies. If you leave it at normal, the interface will stay up, but a syslog message will be generated.

I'll post the router config seperately if you would like to comment.

If you would like to, go right ahead.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

Here is the router config. The serial interface is not in use.

ip route 0.0.0.0 0.0.0.0 192.168.1.254 (PIX)
ip route 192.168.1.4 255.255.255.255 FastEthernet0 (NT Term Server)
ip route 192.168.6.0 255.255.255.0 192.168.1.2 (A 1750 router with a WIC-1ENET; there is a little workgroug switch connected to it and a db server and app server connected to it)

Current configuration:
!
version 12.1
service timestamps debug uptime
service timestamps log uptime
no service password-encryption
!
hostname CentOff_1750
!
boot system flash:1:aaa1397.bin
no logging console
enable secret 5 $1$OYao$DnyI58XUCPip6c5OuK8.q1
enable password elephant
!
!
!
!
!
memory-size iomem 25
ip subnet-zero
no ip domain-lookup
!
ip audit notify log
ip audit po max-events 100
!
!
!
!
!
!
!
cns event-service server
!
!
!
!
interface Serial0
 description Ameritech Fractional T1 service DLCI's 17,18
 no ip address
 encapsulation frame-relay
 no fair-queue
 service-module t1 timeslots 1-2
 service-module t1 remote-alarm-enable
 frame-relay lmi-type cisco
!
interface Serial0.1 point-to-point
 description Connected to princeton_1750 DLCI 17
 ip address 10.0.2.1 255.255.255.0
 frame-relay interface-dlci 17
!
interface Serial0.2 point-to-point
 description Connected to Paoli1750 DLCI 18
 ip address 10.0.3.1 255.255.255.0
 frame-relay interface-dlci 18
!
interface FastEthernet0
 ip address 192.168.1.1 255.255.255.0
 speed auto
 full-duplex
!
ip classless
ip route 0.0.0.0 0.0.0.0 192.168.1.254
ip route 192.168.1.4 255.255.255.255 FastEthernet0
ip route 192.168.6.0 255.255.255.0 192.168.1.2
no ip http server
!
!
!
line con 0
 exec-timeout 300 0
 transport input none
line aux 0
line vty 0 4
 exec-timeout 300 0
 password elephant
 login
!
no scheduler allocate
end

 

[unclerico]

as long as the serial interface is in fact not functional, then I would say that there is no reason that you can't remove the router from your network and use the PIX. It's totally up to you. As it is right now, you're more than likely getting ICMP redirects pointing your default traffic from 192.168.1.0/24 to your PIX anyway. Same thing with your traffic destined for 192.168.6.0/24. It's just an extra hop adding latency into your round trip time. Make sure you add a static route into your PIX for the .6 network.

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

ok, i think i'll wait till the weekend then and shut off the router and see if anything 'bad' happens.

i need a couple of extra fiber patch cords and have to wait until the first of the month to use my company credit card again. since i can't do the link between the bldgsright yet, i'm gonna dork around and see if i can do an etherchannel between Switch5 and Switch3. if i get that working i'll feel a lot better about the link between the bldgs.

i upgraded the 2900's over the weekend so they would all be on the same IOS ver. Version 12.0(5)WC17. both 3500's are on Version 12.0(5)WC16 though i do have a WC17 for them available.

i had read up on the channel-group command, but that is not recognized by my switches though port group is. also, udld aggresive and negotiation auto is not supported on any of the switches.

anyway, after looking up a bunch of docs of the port group command, most of the examples just have something similar to this:

Switch3
!
interface FastEthernet0/22
shutdown
port group 1
!
interface FastEthernet0/23
shutdown
port group 1

Switch5
!
interface FastEthernet0/46
shutdown
port group 1
!
interface FastEthernet0/47
shutdown
port group 1

is it really that frickin' simple!!!

i did read a couple of posts where some people who certainly sounded like they are experts said to nail both ends of the etherchannel down tight so right now i have:

Switch3
!
interface FastEthernet0/22
shutdown
duplex full
speed 100
port group 1
udld enable
!
interface FastEthernet0/23
shutdown
duplex full
speed 100
port group 1
udld enable

Switch5
!
interface FastEthernet0/46
shutdown
duplex full
speed 100
port group 1
udld enable
!
interface FastEthernet0/47
shutdown
duplex full
speed 100
port group 1
udld enable

so can i plug in a couple of crossover cables into the ports to connect them, take the interfaces out of shutdown and let'er rip?

there's gotta be a catch...

 

[drewdown]

That should be it my man. I just did this last night with a couple stacked 3750's. The commands/config are slightly different but you have the jist of it.

 

[Bubbalouie]

Hey! It works!

I think I'll go do Switch2 and Switch4!

Two quick questions on fine tuning this. I had stumbled across the UDLD command while looking at the fiber connections between bldgs. Is it doing anything on the straight copper connections between the two switches I have it setup on now?

Right now I have this command:

port group 1

but see where there are other options to it, specifically

port group 1 distribution source

and

port group 1 distribution destination

would i set up one end of the ports as destination and the other as source? would it make any difference?

also, and yes, this makes three... any type of load balancing commands I can add to this.

thanks for all your help and patience!

 

[Bubbalouie]

ok, got all my equpment in and ports configured for the etherchannels between the bldgs.

before i turn all on i wanna make sure i don't blow something by something i noticed in my config.

i set up etherchannel between switch2 AND switch4 in one bldg and Switch3 and Switch5 in another bldg. these switches are local to their respective bldgs in that they are literally right next to each other and i just used some crossover cables to connect them. in each case, i called the Port Group between the switches Port Group 1.

I have set up a Port Group 2 for the conection from Switch1 to Switch2 and a Port Group 3 for the connection from Switch1 to Switch3. These are the EtherChannel connections between the 3 bldgs that I'd like to turn on.

However, I'm a little concerned about the fact that I currently have two port groups called Port Group 1. Is that gonna be a problem when I turn on the new port groups?

 

[unclerico]

No not at all. The port-group information is local to the switch. It's nothing to worry about. I have seen some documentation that when using LACP as the protocol you need to have the port-channel numbers the same on both sides. I have never actually experienced this however

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

ok, i got close. the etherchannel between Switch1 and Switch2 two worked fine but the etherchannel between Switch1 and Switch3 did not.

one port in the bad etherchannel was completely down and the other was passing traffic extremely slow. based on performance of some apps i tested that had to cross the link, probably about 10 mbps instead of 100.

i rebooted both switches hoping that would clear the problem to no avail. the one link that was completely down i decided to take a shot at replacing the media converter. once i did i got a link light on the media converter (which was absent before i replaced the unit) and thought i was in business.

however, that actually brought the link down almost completely. the traffic would not pass the switch though i could telnet into the switch itself. i just couldn't ping anything on the other side of the switch such as the mail server (that made me really popular for about 10 minutes) from my workstation in the other bldg though i could ping it from the switch.

i'm gonna revisit the hardware once i get a chance to look at it again, but i'm kinda hoping based on my description above someone is gonna be able to say:

'Noob! You forgot to xxxxxxxxxxx!'

i couldn't google up enough troubleshooting info before i had to revert back to the original setup in order to get things working again. i did run across the show etherchannel summary command and ran that. the odd thing is if i'm looking at it correctly only one side of the link was down:

(Switch1 is in one bldg and the one in the middle of the layout. Switch2 is in a different bldg and Switch3 is in yet another bldg.)

<code>
Switch2#show ether summ
Flags: d - default D - down
I - in use

Group Ports
----- -----
1 Fa0/22(I) Fa0/23(Id)
3 Fa0/1(I) Fa0/2(Id)
---------------------------------------------------
Switch1#show ether summ
Flags: d - default D - down
I - in use

Group Ports
----- -----
2 Fa0/1(D) Fa0/2(Id)
3 Fa0/3(Id) Fa0/4(I)
------------------------------------------------------
Switch3#show ether summ
Flags: d - default D - down
I - in use

Group Ports
----- -----
1 Fa0/48(I) Fa0/47(Id)
2 Fa0/46(I) Fa0/45(Id)
</code>

if anyone can recommend me some good commands to troubleshoot etherchannel and what they do for me, i'd be indebted. i'm gonna try again and want to be able to make a more rational guess as to what the problem is.

 

[unclerico]

post the interface configuration for all port-channels on the switches including the physical ports. You'll show the configuration output of the port-channel as well as the port members

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[Bubbalouie]

On SW1 ports 1 & 2 (Port Group 2) are trying to connect to SW3 ports 45 & 46 (Port Group 2). The other port groups shown connect successfully to other switches using the same etherchannel config. Only change I've made right now is the 'distribution destination' i added to the bad port group 2 on sw1 based on something I read earlier. SW1 fa0/2 shows blocked but I can't figure it out. When I do a 'shut', 'no shut' on that interface it is instantly blocked again.

----------------------------2924xl----------------------------
SW1#show etherchannel summ
Flags:  d - default     D - down
        I - in use

Group Ports
----- -----
2     Fa0/1(D) Fa0/2(Dd)
3     Fa0/3(Id) Fa0/4(I)

interface FastEthernet0/1
 description _Etherchannel to bldg7_
 duplex full
 speed 100
 port group 2 distribution destination
 udld enable
!
interface FastEthernet0/2
 description _Etherchannel to bldg7_
 duplex full
 speed 100
 port group 2 distribution destination
 udld enable
!
interface FastEthernet0/3
 duplex full
 speed 100
 port group 3
 udld enable
!
interface FastEthernet0/4
 duplex full
 speed 100
 port group 3
 udld enable
!


SW1#show port group 2
Group  Interface              Transmit Distribution
-----  ---------------------  ---------------------
    2  FastEthernet0/1        destination address
    2  FastEthernet0/2        destination address

SW1#show int fa0/1 status

Port    Name               Status       Vlan     Duplex Speed   Type
------- ------------------ ------------ -------- ------ ------- ----
Fa0/1   _Etherchannel to b notconnect   1          Full     100 100BaseTX/FX

SW1#show int fa0/2 status

Port    Name               Status       Vlan     Duplex Speed   Type
------- ------------------ ------------ -------- ------ ------- ----
Fa0/2   _Etherchannel to b notconnect   1          Full     100 100BaseTX/FX

SW1#show spanning-tree brief

VLAN1
  Spanning tree enabled protocol IEEE
  ROOT ID    Priority 32768
             Address 0002.1639.e280
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768
             Address     0002.b978.3340
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec


Port                           Designated
Name    Port ID Prio Cost Sts  Cost  Bridge ID      Port ID
------- ------- ---- ---- ---  ----  -------------- -------
Fa0/2   128.2   128  19   BLK  12    0002.b978.3340 128.2
Fa0/3   128.3   128  12   FWD  0     0002.1639.e280 128.3

----------------------------------------------------------------------------------------------------------------
----------------------------------------------3548xl----------------------------------------------------------
----------------------------------------------------------------------------------------------------------------
SW3#show etherchannel summ
Flags:  d - default     D - down
        I - in use

Group Ports
----- -----
1     Fa0/48(I) Fa0/47(Id)
2     Fa0/46(Id) Fa0/45(D)


interface FastEthernet0/45
 duplex full
 speed 100
 port group 2
 udld enable
!
interface FastEthernet0/46
 duplex full
 speed 100
 port group 2
 udld enable
!
interface FastEthernet0/47
 duplex full
 speed 100
 port group 1
 udld enable
!
interface FastEthernet0/48
 duplex full
 speed 100
 port group 1
 udld enable
!

SW3#show port group 2
Group  Interface              Transmit Distribution
-----  ---------------------  ---------------------
    2  FastEthernet0/46       source address
    2  FastEthernet0/45       source address

SW3#show int fa0/46 status

Port    Name               Status       Vlan     Duplex Speed   Type
------- ------------------ ------------ -------- ------ ------- ----
Fa0/46                     connected    1          Full     100 100BaseTX/FX
SW3#show int fa0/47 status

Port    Name               Status       Vlan     Duplex Speed   Type
------- ------------------ ------------ -------- ------ ------- ----
Fa0/47                     connected    1          Full     100 100BaseTX/FX

SW3#show spanning-tree brief

VLAN1
  Spanning tree enabled protocol IEEE
  ROOT ID    Priority 32768
             Address 0002.1639.e280
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec

  Bridge ID  Priority    32768
             Address     0005.32ec.83c0
             Hello Time   2 sec  Max Age 20 sec  Forward Delay 15 sec


Port                           Designated
Name    Port ID Prio Cost Sts  Cost  Bridge ID      Port ID
------- ------- ---- ---- ---  ----  -------------- -------
Fa0/47  128.1   128  12   FWD  31    0005.32ec.83c0 128.1
Fa0/46  128.2   128  19   FWD  31    0005.32ec.83c0 128.2

 

[unclerico]

Can you post the current configs of those switches??

I hate all Uppercase... I don't want my groups to seem angry at me all the time! =)
- ColdFlame (vbscript forum)

 

[vipergg]

Check your wiring something doesn't add up on one side your ports say notconnect for 1 and 2 and yet the other side says connected though you posted for 46 and 47 instead of 45 and 46 . It can't show connected on one side and not the other this is a physical layer issue .