L2 Etherchannel and Port Security 1

[Quadratic]

This is more of a general question that Cisco docs don't give a really good answer to, but I'm curious if anyone knows why Cisco does not support turning up port-security on an Etherchannel, and what behavior is expected when it is put in place. Most docs just say not to do it, though IOS will let you. Anyone know?

CCNP, CCDP

 

[brianinms]

Well port security is meant for end user ports and that doesn't fit in the realm of etherchannel. I mean why would you want to restrict the number of mac addesses to a etherchannel?

 

[PScottC]

Based on the comments here, here's my interpretation:

Even though you may have a virtualized MAC address for the device that's connected to the switch, the burned-in MACs still exist. The EtherChannel group still uses the burned-in MACs for port to port communication. This means that you would have to permit a minimum of 2 MAC addresses on each port. If you were to attach a virtual server or another switch to the EtherChannel, you would have to permit many more MACs (potentially hundreds). This defeats the purpose of port security.

I found several Cisco docs that stated that the port would be shut down if port-security were enabled. It may be that you can apply security to the Port-Channel interface, but not the individual physical interfaces. Once again, I think you're getting diminishing returns. Most systems that are EtherChannel connected are in secure rooms.

What does your testing show? And what devices are you connecting? I'm currently using Cisco EtherChannel for multi-port WLAN Controller connections and LACP for server connections.

PSC
[—] CCNP_{[blue]x3[/blue]} (Security/R&S/Wireless) [•] MCITP: Enterprise Admin [•] MCSE [—]

Governments and corporations need people like you and me. We are samurai. The keyboard cowboys. And all those other people out there who have no idea what's going on are the cattle. Mooo! --from "Hackers

 

[Quadratic]

In the way it was implimented, a few hundred macs were expected on the channel (and allowed), but the idea of several thousand needed to be avoided. The intended use of port-security was to enforce an oversubscription ratio of macs allowed on ports at different levels of a modular design with very mac-hungry, untrusted edge ports (that is, each port may be allowed a few hundred macs, but the design can't allow all ports to take up their full allotment, so port security at a distribution switch was done to prevent any major oversubscription from affecting more than a single access module.

Having said that, my issue is that Cisco doesn't explain why port security at a logical L2 Etherchannel interface (access link for single vlan) isn't supported. IOS will let you do it, and the only issue experienced was inter-vlan routing to/from hosts off of this channel being ceased temporarily on a per-host basis, while port-security recorded no violations and didn't have enough macs in the CAM table to complain.

CCNP, CCDP

 

[PScottC]

I think that both of us have observed that the documentation is vague. Take a look at this...

CSW01#[blue]sh ether 1 de[/blue]

Group state = L2
Ports: 2 Maxports = 16
Port-channels: 1 Max Port-channels = 16
Protocol: LACP
Minimum Links: 0
Ports in the group:
-------------------
Port: Gi1/0/13
------------
Port state = Up Mstr Assoc In-Bndl
[red]BLAH, BLAH, BLAH[/RED]

Port: Gi2/0/13
------------
Port state = Up Mstr Assoc In-Bndl
[red]BLAH, BLAH, BLAH[/RED]
------------

Age of the Port-channel = 74d:11h:47m:18s
Logical slot/port = 10/1 Number of ports = 2
HotStandBy port = null
Port state = Port-channel Ag-Inuse
Protocol = LACP
[green]Port security = Disabled[/green]

Ports in the Port-channel:

Index Load Port EC state No of bits
------+------+------+------------------+-----------
0 00 Gi1/0/13 Active 0
0 00 Gi2/0/13 Active 0

Time since last port bundled: 43d:14h:10m:11s Gi2/0/13
Time since last port Un-bundled: 43d:14h:11m:37s Gi2/0/13

It would appear that you are allowed to configure port-security on the logical port. I think that the documentation is referring to the physical ports involved in the channel. As to why, I can only speculate.


PSC
[—] CCNP_{[blue]x3[/blue]} (Security/R&S/Wireless) [•] MCITP: Enterprise Admin [•] MCSE [—]

Governments and corporations need people like you and me. We are samurai. The keyboard cowboys. And all those other people out there who have no idea what's going on are the cattle. Mooo! --from "Hackers

 

[brianinms]

If I had thousands of mac addresses traversing an etherchannel link I would rethink the network design.

 

[PScottC]

If you were running IPv6, it would not be inconceivable. Since there is no broadcasting the need to restrict broadcast domains goes away. The maximum recommended subnet mask is /64 of a 128-bit address.

PSC
[—] CCNP_{[blue]x3[/blue]} (Security/R&S/Wireless) [•] MCITP: Enterprise Admin [•] MCSE [—]

Governments and corporations need people like you and me. We are samurai. The keyboard cowboys. And all those other people out there who have no idea what's going on are the cattle. Mooo! --from "Hackers

 

[Quadratic]

True enough Brian. It was inherited, and not easily changed.

CCNP, CCDP

 

[PScottC]

I spent a few more minutes looking at this. I tested this on 3560 and 3750 switches using the current IOS, and actually cannot set up port security on any of my etherchannel ports. The IOS has removed the commands. On a standard port the "switchport port-security" command exists, but on any port involved in an etherchannel (physical or logical), the command is missing.

On a 2950 I tested, the command was available in the port-channel interface, but displayed an error when I tried to configure it.

I've looked all over for documentation that gives a "why", but I can't find it. Have you called Cisco and asked?

PSC
[—] CCNP_{[blue]x3[/blue]} (Security/R&S/Wireless) [•] MCITP: Enterprise Admin [•] MCSE [—]

Governments and corporations need people like you and me. We are samurai. The keyboard cowboys. And all those other people out there who have no idea what's going on are the cattle. Mooo! --from "Hackers

 

[Cluebird]

My questions are: "Why would you want port-security on an Ether-Channel...what is the design objective...what are you trying to accomplish?"

Remember that channeling was designed for switch-to-switch connections as a way to aggregate low bandwidth parallel links into larger bit pipes so you didn't have to upgrade to new hardware when your uplinks became oversubscribed. Channeling also gave us control over load-balancing on the uplinks and helped limit spanning-tree issues with parallel ports. Since the links we channelled were usually placed into trunks, some of the trunk restrictions (no port-security for example) became inherited by the etherchannel. Port-security was an edge-port technology, trunking an uplink technology.

What use is an edge technology on an uplink and what problems arise when that technology is implemented (i.e. how many TAC calls will be created when someone screws up?)?

If you're a large enough client and you have a significant need for the feature, the code-monkeys at Cisco can probably implement the functionality, but so far there are other things that have higher priority.

My 2 cents.

 

[Quadratic]

@ PScottC - This one never made it to the TAC. We ended up working around the problem somewhat, and it's soon to become irrelevant so it was decided to be too much hastle.

@ Cluebird - I disagree with a few of your points here, mainly because I think you're looking at it strictly from the perspective of an Enterprise network. You draw the distinction between a trunk and an edge port, but from a service provider perspective for Metro Ethernet MAN/WAN services (to give one example), those can often be the same thing. There are situations where trunks and even port channels are effectively untrusted demarcation points. In such cases, port-security is important to protect against CAM overflows, NOT to secure a single desktop's usage of an access port. Even portfast and BPDU filtering can be legitimately used on trunk links if they are functionally the network edge.

CCNP, CCDP

 

[PScottC]

Once again this comes back to intrinsic limitations that must be accomodated by the network design. My first thought was QinQ would resolve, but that only adds an extra tag to separate traffic... The original source and destination MACs remain in the frame.

The problem I see is that even if you were to successfully restrict the number of MACs on an etherchannel, you are still subject to CAM overflow. Let's imagine that your switch can maintain 16K MACs in the CAM and you are limiting your etherchannel to 1K of MACs. This means that you can only support 16 customers in this fashion. Once you add the 17th customer your CAM is oversubscribed. I can easily imagine that a provider switch can handle quite a few more than 16 customers.

Just looking around a bit at the search results for "Metro-E design", I can see that MPLS is mentioned as a way around this problem. You simply encapsulate the traffic in the core and deal with these issues only at the edge.

PSC
[—] CCNP_{[blue]x3[/blue]} (Security/R&S/Wireless) [•] MCITP: Enterprise Admin [•] MCSE [—]

Governments and corporations need people like you and me. We are samurai. The keyboard cowboys. And all those other people out there who have no idea what's going on are the cattle. Mooo! --from "Hackers

 

[Cluebird]

Quadratic,

You are correct. I was looking at it from the Enterprise perspective. ISP requirements are definitely different as you are defining the "edges" differently for the various customers. That is why I asked what the design objective was before I tried to provide a general answer. Maybe I'll just answer every question with the favorite response, "It depends..."!

MetroE does present its own challenges with flex-links, dot1q tunneling, and such because you definitely don't want one customer site affecting another. Maybe we can open up the MPLS can of worms next?

 

[Quadratic]

@ PScottC:

The thing is, not all Metro Ethernet services end in L2 switchports facing a customer network, and even among those that are, not all would require an allowance of over 1K CAM entries. Some customer-facing ports would go to MPLS VPN endpoints, for example, where the provider switch might use a routed port for a point-to-point network inside a customer-specific VRF, so the concept of mac-hungry L2 access wouldn't apply.

Also, for ethernet relay services, where a vlan becomes analagous to a frame-relay "circuit" from a provider's perspective, in some cases mac-learning can be disabled outright. Since all traffic has only two endpoints, there are cases where frame flooding (for unknown destination macs) doesn't matter since it only has one path to take. Such a setup can still have redundancy, too, since STP would block the floods in an L2 domain and if the provider has an MPLS core then LSRs wouldn't base forwarding for MPLS VCs on the CAM tables anyway.

So, for a single provider edge switch, you definitely wouldn't need a 1K port-security restriction on every customer-facing port. I'd say 1K is pretty high in general, but if your CAM tables became a major scaling issue for a provider, I'd say establish a baseline of expected customer usage, make recorded port-security violations a part of normal network monitoring (which it should be anyway) and handle extreme cases case-by-case.

As for QnQ, it's useful for preserving available provider vlans and STP instances, if you had a point-to-point ethernet relay service where a customer wanted to exchange tags between their sites. It wouldn't help the CAM for the reasons you mention, though.

@ Cluebird:

In many things, there's definitely more of an emphasis on enterprises than providers. In Cisco texts especially, I notice they often look at the enterprise network as "we" and the provider network as "they". The original scenario in this thread wasn't for a MetroE network (I used it as a classic example of edge trunks), but sicne the question was more about why Cisco doesn't recommend a particular setting for a specific technology, I didn't think more elaboration on use was needed; if I was asking how to redesign a network such that port-security on an etherchannel wasn't needed, I definitely would have explained the scenario at hand in more detail.

CCNP, CCDP