This is one of a multi-part series on the Ethernet switching landscape I’m writing in preparation for a 2-hour presentation at Interop Las Vegas 2014 (hour 1 & hour 2). Part 1 of this written series appeared on NetworkComputing.com. Search for the rest of this series.
One Ethernet switch feature becoming more common in recent years is Multi-chassis Link Aggregation, or MLAG. I have seen this feature referred to in some literature as MCEC, or Multi-chassis EtherChannel. For our purposes, we’ll call it MLAG, as I believe that to be sufficiently generic.
What is MLAG?
MLAG is the ability of two and sometimes more switches to act like a single switch when forming link bundles. This allows a host to uplink to two switches for physical diversity, while still only having a single bundle interface to manage. Also, two switches could connect to two other switches using MLAG, with all links forwarding. Link Aggregation Control Protocol (LACP, 802.3ad) is typically used to negotiate north and south (between host and an MLAG virtual switch or between MLAG virtual switches). East and west (between switches that are members of the same MLAG virtual switch), proprietary protocols are used.
The technologies used to form an MLAG virtual switch are not standard. The MLAG supplied by one vendor can’t be paired east-west with MLAG supplied by another vendor. To be very clear, I mean to say that an MLAG-capable switch from Juniper can’t be paired with an MLAG switch from Cisco to form an MLAG virtual switch. However, a Juniper MLAG virtual switch pair could talk to a Cisco MLAG virtual switch pair with no issue, on the assumption each virtual switch is talking LACP to the other one.
Some MLAG is the result of a unified control plane. There are a number of Ethernet switch stacking schemes where physically distinct switches surrender their control plane to the will of a master switch. The result is a virtual switch or virtual chassis made up of several switches that act like one unified switch. In this case, MLAG is one of the major benefits.
- Cisco cross-stack EtherChannel on the Catalyst 3750/3850 (and presumably others)
- Cisco Virtual Switching System on Catalyst 6500/6800
- HP – Intelligent Resilient Framework (IRF)
- Juniper – Virtual Chassis
Some MLAG maintains separate control planes. There are some MLAG schemes where two switches maintain their own control planes and are managed separately from one another. In these scheme, MLAG is a specific feature that must be enabled and configured on each switch expected to participate in the MLAG scheme. There are generally several design considerations around this including dedicated links and/or VLANs between the switch members, rules about traffic forwarding for loop avoidance, changes to the overall spanning-tree topology, and potentially many other considerations.
- Cisco Virtual Port-Channel (vPC) on the Nexus product line.
- Arista MLAG.
I have worked with both Arista’s MLAG and Cisco’s vPC. Both are similar to configure and operate, with similar concerns. In general, I found the Arista MLAG setup a bit easier, because many of the details are handled for the network operator when compared to the more granular fussiness of the vPC configuration. I wrote a detailed article about vPC setup on the Nexus 5500 for NetworkComputing.com back in 2012.
MLAG is limited to some number of participating switches. That is, depending on the scheme you are employing to achieve MLAG, you will have a specific number of switches that an MLAG bundle can span across. Cisco vPC and Arista MLAG are limited to two. HP IRF is limited to four. Juniper Virtual Chassis is limited to ten. At least, those are the limits of each, last I knew. It is possible these limits have gotten higher with new releases of software. Please let me know in the comments if these have changed, and I’ll update accordingly.
MLAG is a useful technology to present diverse physical paths to hosts, although admittedly VMware and other OS manufacturers can get similar results by load-balancing links in other ways. A network engineer might see MLAG as a tool best suited for eliminating blocked L2 links due to spanning-tree when cross-connecting pairs of switches as in the diagram above where the VLAN domain spans all of the switches in question.
Alternate network designs where specific VLANs are isolated to specific ToR switches would not need to leverage MLAG to achieve this non-blocking path diversity. Rather, L3 from ToR to the aggregation or core layer could be used. That said, isolated VLANs are becoming increasingly rare in data center design, where the ability to put any VLAN anywhere, stretching the VLAN domain potentially all over the data center, is usually a requirement due to virtualization. Where virtual machines might reside at any given point in time is unpredictable. In fairness, the demand of virtualization has given rise to a far more scalable and standard way of achieving L2 path diversity, that of L2 multipath using TRILL or SPB.
We’ll discuss a few equal-cost multipath (ECMP) schemes including both L2 and L3 in the next installment of this series.