From the blog.

Managing Digital Racket
The more I tune out, the less I miss it. But that has presented me with some complex choices for a nuanced approach to curb
Complexity – My Friend, My Enemy
Over my years of network engineering, I've learned that the fewer features you can implement while still achieving a business goal, the better. Why? Fewer

Positioning Cisco’s Ethernet Switching Products For The Enterprise

2,627 Words. Plan about 17 minute(s) to read this.

One thing I can’t help admiring Cisco for is their ability to identify a market niche and stick a product in it – usually a competitive product. And while I wish Cisco would stop identifying new market segments for Ethernet switches, the fact of the matter is that they make a darn good case for why they do so. Each switch line has a place that will make sense to some existing – or potential – customer.

Part of the reason for the abundance of Cisco switch lines is tied to how they are organized as a company. Cisco is an absolutely mammoth organization, pulling in over $12B in sales during their 2013 Q3. About $3.6B of that revenue number came from switches. That’s a huge number being sold to a wide variety of Cisco customers, broadly including at least enterprise, service provider and data center. Announcing a new Nexus 7700 chassis as well as a new Catalyst 6800 product line at Cisco Live 2013 in Orlando, Cisco’s queuing up even more sales and market share.

While all of this is fine for Cisco, a challenge comes for the customers. While still not the sensory assault HP abuses their customers with in their shock-and-awe approach to switch offerings, the Cisco Ethernet switching line is getting a bit muddled. There’s been functional overlap for a while, and the new 6800 line in particular makes this all the more poignant. In fact, Cisco’s greatest competition might be…Cisco. I’m sure that’s not lost on Cisco’s shareholders. A sale is a sale, and at the end of the quarter, does Wall Street really care if the sale came from a Catalyst switch or a Nexus switch? Assuredly, they do not.

The podcast has afforded me the chance to work with a few different Cisco business units over the last couple of years. I’ve learned that Cisco’s BU’s at times compete against each other. For example, the Catalyst crew loves their products; they don’t care about the Nexus products. From what I can tell talking to various folks, the Catalyst team develops their own products. The Nexus team develops their own products. These teams don’t share notes, work for feature parity, test compatibility, or anything else. A Cisco friend of mine described it as, “not an aircraft carrier, but rather a thousand rowboats.”

How do I see the Cisco Ethernet product lines? Based on my long history with the product lines and recent conversations with Cisco product managers, here’s how I understand Cisco to be positioning their products. It’s simple, really. Large campus and enterprises are going to be pushed in the Catalyst direction. Data center customers are going to be pushed in the Nexus direction. For service provider customers, it depends.

CATALYST

For a long time, Cisco has sold a variety of Catalyst switches in just about any form factor and feature set the campuses & enterprises could want. I’d take you through them, but that would be tedious for readers that are probably already familiar with them. The Catalyst line is, in my experience working on many different networks for many years, almost everywhere. The networks that haven’t been running Catalyst products have either been the very low end SMB sites that bought something at the local big box store, or a site where some other switch vendor had managed to beat Cisco on price or where someone knew someone – typically Enterasys or HP. Hey, I’m from New Hampshire, where Enterasys (divested from the once-mighty Cabletron and now tied to Siemens) ruled for quite a few years.

For the least 3 or so years, the Catalyst line has had one gaping hole: 10GbE port density. The mighty 6500E chassis just couldn’t scale as high as what many enterprises were looking for, even with the Supervisor 2T pushing the backplane to its limit. For this reason, many enterprises invested in the Nexus  product line, which offers a variety of 10GbE choices in a variety of form factors. This investment came with a bit frustration for customers, as the Nexus product line runs NX-OS, a different operating system than the Catalyst 6500’s IOS. In addition, the Nexus line did not (and still doesn’t) offer software feature parity with the 6500. So, some Cisco customers went to Nexus against their will. Other Cisco customers went to one of a number of Cisco competitors, following the logic that if they have to make a significant product change anyway, they might as well look at other vendor’s products. Although I don’t have numbers in front of me, I believe it’s fair to say that Cisco lost some switching market share during this time – let’s say roughly 2008-2012. Uncle John‘s gone to a lot of trouble to win lost share back, dumping bad acquisitions and focusing on the core Cisco business.

GameChangerOne demonstration of that focus is the Catalyst 6800 line. There’s nothing revolutionary about this product, but it fills the gaping hole. The 6800 products represent roughly a bandwidth upgrade for the the Catalyst 6500; the positioning of the 6800 is that of 6500 customers who want to keep everything the same – internal operations, feature set, and IOS. This, despite the humorous #GameChanger campaign that still makes me grin. (And in fairness, the 6800 is only one product in that campaign portfolio. Still, let me have my laugh.) The point is that Cisco has identified the weak link in the product line, and eliminated any reason for customers to wander.

Another product in the Catalyst line that demonstrates the target market of the enterprise is the 3850 switch. This product combines switching with a wireless LAN controller. Squarely an enterprise play.

Some overlap comes with the 4500X, which Cisco is billing as an aggregation layer switch. That’s interesting positioning, but one that could be filled admirably with a Nexus 6001. There’s more overlap in the Catalyst line itself, with 3750X’s (perhaps my favorite access switch of all time) competing with the new 3850’s…which compete with the various 4500E chassis lines…which compete with the 6500 chassis lines. All of these do L3 switching. Now, if all you want is a layer 2 access switch, then I suppose the choice in the Catalyst time is reasonably clear.

Cisco has made plain to me through briefings and side conversations that the Catalyst line is for the enterprise & campus. And I guess that’s not a message that’s changed over the years. It’s just that now with the addition of the 6800, the product line once again fills the vast majority of needs at every tier.

NEXUS

Nexus switches are, of course, Cisco’s data center play. Nexus products have certainly sold well, but have been dogged by competitors (notably Arista, with Juniper, HP and Brocade to a lesser degree, in my opinion) who seem to stay just in front of Cisco with a balance of features, ease of use, lower power consumption, and price. Cisco is certainly not the low price leader, although as we all know, no one pays list. But still…with Arista announcing $10K per 100G port including optics in their 7500E line cards, it’s clear they are trying to expand their data center footprint using cost as one of the drivers. Rumor has it that some Cisco pricing changes are coming soon to make this less of an issue, so bug your Cisco rep if you’re getting ready to spend some of your budget money.

For the enterprise customer who has invested in Nexus gear, their likely use case is in their own local data centers. The Nexus 5500s offer line-rate L2 at high 10GbE port densities, and include FCoE and native FC interfaces for those interested. The Nexus 7K has a clever group of tricks it can perform, including OTV and FabricPath. Multi-chassis link aggregation in the form of virtual port-channel is available across the Nexus product line, a rather different way of going about MLAG when compared to the stacking & VSS solutions offered in the Catalyst line. And for an interesting number of top-of-rack design options, the fabric extender (FEX) products are useful as well as inexpensive.

A nifty entrant in the FEX product line is the B22 module that works in a few different blade enclosures. That allows the network to have insight right down to the virtual NIC, which is a useful feature to gain insight into performance issues and aid in troubleshooting. There’s nothing like this in the Catalyst product line. And in case you’re not aware, a FEX requires a Nexus 5K, 6K, or 7K…meaning if the cost and function of a FEX appeals to you, understand that you need to have Nexus switches to plug them into – or they’re bricks. A FEX behaves like a line card in a chassis, only while some chassis line cards might support local switching, a FEX does not; all switching happens at the host switch.

Notably, the Nexus product line offers no PoE options, which perhaps demonstrates more than any other lacking feature Cisco’s intended use of Nexus gear. I’ve been in some “what if” conversations where a user-facing access layer made up of FEXen back-hauled to a Nexus 5K or 7K in a central location makes a sort of sense, but I don’t know anyone that’s actually done it. The lack of PoE makes that a hard sell in many enterprises, as IP phones are  commonplace and other devices benefitting from PoE are making the news.

While closing the gap with each release of NX-OS, some software features in the Catalyst line might not be available in the Nexus line. Depending on your enterprise’s specific needs, you could conceivably run a Nexus campus (I doubt there’s many of these in the wild), or a combined Nexus core and Catalyst edge (probably a more typical deployment in recent years). But from a standpoint of keeping operating systems more or less aligned, I can see some shops choosing to be all Catalyst once again, now that the 6800 line has popped up. This matter of CLI operating system will continue to be an operational issue for as long as network operations revolve around keyboard jockeys…which won’t be forever. But it still matters today.

The newest Nexus switches are not aimed at the enterprise, but rather the large data centers with serious bandwidth and scaling requirements. The Nexus 6004 is a monster 40GbE aggregation or core switch that makes immediate sense in a leaf/spine topology using FabricPath or ECMP L3 acting as an underlay for a multi-tenant environment. The Nexus 7700 chassis offers more aggregate bandwidth than the 7000, along with true front-to-back airflow for easier hot-aisle/cold-aisle deployment, and a power grid that’s a little easier to cope with. Does the average enterprise need a 6004 or a 7700? Not impossible, but not likely.

SUMMARY

In light of all this rambling about Cisco switches, what would I buy if I were planning a greenfield enterprise or campus switching deployment?

  1. I like Cisco switching gear, and have worked on it for many years now. While I have my complaints about software quality and the occasional batch of bad hardware that have showed up on my dock, my overall experience with Cisco Ethernet switches is that they just run. That said, I’m not stuck on the Cisco brand. I believe that the aforementioned HP, Brocade, Arista and Juniper all make Ethernet switches worth looking at. Enterasys has a fascinating line of switches, some of which have a clever ASIC I’ll be writing about in the future. Dell bought Force 10 and has been working on integrating them into their enterprise product portfolio. Extreme is still out there selling a full complement of switches. And there’s a number of other brands out there, including vendors with a “whitebox” switch approach. What I would do is take a look around, forsaking Cisco myopia. If you end up back at Cisco, so be it. Depending on your vendor and reseller relationships, that might be exactly the right thing to do. But be aware of a huge number of legitimate non-Cisco options.
  2. If I was going to buy Cisco for whatever my reasons were, I would not lock myself into one product line or another. I would instead build a list of specific functionality required, and then mix and match whatever was most appropriate. In some cases, this might be less of a technical or feature decision, and more of a cost decision. Capex is going to be significant for either Catalyst or Nexus product lines, but Catalyst in general will probably reduce spend overall. Bundles, licensing and optics will bear on this, so I would (and in fact have been) cross-shop Catalyst and Nexus products. Don’t overlook ongoing opex in the decision, either. Nexus gear tends to cost a goodly amount to cover, although in fairness, Catalyst 6500s have been a kick in the pants for many a SmartNet contract over the years. There is no one right answer, which is sort of the point of Cisco’s plethora of choices. Whatever you’d like to do, Cisco has a tailorable answer likely to make you happy.
  3. For me, I prefer NX-OS over IOS. NX-OS is modular, which presumably has a positive impact on stability, and has a number of CLI improvements over IOS, in my opinion. I shift between NX-OS and IOS all the time, and find that I miss certain small NX-OS behaviors when riding an IOS CLI. Is the OS a reason to consider one switch over the other? Consider your operational processes and ask your operators.
  4. I would look forward in time to how Cisco is going to support network programmability across their product lines. At the moment, it’s hard to go wrong with either the Catalyst or Nexus product lines from a onePK standpoint. But if OpenFlow is now or might become important to you, the future is muddy. OpenFlow functionality and performance is tied to silicon, and Cisco’s not making the strides toward OpenFlow support that other vendors are, despite new silicon showing up recently. In fairness, if we assume silicon design takes 3-5 years to design and bring to a production cycle, OpenFlow wasn’t even on the radar back then. Still, if OF matters to you, then Cisco might not be your ideal switch vendor. Keep half an eye on the OpenDaylight Project as well as the eXtensible Network Controller to monitor whether or not OF gets marginalized as a southbound protocol. (My highly speculative guess is that OF will fade in importance over time. Cisco is big enough and tied so closely to ODL that they can use whatever southbound protocols they like and make it a de facto standard. I don’t see anyone at Cisco waving the OF flag too highly as yet.)
  5. I would finally consider building an enterprise switching infrastructure that supports multi-tenancy. While multi-tenancy usually comes up in the context of cloud or IaaS providers, I believe that the model of a multi-tenant overlay network maps well to most enterprises. All the enterprise has to do is think of its tenants as business units or perhaps as business functions. Building out network containers in the form of overlays (MPLS, VXLAN, NVGRE) allows an IT infrastructure to securely segment discreet business operations. This is useful for isolating data sets from one another (hello, PCI-DSS & HIPAA!) and keeping development environments separate from production environments, all while leveraging the same physical underlay network. Enterprises could better utilize available network capacity in this way, while at the same time reducing the network’s physical footprint. How well do the Catalyst and Nexus product lines map to this sort of a design? That depends on what sort of overlay interests you and just where you see your tunnel endpoints existing (i.e. in hardware or out at the edge in a software switch).

Happy shopping, Cisco fans.