From the blog.

Managing Digital Racket
The more I tune out, the less I miss it. But that has presented me with some complex choices for a nuanced approach to curb
Complexity – My Friend, My Enemy
Over my years of network engineering, I've learned that the fewer features you can implement while still achieving a business goal, the better. Why? Fewer

Production Designs Are Compromise – And That’s Okay

744 Words. Plan about 4 minute(s) to read this.

During a bout of stress & jet lag induced insomnia, I logged onto Twitter in the wee hours to see what folks were saying. I found a number of twitpisses in my timeline. Twitpisses are arguments in which people assert that Someone On The Internet Is Wrong in 140 characters or less. After said assertion, a debate is held, using one of the worst platforms for intelligent discourse imaginable.


Twitpisses are usually along the lines of “I’m right. You’re wrong…and probably stupid. Don’t breed.” At least, that’s how the sentiment comes across, considering the vitriol with which some of these tweets are delivered. When these arguments are about technology, which they nearly always are considering who I follow, I’m taken aback.

Technology is not always about what’s best, what’s ideal, or what’s perfect. Concepts like “best,” “ideal,” and “perfect” live in people’s minds, and rarely escape into the real world. Why? Technology designs are created in the context of very real — and usually unyielding — constraints. As a result, production designs are almost always exercises in compromise.


What are these constraints that can force technological compromise? They are many. We’ll look at a few that top my list.

1. Money. Speed costs money. Features cost money. Budgets are not infinite. Therefore, any technology project is bound by limited capex and opex. By definition, a project with cost constraints cannot get the best technology available. Not having access to the best impacts design. Certain network design choices require funding. If you don’t have the cash, you can’t execute the design. Result? Compromise.

2. Expertise. Some technologies are harder to implement and operate than others. We can talk about software defined whatever for as long as you like, but software is merely moving complexity around. Complexity is not removed; at best, it is abstracted. No matter what the future of the enterprise data center might be, it will remain complex, and experts of some sort will be required to run those IT environments.

However, not every IT shop will have the expertise to take advantage of every new technology. In addition, some organizations are unwilling to put the time into training their staff on new technology, meaning that staff is not likely to recommend technology they are unfamiliar with or can’t easily become familiar with. Therefore, a resulting design ends up being the best it could be for a given organization, considering their expertise constraint.

3. Time. Bringing anything new to life in the IT world is time consuming, but often projects exist with a hard deadline. The business must have IT capability X implemented by time Y. If X is not available by Y, there is a business consequence, probably a financial one. Therefore, a common project constraint is time, and this very often introduces compromise into a design.

For example, would the average network engineer working under a tight deadline consider a new-to-them approach to layer 2 fabric, if they are very comfortable building a robust spanning-tree design? They might not. Is spanning-tree the “best” option for an L2 forwarding design? Decidedly not. Yet, faced with what is known versus what is new, what is the average engineer going to choose? They will go with what they know they can make work successfully in the time given.

Why is this okay?

Production designs being less than ideal are okay because they have to be. There are reasons different designs go into place, reasons twitpissers probably know nothing of, perched in their ivory towers with their unlimited budgets and infinite knowledge. Compromise doesn’t make a given design wrong or inadequate. In fact, the fact that the design is in production and working for a business demonstrates a design’s probable adequacy.

I’m not trying to defend genuinely bad design. Bad designs — more often utter absence of design — certainly exists in production environments. (I’m one of those people called in to bring order to chaos. I’ve seen…things. Horrible things.) For purposes of this article, I’m bringing attention to the silly arguing among competent engineers who live in an idealistic world, where they seem to think every design is supposed to look like a Cisco validated design guide. Almost no real world network looks like a reference guide, because they can’t due to any number of constraints that introduce compromise.

We all do the best we can with what we’ve got.