Those who know me also know that I read a lot. Much of my reading is technical – I try to spend at least a half hour each day on a technical topic as a way of sharpening the saw.

In one of my recent reads, the authors tossed out a sentence that first struck me as provocative:

From a business perspective, creating clean and well-written is very hard to justify.

My first reaction was probably the same as yours - "Say what?!?" Fortunately, they immediately went on to clarify exactly what they meant by that:

It's not that the business wants bad or messy code to be written, it is that our work is typically based upon business requirements. The business wants those requirements delivered and for the team to be ready to move on to other pieces of work as soon as possible. To that end, you need a good compromise.

Ah, much better.

Anyone who has worked in the software industry for any length of time has likely encountered a system that is not well designed. At best, this may be a minor inconvenience. At worst, however, the code we are being asked to maintain may be teetering under the weight of the hacks and shortcuts that have built up over time. One of the things that has to be experienced to be appreciated is how systems, most especially infrastructure systems, last far longer than anyone expects.

(Engage Way-Back Machine)

Many moons ago, when "the web" was still just a gleam in Tim Berners-Lee's eye and computers had only recently become "personal," I working in the R&D division of a major company that had an antiquated (and, to them, almost completely useless) mainframe-based time tracking system.  The input wasn't flexible enough to allow categories of work to be defined, it couldn't cope with the concept that someone might work on more than one project at a time, or for more than 40 hours a week, and getting a new report required writing COBOL.  As a result, although we all dutifully filled out our time ticket each week and sent it to be keypunched, about the only useful information that it was able to provide management was how many vacation days each employee had taken.  The division had several Sun mini-computers, however, and my boss suggested that it would be possible to create a more flexible system for the division and run it locally.  There was skepticism that a data entry interface could be built that the secretaries could use, so I was asked to knock together a prototype system to demonstrate the general concept - a few input screens running on VT100 terminals and a sample report or two to illustrate what the system might be able to output.  I quickly hacked together a prototype over a 3-4 week period, with the intent that those concerned could play with it for a few weeks to get an idea of what the "real" system could look like.  Those of you who have been around the block can see what's coming - despite periodic lobbying for the time to do the rewrite, that system remained in use for nearly ten years, and I ended up continuing to do contract maintenance on it long after I had left the company.

My partner Rob once summed things up very well:

  • The young engineer focuses on "We can build this."
  • The more seasoned individual thinks "We can build and sell this - and make a profit doing it."
  • The successful executive, however, says "I have only so many resources, and I can make more money building and selling this than that."

I'd like to say that the longevity of my code was a testament to the elegance and flexibility of its design, but the simple fact is that it lasted because it was "good enough," and it was cheaper for them to even buy consulting hours from me to keep it creaking and wheezing along than it was to replace it with a properly designed system that might have been slightly faster or slightly smoother to operate.  The division just had higher priorities for its budget.

Lesson 1: Current systems don't get replaced with completely new systems unless there's no alternative. "We need to rewrite it from scratch" will always get serious push-back. Expect to be living with the code you're now writing for a long time.

I'm not suggesting that this particular project should necessarily have been approached as a full-fledged development with specifications, test plans, etc.  It truly was intended to be a throw-away prototype.  But sometimes systems that work take on a life of their own.  We used to joke that there was value in demonstrations of research projects being rough around the edges, because if a demonstration was too polished, marketing would want to add it to the catalog next week. This is not the only "sample" that I've written that morphed into a product unexpectedly.

Lesson 2: Think about your software design, even the code you're "writing to throw away."  It may get diverted on its way to the trash can.

Probably the most important thing I learned from this project was the long-term pain of the short-sighted decision.  There was a dreadful architectural flaw in the way this system stored data.   The flaw permeated enough of the code, however, that it would have required a major rewrite to eradicate.  Since the expected rewrite never occurred, I lived with it for years.  It would not have taken much longer to write that part more elegantly and maintainably at the outset - maybe 2-3 days out of a month-long project - but "quick and dirty" had the motto, since I expected the software to have a lifetime of weeks, not years.

Lesson 3: Before succumbing to the temptation of doing something the quick way, as opposed to the right way, remember that it may be you, and not some other poor engineer, who will be living with that decision for the next decade.

To some, this may seem a narrow and somewhat contrived example.  After all, if you know you're designing a production system, you don't make such short-sighted decisions, right?  Perhaps true, but there's always pressure to deliver faster, and so temptation lurks just over your shoulder.  In addition, systems evolve in ways we can't anticipate, and your requirements a year or two from now may look very different from what they are today.  It's easy for cruft to gradually accumulate as you try to track a moving target.

This is particularly the case in so-called "agile developments" or "agile teams."  Don't get me wrong - I am 100% in favor of techniques that allow a team to rapidly add or modify features in response to customer needs.  But one of the advantages that the "old-fashioned" waterfall approaches had (have) is that they tend to have "planning phases" in which the architects and designers can spend some time working out what portions of the system's architecture may need to be reworked to be in line with the new requirements, and the longer intervals between releases are more conducive to doing such rework when necessary.  If too much emphasis is placed on "a new feature release every week," it's easy to rationalize the quick approach over the correct one, and for a team to ever-so-slowly "code themselves into a corner." When the balloon payment comes due on the buildup of "technical debt," it is then difficult or impossible for the team to extricate themselves without a very significant break in the flow of features.

In my mind, this places a greater-than-average burden on agile teams to discipline themselves to monitor their architecture and code quality, and to root out "code smells" early, before they taint too much of the system. It also creates a staffing challenge, since the detection of code smells is difficult to learn other than via experience. Thus, it can be hard to integrate a junior software engineer into an agile team without risking an increase in the buildup of debt due to his/her inexperience. (This is one of the places that "pair programming" can shine.)

Finally, it emphasizes the need for tools to help teams monitor their code. While tools can rarely evaluate high-level architecture, they can automatically and systematically monitor a number of the low-level signs that issues are creeping in. Fortunately, there are plenty of these available - individuals and teams simply need to commit to their use, and to understand what they are saying about the code.  As you might expect, I plan on discussing some of these tools and processes here from time to time, so stay tuned...

References: