Of Engineering and Software

Everyone has it - the one issue they love to ponder. It is usually something of little practical significance. Almost always it is uninteresting to anyone else and they will refuse to be dragged into the discussion with you. I will admit mine - it is "Is software development engineering?".

I recently watched this video of Bob Martin, Michael Feathers, and Pete McBreen talking about the state of the software development industry. In general, their argument is one that I care about deeply and agree with - the only way to write quality software is for everyone on the development team to be obsessed with writing clean code. That makes software development more about attitude, values, and craftsmanship than about technology or education.

One interesting distinction Pete McBreen and company make is between two contrasting approaches to software he defines as "engineering" and "craft". He explains that the engineering method is expressed in the belief in formalization of software development so that developers would be reduced to factory workers, diligently translating formal design documents into machine executable code. It is that approach to software, Bob Martin and the others argue, that has lead to companies managing developers as if they were interchangeable cogs, which in turn leads to poor code and the regrettable current state of the software industry.

This is a line of reasoning I highly identify with, but I would like to challenge the implications it makes with regards the the "engineerity" of software development. I do not think any engineering activity is as formalized and as predictable as we software developers like to think it is. In essence, the idea that other engineers have a highly disciplined and formal design process, enabling anyone to switch on and off projects as required, is a myth. I will concede my supporting arguments are circumstantial, but I would like to lay them out for discussion.

I come from a family of engineers: my mother has a degree in civil engineering and my father is a road engineer. Since I, like most developers, have wondered about the nature of software, I make a point to talk to my parents to try to draw similarities or identify differences between what I do and the type of work they are involved in.

In one of the most revealing discussions, I asked my dad about what his blueprints need to go through in order for anyone to begin thinking about building a road from them. Unfortunately, the details of the process evade me, but what stood out was that the risk of mistakes was mitigated through a prolonged and serious scrutiny of the blueprints by a series of highly educated committees. My father was adamant that there were no processes or techniques that would assure an error-free design. It just happened to be that by having enough trained eyes look at the same plans, they could spot pretty much all mistakes.

In another conversation, my mother mentioned an aspect of her work that I previously thought was unique to software - she asserted there was a huge variance in the productivity of engineers based on their competence. She reckoned a good engineer is easily ten times as productive as a mediocre one. Does that sound familiar? This was especially interesting because: (1) I trusted my mom knew what she was talking about since she has owned her own engineering company for nearly fifteen years and (2) she had worked in two related, but significantly different engineering disciplines (civil and HVAC engineering). If we, software developers, observe the same differences in productivity in our industry and attribute it to the "craft" nature of our work, we would also have to concede that there is more "craft" involved in other types of engineering than we previously suspected.

This has lead me to rethink what the criteria for describing an activity as engineering are. It seems that an engineering discipline is one that is able to apply scientific knowledge and consistently deliver fully functioning artifacts that serve a particular purpose. It seems that consistency and reliability is where software development is lacking. However, as I attempted to argue previously, consistency in other engineering disciplines does not stem from the formality of the design process, but from the ease with which mistakes can be identified and corrected.

So why does software perform so poorly on those metrics? Here I can only guess, but I have factors I would like to explore further:

1. The forces affecting software development are unclear. There are a lot of specialized disciplines with very different concerns (this is something Bob Martin, Michael Feathers, and Pete McBreen touch upon in the video above). This makes it very hard to extrapolate principles that apply universally to all of Computer Science.
2. The act of software engineering is the act of language design (this is not an original thought - I know I read this somewhere, but I cannot remember where). That means that in any software system there will be, by definition, room for ambiguities and subtleties. This means that our main means of communication as software engineers (code) cannot be as reliable as a civil engineer's plan. This often means it takes as much familiarity with the code to find bugs in it when reviewing it as it does to write it in the first place.
3. Building software is cheap. We simply do not have, nor wish to spend, the resources to scrutinize our code as much as the artifacts of any other engineering discipline.

So what does that mean for you and me - the people in the trenches, writing code every day? To me it means that Pete McBreen is right about the bigger picture. It is unlikely that software development will ever become a manufacturing process. It means we should take the time and make the effort to become better developers. We should embrace the values that lead to great software and hope that our dedication will pervade our industry.