Going Back to Graduate School

Three months ago I decided to quit my job to go back to school. There were many reasons, but the biggest was that for the first time in my life I felt like going to school would be an opportunity to grow, rather than the next obvious step in my life. Working in the industry had assured me I loved building software and taught me how to do it well(-ish). I became obsessed with getting better as a software developer and I felt the time was right to indulge in that obsession and get a post-graduate degree.

Whether my decision was sound is to be determined (it certainly came with significant compromise). I have only been back in school for a month and I certainly cannot tell yet. However, there are some things about the state of the software industry that are beginning to make sense.

I should step back for a second and provide some context. When I graduated college I, like many, was scared. I had been a good student (especially in Computer Science courses) and had pushed myself more than most. However, I did not feel like I knew anything about what it took to be a programmer. I was aware there were good ones and bad ones, which only made me more nervous because I could not begin to imagine how I would distinguish between them. Which kind was I?

I got an answer my first week on the job. I was not a programmer at all. I was completely clueless about what it was we were doing in the deepest possible way. It wasn't a case of "I didn't know the language/framework/architecture". I would sit and code with people and I could not follow what we were doing or why we were doing it. I was profoundly lost.

Thankfully, through hard work I gradually got better until I became an actual asset to the team. It was an extremely emotional and satisfying journey. At the end of it, however, there was a question nagging me - why did I have to go through this? Did I not invest four years of my life studying how to perform at this job? I'm aware no one learns what they need to do their job at school, but how low are we setting the bar here? From a purely practical standpoint, there was a real case to be made that my undergraduate degree was useless.

So now, backed with whatever wisdom I attained working in the industry, I'm in an environment where I can try to get some clarification on this topic. I have no conclusions yet, just three interesting observations:

1. It is bizarre how little coding we do.

It has been a month and I have only needed to code about 20 simple lines of AMPL. That's considering I have 6 subjects. Yes, there are projects coming that will involve code. Yes, coding is not really what computer science is about. Still, it does provide some insight into the original question of why I was so lost.

2. There is a lack of stress on code quality.

Take the AMPL assignment mentioned earlier. AMPL is essentially a DSL for specifying mathematical models that can then be solved by a machine (in order to maximize or minimize some objective formulated in the model). To solve a model, the solver requires input data that it can use for simulations. For understandable reasons, if in the input data file you refer to a quantity by a certain name, the model needs to refer to it by the same name. Now, since the data is provided by the professor, they implicitly choose the names for the variables in your model. In this specific instance the model called for a set of materials a particular shipping company was willing to provide. In the data file that set was named M. Other quantities were also given single-letter names (not all of which made sense because the professor is Italian and had used the first letters of the Italian words). This was a problem for me after a while because I kept forgetting which letter stood for which quantity. I ended up just editing the data file with more descriptive names. The result was extraordinary - the whole model almost read like English. I'm posting it here, hoping that it would be clear what it is it does.

set Materials;

param Weight{Materials};
param Volume{Materials};
param Gain{Materials};
param MaxWeight;
param MaxVolume;

var included{Materials}, binary;

maximize TotalGain:
    sum{material in Materials} Gain[material] * included[material];

subject to WeightConstraint:
    sum{material in Materials} Weight[material] * included[material] <= MaxWeight; 

subject to VolumeConstraint:        
    sum{material in Materials} Volume[material] * included[material] <= MaxVolume; 

Even if you do not understand it, it looks like good code, doesn't it? It is interesting that we are not pushed to do this all the time when it is so simple.

3. There are courses where figuring out what to teach is nearly impossible.

Take my Software Engineering course. What practices are there that are accepted in the industry? I can only think of unit testing, and even there it's far from unanimous. The solution in both institutions I have attended has been to mention all the weird things that we are currently experimenting with and then teach some subset of relevant standards. This is clearly not a great start, but it gets worse considering most of the standards are bizarre and of dubious value (I'm looking at you, SRS).

Observing this I have no answers. Actually, it makes me wonder if CS education related at all to what you need in the industry. I think this is an interesting one to try and answer. Therefore, I've decided to run a small experiment - I will try to run my school projects the we would have done in my previous job. That means estimating based on velocity, delivering features incrementally (and reviewing with professors for feedback), TDD, build scripts, acceptance testing instead of documentation where possible, etc. I'm curious to see if that translates into better academic results. I will post here as things come up.

Of Engineering and Software

Everyone has it - the one issue they love to ponder. It is usually something of little practical significance. Almost always it is uninteresting to anyone else and they will refuse to be dragged into the discussion with you. I will admit mine - it is "Is software development engineering?".

I recently watched this video of Bob Martin, Michael Feathers, and Pete McBreen talking about the state of the software development industry. In general, their argument is one that I care about deeply and agree with - the only way to write quality software is for everyone on the development team to be obsessed with writing clean code. That makes software development more about attitude, values, and craftsmanship than about technology or education.

One interesting distinction Pete McBreen and company make is between two contrasting approaches to software he defines as "engineering" and "craft". He explains that the engineering method is expressed in the belief in formalization of software development so that developers would be reduced to factory workers, diligently translating formal design documents into machine executable code. It is that approach to software, Bob Martin and the others argue, that has lead to companies managing developers as if they were interchangeable cogs, which in turn leads to poor code and the regrettable current state of the software industry.

This is a line of reasoning I highly identify with, but I would like to challenge the implications it makes with regards the the "engineerity" of software development. I do not think any engineering activity is as formalized and as predictable as we software developers like to think it is. In essence, the idea that other engineers have a highly disciplined and formal design process, enabling anyone to switch on and off projects as required, is a myth. I will concede my supporting arguments are circumstantial, but I would like to lay them out for discussion.

I come from a family of engineers: my mother has a degree in civil engineering and my father is a road engineer. Since I, like most developers, have wondered about the nature of software, I make a point to talk to my parents to try to draw similarities or identify differences between what I do and the type of work they are involved in.

In one of the most revealing discussions, I asked my dad about what his blueprints need to go through in order for anyone to begin thinking about building a road from them. Unfortunately, the details of the process evade me, but what stood out was that the risk of mistakes was mitigated through a prolonged and serious scrutiny of the blueprints by a series of highly educated committees. My father was adamant that there were no processes or techniques that would assure an error-free design. It just happened to be that by having enough trained eyes look at the same plans, they could spot pretty much all mistakes.

In another conversation, my mother mentioned an aspect of her work that I previously thought was unique to software - she asserted there was a huge variance in the productivity of engineers based on their competence. She reckoned a good engineer is easily ten times as productive as a mediocre one. Does that sound familiar? This was especially interesting because: (1) I trusted my mom knew what she was talking about since she has owned her own engineering company for nearly fifteen years and (2) she had worked in two related, but significantly different engineering disciplines (civil and HVAC engineering). If we, software developers, observe the same differences in productivity in our industry and attribute it to the "craft" nature of our work, we would also have to concede that there is more "craft" involved in other types of engineering than we previously suspected.

This has lead me to rethink what the criteria for describing an activity as engineering are. It seems that an engineering discipline is one that is able to apply scientific knowledge and consistently deliver fully functioning artifacts that serve a particular purpose. It seems that consistency and reliability is where software development is lacking. However, as I attempted to argue previously, consistency in other engineering disciplines does not stem from the formality of the design process, but from the ease with which mistakes can be identified and corrected.

So why does software perform so poorly on those metrics? Here I can only guess, but I have factors I would like to explore further:

1. The forces affecting software development are unclear. There are a lot of specialized disciplines with very different concerns (this is something Bob Martin, Michael Feathers, and Pete McBreen touch upon in the video above). This makes it very hard to extrapolate principles that apply universally to all of Computer Science.
2. The act of software engineering is the act of language design (this is not an original thought - I know I read this somewhere, but I cannot remember where). That means that in any software system there will be, by definition, room for ambiguities and subtleties. This means that our main means of communication as software engineers (code) cannot be as reliable as a civil engineer's plan. This often means it takes as much familiarity with the code to find bugs in it when reviewing it as it does to write it in the first place.
3. Building software is cheap. We simply do not have, nor wish to spend, the resources to scrutinize our code as much as the artifacts of any other engineering discipline.

So what does that mean for you and me - the people in the trenches, writing code every day? To me it means that Pete McBreen is right about the bigger picture. It is unlikely that software development will ever become a manufacturing process. It means we should take the time and make the effort to become better developers. We should embrace the values that lead to great software and hope that our dedication will pervade our industry.

Books

Sometimes, it is truly embarrassing to be a software developer. Here is a picture of the "Computers" section of the Harvard bookstore:



"Mathematics" is better, but far from brilliant:



It is generally believed that programmers do not read books. In fact, people are trying to make money by exploiting that fact. I don't find that surprising considering it is so hard to find quality books while casually browsing. Even "hits" like Beautiful Code are a rarity in bookstores. Sad...

Method Naming and Code Readability

I've been trying a different approach when naming methods. It is difficult to define it precisely, but the gist of it is in approaching each line of code as a sentence. Let me illustrate with a trivial example. Suppose in our application we were keeping track of information on widgets. We would need clearly to persist some information stored on a Widget object. Previously, I would write the method signature to store a Widget like so:

public void storeWidget(Widget widget) {
//make it happen
}

This is fine, but a bit clunky considering most clients would use the method in this pattern:

Widget widget = new Widget()
//do things to the widget
widgetRepository.storeWidget(widget)

To avoid that, I've been looking for opportunities to name methods in the following fashion:

public void store(Widget widget) {
//make it happen
}

Which means our client's code would look like this:

Widget widget = new Widget();
widgetRepository.store(widget);

I know this is probably a little too trivial to bring up, but I feel that it makes a difference in readability overall. Consider another example:

Year year = Year.fromInt(2008)

vs

Year year = Year.of(2008).

It seems that if you get this right, you might get your code to read a lot better with minimal effort. However, this quickly leads to a couple of things happening. Firstly, it forces you to become a lot more interested in language. If you have a rich vocabulary and deep knowledge of sentence structures, it is much easier to come up with terse, meaningful names. Secondly, it causes you to fall out of love with the standard Java getX() naming strategy, because it rarely allows for a fluid read. The second point actually makes me a bit nervous. I know a lot of people that are much smarter than I am have debated for a very long time over what the standard accessor Java syntax should be. It feels wrong to go against their judgment, but it I'm finding it difficult to think of counter-arguments. I promise to post about how this bites me in the end.

Google Maps Mash-Up

I've wanted to check out the Google Maps API for over a year now, but somehow I never got to it. Perhaps it's because of my limited knowledge of JavaScript that I didn't think I would be able to do anything cool easily. Silly me - the folks at Google have, of course, made the thing surprisingly easy and fun to use.

I was thinking that it would be great to have some code that would just let you list a bunch of addresses and it would use the API to automatically look those up and add them to a map. Simple enough concept, right? I was almost certain there are one thousand implementations of this somewhere on the web. To my surprise, I couldn't find any - it was time to write my own. Cue fun!

After an hour or two of messing about and some refactoring, I ended up with the following JavaScript:

var map;
var geocoder;

function load(addressArray) {
if (GBrowserIsCompatible()) {
map = new GMap2(document.getElementById("map"));
geocoder = new GClientGeocoder();

//model map: http://maps.google.com/maps?ie=UTF8&ll=39.842286,-95.800781&spn=29.450796,59.414062&z=4&om=1
map.setCenter(new GLatLng(39.842286, -95.800781), 4);

map.addControl(new GLargeMapControl());
map.addControl(new GMapTypeControl());

addToMap(addressArray);
}
}

function addToMap(addressArray){
for(i = 0; i < addressArray.length; i++) {
geocoder.getLatLng(
addressArray[i],
function(point){
showMarker(point);
});
}
}

function showMarker(point) {
var marker = new GMarker(point);
map.addOverlay(marker);
GEvent.addListener(marker, "click", function() {
zoomTo(point, map.getZoom() + 5);
});
}

function zoomTo(point, zoomLevel) {
map.setCenter(point);
map.setZoom(zoomLevel);
}

function zoomToAddress(address) {
geocoder.getLatLng(
address,
function(point){
zoomTo(point, 15);
});
}


Therefore, all you need to do is save the above to a file called "googleMapsMashUp.js", and import it into an HTML file like so:

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="content-type" content="text/html; charset=utf-8"/>
<title>Messing with Google Maps</title>
<script src="http://maps.google.com/maps?file=api&amp;v=2&amp;key=ABQIAAAAirskmcPxdaBnRV4VFH7YkhT2yXp_ZAY8_ufC3CFXhHIE1NvwkxSlN6WRvp7vscdHFS6a1w7N2-4iXw"
type="text/javascript" />
<script language="JavaScript" src="googleMapsMashUp.js" />
</head>
<body onload="load(['Quincy Market, Boston, MA',
'350 New Campus Drive, Brockport, NY 14420',
'55 Morton St, NY, NY'])" onunload="GUnload()">
<div id="points" style="width: 250px; float: left">
<a href="maps.html">All</a>
<ul>
<li><a href="#" onclick="zoomToAddress('Quincy Market, Boston, MA')">Quincy Market</a></li>
<li><a href="#" onclick="zoomToAddress('350 New Campus Drive, Brockport, NY 14420')">SUNY Brockport</a></li>
<li><a href="#" onclick="zoomToAddress('55 Morton St, NY, NY')">New York Apartment</a></li>
</ul>
</div>
<div id="map" style="width: 750px; height: 450px; float:right"></div>
</body>
</html>

After doing that, you should get a page that looks like this:


There are several improvements that can be done, but as a preliminary version this does what I wanted it to do. Hopefully you'll find it useful, too. Enjoy.

Fall in Boston

Boston is really beautiful in the fall. This picture is of the Chestnut Hill Reservoir (right outside of my apartment building).