This is a departure from my usual discussions of politics, law, and current affairs – I want to delve into another area of interest for me – software engineering, and in particular what constitutes useful documentation. 

I know it’s not fashionable among software developers to do “tedious” work like actually documenting design and architecture decisions.  Outside of domains where such documentation is absolutely mandated for regulatory reasons, the thinking tends to be “let’s be Agile” and just talk about it as we go.  That’s lovely for one-off projects where you might have a few thousand lines of code that is solving a singular and specific problem.  It absolutely sucks when you are creating infrastructure that you expect other developers to use. 

Here’s the problem as I see it:  Infrastructure software in particular generally depends on the user having a working understanding of how it works in order to use it in any meaningful way.  A person coming into your infrastructure a decade after you made the first design decisions has no idea what those decisions look like, or the approach to partitioning the domain that you as a developer and architect took.  It is wholly inadequate to tell us “Oh, it’s all in the API documentation” – although I can deduce certain things from the API, the public API still represents a wall to understanding the underlying architecture of the software and the assumptions it makes. 

My first exposure to this came back in the 1990s when a nameless software vendor decided to switch their implementation of the UNIX socket system from being based on BSD to being based on AT&T System V UNIX, and I had to migrate the product I was working on at the time from one to the other.  The result was catastrophic because what the vendor attempted to do was shove a “compatibility layer” in so that software using BSD socket conventions would still compile.  Sounds good right?  No – the result was catastrophically wrong. 

Here’s where documentation came to “save the day” – although the process of getting there was about as much fun as removing porcupine quills from your backside.  It turns out that BSD sockets had different behaviours behind the scenes than System V sockets – in particular when handling certain kinds of faults. 

Sorting out the issues ended up taking a whole bunch of effort and eventually determining that for semantic reasons, you cannot successfully implement the BSD socket API on top of System V sockets (although the other way around can be done!).  If it wasn’t for actual documentation of the specifications, it would have been next to impossible to make that determination.  

You might say “well, that’s a very specific case, and it has been sorted out, so?”.  

My response to that is this:  Even in the world of Open Source software, the same issues arise regularly.  Software is complex, and not everybody has the time – or inclination – to tear apart the code you have written to figure out the underlying architecture and design decisions.  Most people want to be able to _USE_ your software, but that really does mean making visible the fundamentals of the underlying design so that when we look at the public API you provide we can understand what we are interacting with. 

API-only documentation is absolutely useless if the intent behind that API is not made visible.  Likewise, good documentation has to be searchable, indexed, and approachable from different perspectives.  _YOU_ as a developer of a particular piece of functionality may be a domain expert both in the software and the problem domain, but not everybody who comes to your software will have the same background and depth of knowledge you do.  Your documentation should reflect a working understanding of that. 

Good design documentation – even if it is written “after the fact” is still valuable because a decade after you wrote the software, someone else may encounter it and have to pick up where you left off.  In my professional career, I have had to jump into code that someone else wrote many times.  The people who bothered to document their designs made my life much easier in the long run (and usually wrote far better solutions).  

Yeah – I get it – not everybody wants to sit down and write out plans for what they are going to build – it’s way more gratifying to get into coding and make things work.  You know what?  Good design that you can actually document is almost always going to provide you with a better solution in the long run – and future you may even benefit from it.