Part 7 – Documentation & Verification, Better Than More Integration?
At some point in any projects life cycle you should start thinking about the need for documentation and verification. This post and probably the next few will be dedicated to exploring these aspects.
As a structural engineer, if you are designing a building you have to produce some drawings, calculations, specifications, etc to describe what is being built, and someone probably needs to check these deliverables. Often the drawings are the only deliverable, so its all the end user ever sees out of your blood sweat and tears. You’ll be judged on this alone if it contains mistakes, omissions, etc. Small omissions can cost you or the client are lot of time or money to put right if the contractor has made no allowance for the items. As an engineer designing something tangible like a building, you simply cannot get away without producing drawings, someone needs to produce your design after all and they are going to need some instructions.
Software documentation is a little the same, and also a subtly different in some ways, how many people will ‘read the manual’ or delve directly into your code? Irrespective of the fact that people don’t think they need manuals or instructions, there comes a time when having a ‘manual’ can be quite important.
Whether this is as simple as comments or docstrings in your code to explain logic, an Excel sheet explaining how to use it and limitations of a spreadsheet, or something more sinister like an actual real manual, specification, blog post (such as this!), it doesn’t really matter as long as it gets across whatever message you need to convey.
''' this is a docstring, showing what a docstring looks like. It can be several lines long. They can play an important part in automatically generating documentation, more on this later. ''' # this isn't a docstring, its a comment, it's a single line affair. # every line needs a hash '#'
The end goal should be to provide enough information to convey to someone (most importantly you in the first instance) how to follow your code (and sometimes more importantly your logic or thinking at the time you wrote it). Good documentation also plays an important role in providing guidance on using the end product appropriately (assuming its being read/complied with). Even if it’s only of benefit to you in five years time, when you try to decipher what you did back in the day, hopefully it will help unravel the web you created. I look back at some of my own half finished spreadsheets and wonder how I’d every start up again on it so long after downing tools, because the documentation and any code comment are essentially nonexistent. Its like going back and reading some calculations you did 10 years ago, good luck on figuring out what you were on about if you did them in a hurry.
Similarly the need for thorough verification of design outputs is extremely important where calculations are involved. History has taught us the wrong answer in terms of structural engineering calculations or gaps in knowledge can have serious consequences! Stuff tends to fall down if you make enough mistakes, invariably soft squishy humans don’t tend to fair that well during these events.
If you are one of those people who think you never make mistakes, think again. I’d feel fairly confident placing a bet that this opinion of yourself is possibly wrong. Its like a casino, the house always wins on this one if you play the long game. Everyone makes mistakes, structural engineers at least quite regularly do in my opinion. We have insurance because we need it from time to time for good reason. You’ll learn to live with it and develop that good old engineering judgement over time, which sets off imaginary alarm bells in your brain when something doesn’t quite feel or look right. So you can investigate further or ask for independent advice of those who know more than you.
<Rant on/> Confessions of a Peer Reviewer..
Now, how someone deals with being presented with a mistake they have made says a lot about that person. I used to do a lot of structural peer reviews, trust me when I say I’ve seen a lot of mistakes in engineering calculations, structural schemes, and drawings to last a lifetime. I’d like to say most people react positively to being presented with their mistake or lack of understanding, but I’d be lying again. Many varied reactions will be observed, most not conducive to actually solving or rectifying the mistake in a timely manner:-
- the minority actually own the problem, are concerned, and focus on addressing it. I like these people, surround yourself with these types of people
- some deny all knowledge, it wasn’t them who made the mistake so they have little interest in understanding, let alone addressing the issue
- some think it will be ‘ok’ in an incorrect application of engineering judgement, I don’t want your ‘thinking’, that is what got us in this situation in the first place. I’d rather have some proof thanks, like the calculations I sent you showing it didn’t work
- some are non believers, you can explain an issue fifteen times in fifteen different ways, and they still don’t see the issue
- some act like they could never have made a mistake so you must be the one who is wrong, yeah I just like creating issues out of thin air for the thrill of it
- some just don’t know what they don’t know, the most dangerous kind of engineer, all the other points rolled into one. These people can be very hard to convince
- some argue the point, I honestly don’t want to see 60 more pages of calculations trying to prove you don’t need that extra bolt in a connection that I demonstrated you needed. Because 16 storeys above the point of interest you managed to prove that the bathroom tile being used is 11.56% lighter than you thought and therefore it’s just fine as it is. Just put the damn bolt in and we can both make some money
- some act like it’s actually your fault that you found a mistake, because you are essentially doing them a favour by finding their mistake so they can correct it, don’t take it so personally, it isn’t personal
- some throw you under the bus with the client. Projects being delayed or requiring extensive revisions because I found too many mistakes, somehow that is my fault because I’m being too thorough. You put yourself in this position buddy
Overall just accept a mistake if its a mistake, and put it right and be civil and you’ll have my respect as a fellow engineer. Though I’m happy to be proved incorrect if you can provide evidence, sometimes the calculations are so poor that you make mistakes trying to read their mind on what they were trying to do.
<Rant off/>..
So back on topic basically, in a nutshell I’m saying because we all make mistakes, then software will on the balance of probabilities have mistakes, its inevitable. Mix humans with anything and eventually you’ll get human error. Always give people raising issues or mistakes with you some air time and proper consideration, don’t be that guy or girl who thinks they don’t make mistakes so outright dismiss it without at least some consideration. Sometimes the way the issue is communicated it might not make immediate sense, drill down into it until you understand it. Similarly if you are raising the mistake, provide sufficient evidence, examples, etc. Fronting up and going your software doesn’t work, it gives the wrong answer with no examples doesn’t help anyone diagnose the issue without a game of 50 questions.
Incorrect results discovered even in one aspect of software can lead to doubt about all results even if they are actually correct. Thats just human nature, once bitten twice shy. The reality is that something like 90%+ spreadsheets have errors in some form or another based on the general consensus of research into the subject. Don’t be that person adding to the wrong side of the ledger on this aspect… verify, verify and verify.
Part of planning and implementation of any successful project is recognising the importance of documentation and verification, and dedicating sufficient time for these aspects to be undertaken. It should not be an afterthought or a box ticking exercise, it should be taken seriously. Rushed documentation equates to a rushed product.
This is especially true for something that will be released to the masses, often personal projects slink by with no documentation and little verification, but these projects often get the least amount of scrutiny because its only you using it and you know the inherent limitations.
These personal projects like spreadsheets also have a tendency to escape out into the wild and tend to morph into things other people in the office who are attracted to shiny new things will want to use blindly in sheep like fashion. Because they want to take the easy route 9 out of 10 times, it requires less thought. Recipe for disasters ahoy… I’ve personally seen it happen on a few occasions where engineers have produced a spreadsheet for one specific purpose/job, and someone else has found it on the company network and used it for other nefarious purposes that it was never intended for and ultimately ended up with something deficient being constructed.
Conclusion
Basically documentation and verification are good habits, and you need some discipline to ensure its a continual process.
Both are quite essential if you wish to disseminate your hard work to others. The next few posts will examine both verification and the documentation aspects in more detail, with some specific examples to check out if my formulation is actually correct so far, so I can say I’ve done my bit verification wise.