What You May Not Know About Unit Testing — Part 1

I come from an Electrical Engineering background. Engineers in part use applied physics and mathematics to build mathematical models and evaluate and test those models before anything is physically constructed.

When I worked as an electrical engineer, we built such models and simulated an entire mainframe before any bit of silicon was manufactured. It can be prohibitively expensive in terms of time, labor and building materials to erect 50 floors of a building only to find that the structural engineering is flawed and untested.

Software developers often do not have such physical restrictions. Bugs in social media software may be annoying, but their ramifications are usually mild. Yes, it can cost time and money to fix software, but the “building materials” are comparatively inexpensive; they are mostly text files that are edited.

We have known since the 1970s that the longer it takes to find a bug, the more expensive it costs to fix it. I have seen release dates slip and budgets overrun due to regression errors found late in the release cycle.

I once saw a slide deck that had a slide with a quote attributed to Boeing. I am not sure if the quote were truly from Boeing, but the quote was “If you can’t find the time to do it right, then how can you find the time to fix it?” Yet this is what usually happens with the SDLC.

Software typically does not have the safety ramifications that industries such as aviation has. If Boeing creates defective software, then catastrophes such as the 737 Max accidents may happen. By the way, aviation is an industry for which more rigorous software testing practices are prescribed by the FAA. Most of us are not regulated in such a manner, except perhaps by the customer.

It is true that any non-trivial software system cannot be proven to be free of defects. Edsger W. Dijkstra opined, “Program testing can be used to show the presence of bugs, but never to show their absence!”

I believe that we can and should do better. Customers have come to accept that software is buggy, but it does not need to be that way.

This article is the first of a series of hopefully weekly articles in which I hope to better elucidate unit testing practices and base those practices based upon solid engineering and mathematical principles which most software developers do not consider. Some of the topics to be discussed are:

· What is a unit test?

· The role of unit tests in a testing hierarchy

· Properties of good unit tests

· How do I test private methods, or should I?

· Types of unit testing including mutation testing.

· Is code coverage all people claim it to be?

I hope to keep the discussions high-level and use pseudo-code as much as possible. My hope is that this series provides big ideas to consider versus dogmatic practices to follow and that better unit testing and higher quality software are results.