37.

Incorporate Debug Tests into Unit Test Suites

Because one small test will often lead us to discover the true source of a bug, it follows that effective debugging should involve lots of testing. There's simply no way to build a reliable system without thoroughly testing it.

Although many programmers may be unfamiliar with the XP style of coding (in which writing the tests is interleaved with writing the code), there is one form of testing that everyone does while coding: debugging. Some programmers write these tests using print statements, others write them to work with a standalone debugger, others with the debugger in their IDE of choice.

Quite often, you will see programmers discard these tests once they've gotten the program to run correctly. This is a waste of very good tests. Why not incorporate them into the unit test suite over a program? After all, if one of them were to exhibit an unexpected result, we'd want to know about it.

Incorporating debugging tests into the unit tests can be done with relatively little effort, but there are some adaptations that have to be made. Frequently, we'll write tests for debugging to display information about the internal state of the program, whereas we'll write unit tests to signal only if they fail. That's because debugging and unit testing serve different purposes.

When debugging, we are trying to form a more accurate model of the program's behavior, so as to diagnose a bug. But when testing, all we want to know is whether the code passed the tests. If instead we wrote the unit tests to print out a result and then manually checked that it was what we expected, we'd waste a lot of time (or, more likely, we would seldom run the unit tests). So, unit tests tend to be written such that the result is solely one of "pass" or "fail".

Debugging tests can still be used as unit tests with only slight modification. Consider that when the program is working correctly, the result of running a debugging test will match some expected result. It is straightforward to modify such a test so that, instead of simply printing out this result, it compares the result to what's expected. It can then be incorporated into a unit test suite quite easily.

Just as debugging helps in developing a large suite of unit tests over a program, unit tests can help significantly when debugging. When diagnosing a bug, if you can first run a suite of unit tests and verify that they pass, you can rule out a huge number of potential explanations for a bug. In this way, unit tests allow you to leverage your cognitive energy when modeling program behavior. This is yet another way that debugging is like performing a scientific experiment.

When a physicist forms an explanation of an experimental result, he automatically rules out all sorts of explanations that would defy a set of accepted principles about the way the world works. For example, he assumes that the results of his experiment will not change depending on the current weather conditions on Jupiter (well, unless he is performing an experiment on Jupiter), or depending on what he plans to eat for dinner. Unit tests enforce accepted principles of program behavior.