45.

We explain that effective debugging is best carried out in the manner dictated by the scientific method.

Software as Immortal Machine

A software system is a type of machine that can be run, stopped, and modified. But unlike other machines, software possesses a very special form of elegance: every copy is exactly the same and its parts never wear down. Granted, the hardware on which it runs may eventually degrade with time. But the software is not really a physical system-it's a system of designed relationships.

Because a software system doesn't wear with time, it continues to act precisely in the manner we have told it to. We, the programmers, are solely responsible for any misbehavior. Since there is an absence of extraneous noise in the system (as well as no physical wear to deal with), narrowly focused reasoning about a software system, considering only the properties of that system in isolation from the rest of the world, can pay off as an appropriate way to deal with programming issues. The payoff can be much higher than in other contexts in which we are less certain about the facts we have to work with.

Tip 

Because a software system doesn't physically degrade, any erroneous behavior is the sole responsibility of the programmers.

For just one example of how narrow reasoning doesn't work so well in other contexts, consider how the following statistical information could be interpreted:

Suppose you plan to open a martini bar, and so as to optimize your inventory on the opening night, you're trying to determine what percentage of people prefer gin martinis and what percentage prefer vodka martinis. You decide to experiment, so you take a survey and discover that, sure enough, approximately 61.27% of your respondents prefer vodka martinis. That must mean that more people prefer vodka martinis than gin martinis-maybe.

It might also mean that your random sample is not indicative of the percentage among the whole population. Or it might be that 15% of your respondents simply filled in the top answer ("vodka") on your survey without really thinking about it. Or that 5% of your respondents accidentally filled in the wrong circle, or ... you get the idea. At any rate, your sample population was wrong; on your first night in business, more people ordered gin martinis than vodka martinis.

In the wonderful world of software debugging, there is no noise in the data. If our program exhibits a bug, there is a problem. When we examine the value of some variable at runtime and it turns out to be 8 when it should be 12, it's not just the result of a bad sampling-we know that the value is not being set correctly. The ability to deduce properties of programs with unswerving certainty is our greatest asset when fixing programs. Sherlock Holmes should be the patron saint of software debugging.

Tip 

In the wonderful world of software debugging, there is no noise in the data. That puts the responsibility for system behavior squarely in the lap of the programmer.

How Much Does Your Software Weigh?

I am reminded of a story told to me by Professor Claire Cardie of the AI laboratory at Cornell University. She was once involved in a team to develop guidance software for the space shuttle. NASA required every project developing technology for the shuttle to determine how much its contribution weighed. For weeks, Claire's team insisted that its software didn't weigh anything! But NASA didn't buy it. One day, someone proposed that they could store the software on old-fashioned punch cards, and weigh the resultant stack of cards (that weight would still be insignificant compared to the weight of the space shuttle). But then someone pointed out the problem with that approach: the software doesn't consist of the cards, it consists of the holes.

Ultimately, it was decided to simply enter the smallest weight NASA would allow.

As I mentioned in 4, the act of diagnosing a software bug has a lot in common with performing a scientific experiment. In both cases, we want to take the following steps:

1. Observe the system under investigation.

2. Form a hypothesis to explain the observations.

3. Test this hypothesis with new experiments.

4. Repeat until you arrive at a hypothesis that explains all of the observed behavior.

Now, this is a reasonably good approximation of what a scientist does when experimenting, and, historically, it has worked pretty well. But for scientists, complications often arise due to the inevitability of noise in the data, faulty or inaccurate measuring devices, or other reasons. The good news for a software debugger is that in the programming world, these complications simply don't exist; the "scientific method" we described applies even better to software than it does to a physical system! By practicing this method, we can be quite effective in diagnosing bugs.

The most important part of this process (and the part that is most often overlooked) is the development of a hypothesis as to why the system is behaving as it is. To do this effectively, you need to form a model of how each component of the system interacts with the others. Some of these components will be modeled only at a high level. Others (the ones believed to be the direct cause of a bug) will be modeled quite precisely.

As new tests are performed on the system, it is essential that the programmer updates this model to account for the results of those tests. Again, since we are dealing with a digital system, it's never okay to ignore a piece of data when forming a model. If your model doesn't explain even one small test result, your model is incorrect. There is no room for shades of gray in the black-and-white world of software development.

This last point is perhaps the single most important lesson of debugging. The one small result that was ignored is inevitably the key to the true problem with the system. Don't be fooled into inferring, from the fact that some anomalous behavior is hardly noticed, that the implications on the structure of the system are also miniscule. In the world of software, small bugs can result from big problems with the source code. Similarly, small features of the source code can have large and nonlocal impact on the observed behavior of the whole system.

Small Anomalies, Big Problems

Here's one example of how a small anomaly can be the cause of much bigger problems.

In an early version of DrJava, my Ph.D. advisor, Robert Cartwright (we call him "Corky"), issued a bug report indicating that one of his Java files was displaying improperly when opened. Mangled text was appearing in the file. Because Corky's code was written in an experimental extension of the Java language, we immediately assumed that the problem was related to displaying this extended language.

I tried reproducing the bug while running Windows. I opened up the bug report in Internet Explorer, copied the text of the file, and pasted it into DrJava. The text displayed just fine. I tried eliminating all uses of the experimental language features; that action had no effect.

Next, because the bug originally occurred on Linux, I tried reproducing it on that platform. I opened the bug with Netscape Navigator. (Navigator wouldn't let me open the Java file directly in the browser, so instead I saved the file directly to disk.) Eureka! When I opened the saved file on Linux, I was able to reproduce the bug. Did that mean the bug was specific to the Linux JVM I was running?

Not so fast. When Corky rebooted his machine to Windows and opened the file, he was able to reproduce the bug, so we knew it wasn't Linux-specific. That left us in a quandary. Why couldn't I reproduce the bug on Windows?

Suddenly, a small discrepancy blazed up in my head! I remembered that I had transferred the file into DrJava differently on the two different platforms. On Windows, I had copied and pasted it out of Internet Explorer. On Linux, I had explicitly saved the file from Navigator. This seemed like a minor detail, but as soon as I directly saved the file on Windows, I was able to reproduce the bug on that platform too.

This provided a key clue about the true cause of the problem: the offending file contained tabs! But when I copied and pasted the file out of Internet Explorer, the tabs were converted to spaces.

Normally, DrJava inserts only spaces into a file. When opening a file, DrJava converts all tabs into spaces. In this broken version, it converted each tab to two spaces. Because this was done in a subclass of javax.swing.text.PlainDocument, it broke an undocumented invariant in that Swing class. This invariant was discovered by Brian Stoler, the lead developer of DrJava at the time. In his words:

The standard Swing code that handles reading from a file into a Document (in DefaultEditorKit.read()) assumes that each chunk it inserts will have the effect of changing the length [by] exactly the same amount as its argument. I proved this by changing the tab remover to replace it with only one space; that got rid of the problem!

So, now, DrJava inserts only one space per tab (but you can easily reformat the code in your file once it's opened).

As this example demonstrates, it's the details that are relevant when diagnosing bugs. The minor detail as to how the text was transferred into DrJava when trying to reproduce the bug provided the key clue as to what the underlying problem was.

Another point that this example illustrates: don't overlook the simple explanations when trying to explain a bug. We were sure that the problem had something to do with the complexities of rendering text from an extended version of Java, but really the problem was just with tabs. More often than not, bugs are caused by simple problems.