We can summarize the points of Lord Peter’s methodology that apply to software debugging under the following categories:
Use alibis as clues
Eliminate impossible causes
Reason based on facts
Use the power of logic
Use a system for organizing facts
Exercise caution when searching
Use gestalt understanding
Show how something could be done
Once again, we apply the analogy of viewing software defects as crimes to be solved. We must answer the following questions:
Who did it?—suspect
How did the culprit do it?—means
When did the culprit do it?—opportunity
Our approach to motive is somewhat different, since we assume that all defects were caused accidentally, as we explained previously. We are interested in an answer to the question, why did this problem happen? We treat the why question the way an accident investigator would, rather than the way a detective would. The detective seeks to assign guilt for a crime. The investigator seeks to find contributing causes, to prevent the accident from occurring again.
In Have His Carcase [Sa32], Lord Peter works with the mystery writer Harriet Vane. They search for the murderer responsible for a body Harriet found on a beach while on holiday. After a serious investigation, Lord Peter works his way through a long chain of deductions while conversing with one of the suspects:
“Always suspect the man with the cast-iron alibi”—was not that the very first axiom in the detective’s book of rules? And here it was—the cast-iron alibi which really was cast-iron; meant to be scrutinized; meant to stand every test—as how should it not, for it was truth!
Later on in Have His Carcase, Lord Peter realizes that the time of death wasn’t what the inquest determined, and the alibis of the suspects pointed to the correct time.
“Yes,” he said when he’d recovered himself a little, “but here’s a snag. If he might have died at any time, how are we to prove he did at twelve o’clock?”
“Easy. First of all, we know it must have been then, because that’s the time these people have an alibi for. As Sherlock Holmes says somewhere: ‘Only a man with a criminal enterprise desires to establish an alibi.’”
An alibi has three components: time, place, and corroboration. Detectives attempt to break alibis with questions like the following:
Was the time of the alibi actually the time of the crime?
Did the suspect have time to move from the place of the alibi to the place of the crime within the time period that the crime was committed?
Could the corroboration be accidentally or deliberately wrong?
A program that has side effects has no alibi for the time a problem occurred or the place it occurred. A side effect causes a change in the state of computation outside the context in which it’s presently occurring. Software side effects come in several forms. Some languages, such as C and C++, have pointers. Pointers allow a program to modify memory at any address to which they point. Other languages, such as Fortran, have aliases. Aliases, like pointers, make it possible for a module to modify storage in places far removed from the immediate context. Some languages, such as C++ and Java, support exceptions. Generating an exception that isn’t handled in the immediate context can cause code in arbitrary parts of the system to be executed.
The debugging equivalent of an alibi is the assertion that a given system component can’t be the cause of a problem. Software engineers break alibis when they search for bugs with questions like the following:
Was the time the defect occurred distant from the time it was mani fested due to side effects?
Was the location of the defect distant from the location it was manifested due to side effects?
Is the information giving a component an alibi misleading or incorrect?
In The Unpleasantness at the Bellona Club [Sa28a], Lord Peter is asked to investigate the circumstances of the death of the ancient General Fentiman. The inheritance of a large fortune depends on determining precisely when he died. After an exhumation and autopsy determine that the general did not die naturally, Lord Peter has the following conversation with his friend Detective-Inspector Parker:
“What put you on to this poison business?” [Parker] asked.
“Aristotle, chiefly,” replied Wimsey. “He says, you know, that one should always prefer the probable impossible to the improbable possible. It was possible, of course, that the General should have died off in that neat way at the most confusing moment. But how much nicer and more probable that the whole thing had been stage-managed. Even if it had seemed much more impossible I should have been dead nuts on murder.”
In Strong Poison [Sa30], Lord Peter Wimsey first encounters the mystery writer, Harriet Vane. She’s on trial for murder, and he’s determined to find evidence to clear her. As he interrogates some of her friends, one of them suggests an alternative suspect, based on the following reasoning:
“I merely proceed on the old Sherlock Holmes basis, that when you have eliminated the impossible, then whatever remains, however improbable, must be true.”
Lord Peter agrees with the principle, but disagrees with its application, and notes that the quote actually originates with Auguste Dupin, a fictional detective created by Edgar Allen Poe.
In Unnatural Death [Sa27], Lord Peter decides to investigate the death of an aged and ailing woman, after the country doctor who attended her suspects something other than a natural death. After hearing the doctor’s story, he says the following:
“Do you know,” he said suddenly, “I’m feeling rather interested by this case. I have a sensation of internal gloating which assures me that there is something to be investigated. That feeling has never failed me yet—I trust it never will…. I always make it a rule to investigate anything I feel like investigating. I believe,” he added, in a reminiscent tone, “I was rather a terror in my nursery days. Anyhow, curious cases are rather a hobby of mine.”
Exercising curiosity is one way we leverage the labor of finding the cause of a defect. Leveraging investigation work across several problems is an important way to increase programmer productivity. Software-development methodologies and tools are several decades behind hardware-development methodologies and tools. As a result, software developers must look for innovative ways to improve their productivity. People who understand economics know that productivity improvements are the only noninflationary way to increase wages. Most programmers are interested in increasing their wages, so exercising curiosity should be important to them.
We use curiosity when we investigate potentially related problems, even before they’re reported. The quality of software products in comparison to hardware products is so pathetic that software developers must find “force multipliers” if the gap is ever to be closed.
Answering the following questions will help you exercise your curiosity:
Is there another defect of the same root cause:
In this section of code?
In this procedure?
In closely related procedures?
Is there another kind of defect:
In this section of code?
In this procedure?
In closely related procedures?
In Clouds of Witness [Sa27a], Lord Peter investigates the death of his sister’s fianc . Lord Peter’s brother stands accused of the murder and refuses to account for his movements at the time of death. After sending some evidence to a chemist for analysis, Lord Peter has the following conversation with his extremely efficient butler:
“Must have facts,” said Lord Peter, “facts. When I was a small boy I always hated facts. Thought of ’em as nasty, hard things, all knobs. Uncomprisin’.”
“Yes, my lord. My old mother … she always says, my lord, that facts are like cows. If you look them in the face hard enough they generally run away. She is a very courageous woman, my lord.”
Facts aren’t complete without qualification. “It doesn’t work” is the most useless of error reports. The following information should be included with a basic defect report for a programmer to diagnose the problem most effectively:
Who observed the defect?
What was the input?
Where was the software run (hardware platform, operating system)?
When did the defect occur?
How often does the defect occur? (always, sometimes, once)
In Have His Carcase [Sa32], Lord Peter works with the mystery writer Harriet Vane to find the murderer responsible for a body Harriet found on a beach while on holiday. While interviewing one of the people who had a motive for killing the victim, Lord Peter has this Boolean revelation:
Mr. Weldon grappled for some moments with this surprising logic, but failed to detect either the petitio elenchi, the undistributed middle or the inaccurate major premise which it contrived to combine. His face cleared.
“Of course,” he said. “Yes. I see that. Obviously it must have been suicide, and Miss Vane’s evidence proves that it was. So she must be right after all.”
This was a syllogistic monstrosity even worse than the last, thought Wimsey. A man who could reason like that could not reason at all. He constructed a new syllogism for himself.
“The man who committed this murder was not a fool.”
“Weldon is a fool.”
“Therefore Weldon did not commit this murder.”
Aristotle originally codified the principle of noncontradiction, which says that nothing can both be and not be at the same time in the same respect.
If you believe that a phenomenon observable by the human senses is both true and false, you’re going to have a hard time using the scientific method. All classical reasoning methods depend on this principle.
If you don’t know what the law of noncontradiction is, or you don’t know how to apply the rules of logic to everyday discourse, you need to upgrade your thinking skills to be an effective bug finder. There are a number of excellent books that teach critical-thinking skills, including the proper use of logic. See the bibliography for recommended reading in developing your critical-thinking skills.
In Whose Body? [Sa23], Lord Peter is asked to investigate the accusations against an acquaintance of his mother’s who found an unidentified naked man dead in his bathtub one morning. As he discusses Scotland Yard’s analysis of some evidence with Detective Parker, Lord Peter describes the situation as follows:
Then Possibility No. 3 is knocked on the head. There remain Possibility No. 1: Accident or Misunderstanding, and No. 2: Deliberate Villainy of a remarkably bold and calculating kind….
Following the methods inculcated at that University of which I have the honor to be a member, we will now examine severally the various suggestions afforded by Possibility No. 2. This Possibility may be again subdivided into two or more Hypotheses.
The trick with enumerating possibilities is knowing when you’re done. You can enumerate possibilities by using the principle of noncontradiction and a binary tree. Repeatedly divide the logical possibilities that describe the situation into two alternatives. Place one possibility on the left-hand side of the bottom branch, and place the other possibility on the right-hand side. When there are no more logical possibilities, look at the leaves of the tree. They are the possibilities.
In Have His Carcase [Sa32], after several days of fact-finding, Lord Peter and Harriet Vane decide that it’s time to bring some order to what they have learned.
At ten o’clock Wimsey and his collaborator sat down before a neat pile of scribbling paper. Harriet was inclined to be brief and businesslike.“What system are we going to adopt about this? Do you favor the Michael Finsbury method by double entry as in ‘The Wrong Box’? Or one of those charts, made out in columns, with headings for ‘Suspect,’ ‘Alibi,’ ‘Witnesses,’ ‘Motive’ and so on, worked out in percentages?”
“Oh, don’t lets have anything that means ruling a lot of lines and doing arithmetic. Let’s behave like your Robert Templeton, and make a schedule of Things to Be Noted and Things to Be Done. That only means two columns.”
In Gaudy Night [Sa35], the mystery writer Harriet Vane attends a reunion at her Oxford alma mater and finds herself in the midst of a campaign of intimidation and murderous threats being waged against the faculty, staff, and students. When Lord Peter shows up to visit his hospitalized nephew, Miss Vane asks him to join in her search for the culprit.
Harriet handed him the loose-leaf book and an envelope containing the various anonymous documents, all endorsed, where possible, with the date and manner of publication. He examined the documents first, separately and carefully, without manifesting surprise, disgust, or, indeed, any emotion beyond meditative interest. He then put them all back in the envelope, filled and lit a pipe, curled himself up among the cushions and devoted his attention to her manuscript. He read slowly, turning back every now and again to verify a date or a detail. At the end of the first few pages he looked up to remark, “I’ll say one thing for the writing of detective fiction: you know how to put your story together; how to arrange the evidence.”
There are lots of good ways to organize a search for the explanation of a defect, but few programmers use any system. Here are several different systems for describing a software defect:
Input, output, deviance
Observation, hypothesis, experiment
Who, what, where, when, how much, how many
Test case, reference, delta
The first method describes the problem in terms of a “black box.” First, it describes the particular input that caused the problem. Next, it describes the output or behavior that resulted from this input. Finally, it describes the deviance from what was expected. This method is useful when the person reporting the problem is unaware of how the software works.
The second method describes the defect in terms of the scientific method. It begins with an observation of how the software is behaving. This is followed by a hypothesis that attempts to explain why the observed behavior is occurring. It concludes with an experiment that should verify or falsify the hypothesis.
The third method might be called “the reporter’s method.” It describes the problem by answering the questions posed by the standard interrogative pronouns. It is also useful when the person reporting the problem is unaware of how the software works.
The fourth method could be called “the tester’s method.” It describes the problem by referring to a test case that is failing, the document that is the basis for the test case, and the difference between the reference document and the behavior shown when processing the test case.
In Whose Body? [Sa23] Lord Peter is asked to investigate the accusations against an acquaintance of his mother’s who found an unidentified naked man dead in his bathtub one morning. After running down a number of possible leads to dead ends, Lord Peter goes to visit his friend Detective Parker at his home.
Peter took up the book his friend had laid down and glanced over the pages.
“All these men work with a bias in their minds, one way or other,” he said; “they find what they are looking for.”
“Oh, they do,” agreed the detective; “but one learns to discount that almost automatically, you know. When I was at college, I was all on the other side—Conybeare and Robertson and Drews and those people, you know, till I found they were all so busy looking for a burglar whom nobody had ever seen, that they couldn’t recognize the foot-prints of the household, so to speak. Then I spent two years learning to be cautious.”
“Hum,” said Lord Peter, “theology must be good exercise for the brain then, for you are easily the most cautious devil I know.”
The obvious advice to give is, don’t make assumptions. Unfortunately, this is easier said than done. Here is a method for minimizing the effect that assumptions will have on your search for the cause of a bug:
List all the assumptions you know you’re making.
List assumptions you have made in the past that turned out to be false. Keeping a notebook of your debugging efforts will help here.
List a procedure to validate each assumption, in case you run out of hypotheses and still haven’t found the defect.
Don’t start with the following emotionally comfortable hypotheses:
The defect is in code written by someone else.
The defect is in a tool you’re using.
The defect is in the operating system or hardware.
The defect was a fluke and won’t happen again.
In Whose Body? [Sa23] Lord Peter is asked to investigate the accusations against an acquaintance of his mother’s who found an unidentified naked man dead in his bathtub one morning. As he stayed up late one night, trying to put the pieces of the puzzle together, he had the following experience:
And then it happened—the thing he had been half unconsciously expecting. It happened suddenly, surely, as unmistakably, as sunrise. He remembered—not one thing, nor another thing, nor a logical succession of things, but everything—the whole thing, perfect, complete, in all its dimensions as it were and instantaneously; as if he stood outside the world and saw it suspended in infinitely dimensional space. He no longer needed to reason about it, or even think about it. He knew it.
The American Heritage Dictionary of the English Language defines gestalt as “a configuration or pattern of elements so unified as a whole that its properties cannot be derived from a simple summation of its parts.”
The human brain performs many types of visual pattern matching far more effectively than any software system available today. If you’re working with large amounts of data, display it graphically. This will enable your brain to recognize larger patterns you might not recognize from looking at numerical values.
Excessive reliance on interactive debuggers limits your ability to see the big picture. Even the largest monitor can only display on the order of ten thousand characters. While the debugger can provide a window on extremely large data aggregates, the scrolling and paging required to see the entire collection limits the pattern-recognition ability of the human brain.
You can see some patterns only by covering a desk or conference room table with diagnostic output. Younger programmers may tend to be more oriented toward looking at information on a video display. Those of us who can remember times before high-capacity bitmapped video terminals may tend to make more use of paper listings. Don’t be bound to just one medium—choose the medium that helps you see patterns.
Color-coding items in a listing is an effective way to make your brain pick out patterns. Some sophisticated debugging tools provide array visualization tools, which convert large amounts of program data into colorcoded display, such as isobar charts and temperature charts. Use these tools if they’re available to you.
In Busman’s Honeymoon [Sa37], Lord Peter and the mystery writer Harriet Vane are on their honeymoon in a quaint country house they have just purchased. Unfortunately, someone decides to murder one of the locals and deposit the body in their basement, right before they take possession of their home. Naturally, they must help the local constabulary investigate this mystery.
“You have no idea,” said Peter, irrelevantly, “how refreshing it is to talk to somebody who has a grasp of method. The police are excellent fellows, but the only principle of detection they have really grasped is that wretched phrase, ‘Cui bono’? They will hare off after motive, which is a matter for psychologists. Juries are just the same. If they can see a motive they tend to convict, however often the judge may tell them that there’s no need to prove motive, and that motive by itself will never make a case. You’ve got to show how the thing was done, and then, if you like, bring in motive to back up your proof. If a thing could only have been done one way, and if only one person could have done it that way, then you’ve got your criminal, motive or no motive. There’s How, When, Where, Why, and Who—and when you’ve got How, you’ve got Who. Thus spake Zarathustra.”
There are several ways you can use to show how a defect could occur.
Write a list of cause-and-effect pairs that show a logical chain of events that could lead to the symptom of the defect.
Draw a series of diagrams of a data structure showing how it could come into its current erroneous state.
Draw a tree showing the control-flow decisions that could result in the undesired behavior.