7.2 Evaluation mechanisms

Every debugging strategy consists of three parts: a set of assumptions, a control structure, and an evaluation mechanism. While the set of assumptions and the control structure are unique to each strategy, the evaluation mechanisms are independent and can be used interchangeably with different strategies.

The purpose of the evaluation mechanism is to tell you whether you’re making progress. If you have been using a strategy for a while and the evaluation mechanism says that you aren’t making progress, it’s time to apply another strategy and, perhaps, try a different hypothesis.

There are several ways to evaluate whether a strategy is working.

The number of plausible hypotheses disproved should be increasing.
The effort required to reproduce the problem should be decreasing.
The size of the input required to reproduce the problem should be decreasing.
The amount of code excluded from consideration should be increasing.

These evaluation mechanisms are listed in increasing order of strength. The number of plausible hypotheses may be quite large, and more importantly, isn’t known at the time you start looking for a bug. On the other hand, the size of the input that initially causes the bug report, and the size of the program that has the reported bug, are fixed and are known when the search begins.