Our brief survey has shown that a number of cognitive psychologists have developed similar models of human thought processes. While they don’t claim that the human mind is physically structured as the information-processing models suggest, they have presented a great deal of experimental evidence that these models do describe the apparent behavior of the human mind.
One point of similarity among these models is the notion of conscious controlled thought and subconscious automatic thought as different modes of mental activity. Conscious thought is slow, effortful, and sequential. Subconscious thought is fast, effortless, and parallel.
A second point of similarity among these models is the notion of cognitive structures. Working memory is the domain of conscious thought, while the knowledge base is the domain of subconscious thought.
The procedural part of the knowledge base contains interrelated data structures with a uniform format. One popular format for these structures to take is a set of rules, each with a set of conditions to be satisfied, and a set of actions to perform if the rule holds. Many cognitive psychologists believe that human thought is mostly pattern matching the activation conditions of rules, rather than logical deduction.
The study of human error is a relatively recent phenomenon. Researchers have developed several similar taxonomies of human error. The initial similarity developed from common definitions of basic terms. Further similarities developed from Rasmussen’s hierarchy of human problem-solving activities and resulted in cataloging failure modes observed in research about specific issues.
Errors occur when a planned sequence of mental or physical activities fails to achieve its intended outcome. Slips and lapses are problems that occur when a plan is executed incorrectly. Mistakes are problems that occur due to formulating a plan incorrectly.
Rasmussen developed a three-level classification of human performance: skill-based behavior is controlled by stored sequences of actions in time and space; rule-based behavior is controlled by stored if-then rules; knowledge-based behavior is controlled by conscious logical and analytical reasoning.
An experienced programmer performs skill-based actions to type any number of low-level programming constructs without conscious thought. Novices have to think about how to compose these constructs. Rules encode the experience of an expert programmer in a way that can be applied with much less effort than a novice will apply to solving the same problem.
When novices in a given competence domain make errors, they come from incompetence and are mostly random. When experts in a given competence domain make errors, they mostly come from misplaced competence.
Inattention occurs when you fail to make an attention check at an important point while executing a routine sequence of actions. Overattention occurs when you make an attention check at an inappropriate point while executing a routine sequence of actions.
Capture errors occur when your attention is claimed by an internal or external distraction at a critical point in the execution of a sequence of actions. When it’s time to continue execution of the sequence, a sequence that has stronger weight and has the same initial set of steps hijacks the execution.
Interruption errors occur when your attention is claimed by an internal or external distraction at a critical point in the execution of a sequence of actions. When your attention returns to executing the sequence, you omit one or more steps from the sequence and continue on.
Delay errors occur because a delay intervenes between the time you decide to do something and the time you’re in a position to start executing the sequence of actions. When your attention returns to the task at hand, it’s no longer in control, and you stop doing the planned action.
Multiple match errors occur because the recognition schemata accept a match for an object that looks similar, is in a similar place, or does a similar job. Many of the anecdotal cases of this error involve putting things into, or taking things out of, the wrong container.
Mixed sequence errors occur when two currently active sequences or two elements of a sequence become entangled. This can occur because the current sequence triggers others with which it’s associated or because attentional resources are being divided between two sequences that are both being executed at the same time.
Sensory interference errors occur when you’re performing an action sequence and the arrival of sensory data intrudes into the active action sequence. Overattention errors manifest themselves as omissions, repetitions, and reversals. They are more likely when the task at hand involves a series of steps mixed with periods of waiting between the steps. Silverman documents a number of errors that occur during skill-based reasoning.
Misapplication of good rules occurs when a rule that has proven to be effective in the past is applied in a situation that has the indicated conditions, but also contains additional conditions that require the execution of a different rule. Application of bad rules occurs when not all of the conditions of a situation are represented correctly or when the action corresponding to a set of conditions will result in an ineffective or inefficient action.
The first time a person runs into an exception to a general rule, especially when that rule has been an effective solution to the problems encountered, the more likely it’s that they will choose the general rule, despite the exceptional condition. The cognitive psychologists theorize that these situations cause the mind to generate rules to handle exceptions.
In a complex problem-solving situation, a person may be exposed to a variety of information as he or she evaluates rules. Contrary indicators can influence the choice of a general rule over a specific one, and the irrelevant information can cause problems just by distraction.
If a number of conditions are relevant to the problem being analyzed, it’s possible that some of them will simply fall out of consideration due to capacity problems. One of the possible consequences of information loss is that several rules will match, and the wrong one will be chosen.
A rule that has been successful in the past has greater strength. The first time a person runs into an exception to a general rule, it’s more likely that he or she will choose the general rule, despite the exceptional condition. This is especially true when that rule has been an effective solution to the problems encountered.
Sometimes people apply a general rule when they should apply an exception rule because the weight of the general rule is so strong. As people formulate rules to solve problems, they learn to focus on the key indicators of the circumstances. If they ignore other information when it’s important, an error occurs. Humans are strongly biased to apply techniques that worked in the past, even though the current circumstances are no longer the same.
The condition part of rules can be incorrect in several ways. Aspects of the problem to be solved may be not represented at all in the condition part of a rule that is intended to address the problem. Other aspects of the problem may be represented inaccurately. An erroneous general rule may be effectively nullified by a series of valid exception rules that prevent its activation.
Rule actions can be wrong because someone learned how to do something incorrectly or how to do something that is no longer, or is only sometimes, correct. Sometimes people just learn things incorrectly. When the algorithms they use generate nondeterministic results, they may be shielded from their misunderstanding for a long time.
There are often many ways to solve a programming problem. Sometimes the rules we learn to solve one problem later introduce other problems. There are also rule sets that are sufficient to achieve the goal under normal circumstances, but that occasionally cause problems.
People tend to focus on those aspects of a problem that are emotionally or psychologically interesting, rather than those that are logically important. If we explicitly record the reasons for our choices, we may be less likely to choose the symptoms that are appealing, rather than those that are important.
Research has shown that the storage capacity of the conscious mind performing knowledge-based reasoning is extremely limited. Once again, a written record of our debugging efforts is an important resource for avoiding reasoning errors due to memory capacity limitations. We also know that people give too much weight to those facts that readily come to their minds. The written debugging log can balance selective memory.
Research has shown that preliminary hypotheses formed on the basis of incomplete data early on in a problem-solving process are retained in the face of additional, more complete data available later. If we structure the information that supports and rebuts our hypotheses, we can review both at regular intervals as we assess our progress.
People are often overconfident about the correctness of what they know when they solve problems. This problem can be mitigated as well by giving full consideration to the information recorded in a supporting or opposing structure.
When people review a planned course of action, they often do an incomplete job, not considering all the factors that are relevant to their plan, or not considering those factors in combination. The methods described in Chapter 11 are very good at showing the relationship between contributing conditions to a problem.
People often come to conclusions based on inadequate information. We must ensure that samples are representative in terms of time period and quantity.
Research has shown that problem solvers do a very poor job of dealing with variables that are partially related. Unfortunately, when people are asked to predict the future values of variables that are partially related to other variables, they tend to choose the extreme values, rather than those that tend toward the mean.
People tend to judge causality based on their perception of the similarity between a potential cause and its effect. People also tend to have trouble working with complex processes that occur over time and prefer to deal with a single moment. The cause-and-event-charting and fault-tree-analysis methods are helpful in avoiding theses type of errors.
A number of studies have shown that people don’t make good decisions in circumstances that require assessing probabilities. We don’t currently see an application of these knowledge-based errors to debugging, since the causes of software defects are deterministic.
Performing experimental research on programmers doing debugging is difficult. Much previous research has focused on programming languages and development techniques that are no longer in use. Almost all published research on debugging has used complete novices as subjects, which raises questions about its relevance to professional programmers.