Recording and Analysis of the Results

A significant amount of data was gathered during the usability testing, but most of it is not relevant to the current study and has been reported elsewhere. The data pertinent to the current study were the various strategies used by participants to complete each task.

Strategies Used

A strategy was defined as a sequence of actions that a participant used to produce an answer to task. This definition has three implications:

• Sequences of actions that only accomplished part of a task are not regarded as strategies.

• A sequence of actions that produced an answer is a strategy, even if the answer was wrong.

• Actions carried out in error or recovering from an error were part of a strategy.

Some participants carried out tasks in more than one way, i.e., using more than one strategy. In many cases, a participant would employ some strategy, realize the weakness of the strategy and develop another more successful strategy that resulted in the participant recording an answer. It was decided to record only that strategy which actually gave rise to the participant's answer. This became more of an issue when some participants returned to a task at a later stage, used a different strategy to the original one and altered their original answer. The definition of a strategy was modified to include only that set of actions that led to the participant recording his or her final answer. This definition was found to be quite effective and resolved most ambiguities.