Literature Review

---

Cognitive Skills

Figure 1 illustrates a framework that suggests spreadsheet training influences cognitive skills, namely logical reasoning, spatial visualization, mnemonic skill, and sequencing skill of end users that will subsequently influence the error rate in spreadsheet models developed by the end users. Cognitive skills are related to how individuals acquire, store, retrieve, and utilize knowledge. Different types of cognitive skills are necessary to complete different tasks. When end users develop spreadsheet models, they are highly engaged in problem solving, planning, and perceptual-motor functions. The demand on their working memory is high, as they have to remember the unit task structure and other task information that puts a high load on their memory. While the end user generally will not be putting much in long-term memory, he or she needs to retrieve information from long-term memory to design a spreadsheet model for a problem domain. Finally, accuracy is very important because a small error in the spreadsheet can have a chain effect leading to dramatically misleading numbers. Table 1 provides examples of required mental processes and memory capacities involved in various tasks to compare them with the above attributes with spreadsheet model development, and were based on the "reasoned arguments" of Card, Moran, & Newell (1983) and Olson & Nilsen (1987). Table 1 illustrates that to complete typical everyday tasks requires various levels of alternative cognitive skills.

Table 1: Mental Processes and Memory Capacities Involved in Specific Tasks

	SKILL CHARACTER			WORKING MEMORY		LONG-TERM MEMORY		TASK DEMANDS
Tasks	Problem Solving	Perceptual -motor	Planning	Unit task structure	Memory load	Input to LTM	Retrieval from LTM	Pacing	Accuracy
Typing	Low	High	Low	Low	Low	Low	Low	Low	Int.
Driving car	Low	High	Int.	Low	Int.	Low	Low	High	High
Mental multiplication	Int.	Low	Low	High	High	Low	Int.	Low	High
Balancing checkbook	High	Low	Int.	High	High	Int.	Int.	Low	High
Writing business letter	High	Low	High	High	Int.	Int.	Int.	Low	Int.
CPA doing income tax	High	Low	High	High	Int.	Int.	High	Low	High
Routine medical diagnosis	High	Low	High	High	High	High	High	Int.	High
Spreadsheet task	High	High	High	High	High	Low	High	Low	High
Card et al. 1983; Olson and Nilsen 1987-1988

Figure 1: A Framework for Cognitive Skills, Spreadsheet Training, and Errors

Training can influence various cognitive skills (e.g., Burnett & Lane, 1980; Galotti, Baron & Savini, 1986; Kliegl, Smith, & Baltes, 1989; Baltes & Kliegl, 1992; Sein, Olfman & Davis, 1993; Wenger & Payne, 1995; Wenger & Carlson, 1996; Johnson & Lawson 1998; Mackenzie, 1999). The first cognitive skill proposed in Figure 1, logical reasoning ability, increases with training. Johnson & Lawson (1998) conducted an empirical study placing students into two groups: "expository" learning approach or "inquiry" learning approach. Logical reasoning ability explained significantly more variance in final examination scores for students that were provided with "expository" classes (18.8%) for the entire semester compared to those that were provided "inquiry" (7.2%) classes for the full semester.

A second cognitive skill that has been shown by established research to increase with specific training is spatial visualization. Spatial visualization is defined as the ability to manipulate or transform the image of spatial patterns into other arrangements (Ekstrom, French & Harman, 1976). Individuals with strong spatial visualization ability can maintain multiple representations of objects and systems for manipulation (Pellegrino, 1985) and can outperform those with low spatial visualization ability for complex and creative tasks (Sein et al., 1993; Hutchins, Hollan & Norman, 1985). An experimental study using a popular electronic spreadsheet package, Lotus 1-2-3©, found significantly better results for both comprehension score and task performance for those participants with high spatial visualization ability (Sein et al., 1993).

The third type of cognitive skill in Figure 1 is mnemonic skill. Mnemonic skill is the ability or strategy to encode and organize knowledge as one learns so that it can be more easily retrieved later. Individuals can be trained to increase their mnemonic skill (Kliegl et al., 1989; Baltes & Kliegl, 1992). In one study, subjects participated in 20 sessions (practice and training) and were trained to approximately double their mnemonic skill, as measured by the number of words recalled in serial order (Kliegl et al., 1989). Follow up studies (Baltes & Kliegl, 1992), which extended previous research by allowing participants to backtrack when performance fell below a specified criterion, showed similar results.

The fourth type of cognitive skill, sequencing ability, refers to the ability to put in the correct sequential order a number of individual operations that solve a problem. Wenger & Carlson (1996) conducted a series of four different experiments using specific arithmetic tasks in conjunction with sequencing of up to 12 steps. Some parts of the experiment were also designed with a sub-goal structure, which increased both the speed and the accuracy of the results. The researchers determined that these tasks did result in an increase in the efficient use of working memory.

While previous studies have examined some of the cognitive skills in the framework proposed in this study, many of the previous studies have not been longitudinal. Longitudinal studies can be very insightful. For example, to determine the effects of learning programming skills takes about a semester to result in a change in cognitive skills (VanLehan, 1996). Our current study attempts to add to the extant literature by determining which, if any, of several specific cognitive skills described in the framework in Figure 1 are increased during a semester course on end-user computing.

Using subjects from Survey of Accounting and Personal Computers in Business classes, we test the framework proposed in Figure 1. We then try to identify the specific cognitive skills that are influenced by spreadsheet training. This should help end users focus on those cognitive skills and help develop a more efficient and effective training program. More efficient training programs can be developed by focusing on the cognitive skills that will result in reduced errors in spreadsheet models.

Spreadsheet Errors

Spreadsheet models developed by end users can contain high error rates (Brown & Gould, 1987; Davies & Ikin, 1987; Cragg & King, 1993; Janvrin & Morrison, 1996; Panko & Halverson, 1996; Panko & Havlerson, 1997; Panko & Sprague, 1998). Spreadsheets are relied on for many business applications and undiscovered errors can be a serious problem due to the potential for a magnifying effect. In one of the most highly publicized incidents that occurred in a Fort Lauderdale construction company's bid for a job, the simple (and common) act of inserting a row into a working spreadsheet resulted in a large loss for the company on a $3 million job. The controller of Cummings Incorporated (the construction company) inserted a row to include additional overhead of $254,000 but failed to check whether or not this row was included in the formula that totaled the column. Cummings did win the bid but severely underestimated the cost of the project resulting in large financial losses to the firm (Gilman & Bulkeley, 1984; Ditlea, 1987; Simkin, 1987; Kee, 1988; Hayden & Peters, 1989; Stone & Black, 1989).

Table 2 provides a brief summary of representative studies that document unacceptably high error rates in spreadsheets for professionals in real-world applications, as well as for students in experimental studies (see Kruck & Maher, 1998, for a discussion of spreadsheet errors and proper design procedures). Most spreadsheet model research has primarily concentrated on documenting the existence of high error rates in completed models developed by end users in various situations. Our current research program begins to examine how cognitive skills are altered or modified after participants receive training in proper spreadsheet design methods.

Table 2: Summary of Studies of Spreadsheet Errors

Author(s)	Year	Participants	% of Spreadsheets with Errors
Brown & Gould	1987	IBM employees	44%
Davis & Ikin	1987	Live/real company spreadsheet: major errors inadequate & extremely error prone	21% 53%
Hassinen	1988	Novice students: working on computer working with paper & pencil	48% 55%
Cragg & King	1993	Live/real company spreadsheets	25%
Panko & Halverson Jr.	1994	Business students: working alone working in groups of two working in groups of four	81% 71% 50%
Panko & Halverson Jr.	1995	Accouting students: General business students working alone General business students working in groups of three	68% 82% 27%
Janvrin & Morrison	1996	Upper & master's level accounting & business administration students: ad hoc development group structured systems group	14%[*] 7%[*]
Janvrin & Morrison	1996	Upper & master's level accounting & business administration students: ad hoc development group structured systems group	18%[**] 9%[*]
Panko	1996	MIS upper-division undergrads: working at home working in lab	38% 30%
Panko & Halverson	1996	MBA students Non-accounting & finance upper-division undergrads	57% 79%
Panko & Halverson Jr.	1997	Business students: working alone working in groups of two working in groups of four Accounting & finance students	79% 78% 64% 65%
Panko & Sprague Jr.	1997	Undergraduate students Inexperienced MBA students Experienced MBA students	37% 35% 24%
[*]paper template of solution provided [**]check figure provided

---