Research Methodology

---

This research evaluates the effects that an improvement in processor speed has on individual computing productivity. Specifically, the effect of moving from a Pentium 133 to a Pentium 200 is evaluated. In line with the proposed framework, both the efficiency and effectiveness of end user computing can be evaluated. In the case of the proposed experiment, efficiency is evaluated as a direct impact by measuring the number of exam questions attempted and the number of correct questions completed in a specified time period. The effectiveness component is evaluated by measuring the accuracy of the exams. That is by determining the exam grade.

The experiment consisted of 149 currently enrolled MIS students taking two separate computer proficiency exams. All students had completed introductory computing courses and were familiar with the applications required to be used. The students who participated were college of business students and included all classifications (freshmen through seniors). Participation was mandated as part of course participation by several cooperating faculty. Students were randomly assigned to one of two adjoining labs as they arrived.

One lab contained 133 MHz processor computers and the other contained 200 MHz computers. The computers in both labs were identically equipped except for processor type. Software, operating system and application setup were identical on all machines. Hardware configurations were likewise identical, including RAM, Cache, hard drive and motherboards. No peripheral device requirements other than floppy disk access were required by the student.

The two exams used in the experiment were similar. Each exam had a total of 52 questions and consisted of basic operations to be performed in Windows Explorer, Microsoft Word and Microsoft Excel. The order and number of questions in each section were identical with differences existing only in the specifics of each instruction. For example, exam one asked the student to find the file star.doc and exam two asked the student to find the file circle.doc. The first exam was identical to all other first exams and likewise for the second exam. Each student was given 20 minutes to complete as much of each exam as possible.

Four different configurations for taking the two exams were performed and evaluated to capture and isolate the impact of the processor difference effects and the effects of short-term learning. These applications are labeled A, B, C and D.

As identified in Table 1, configuration A was made up of students who took both the first exam and the second exam on the same 133 MHz based systems. Configuration B students took the first exam on 133 MHz based systems and the second exam on 200 MHz based systems. Configuration C students took the first exam on 200 MHz based systems and the second exam on 133 MHz based systems. And, finally, configuration D students took the first and second exams on the 200 MHz based systems.

Table 1: Computer Assignment by Group

		Second Exam
		133 MHz	200 MHz
First Exam	133 MHz	A 44 students	B 34 students
	200 MHz	C 38 students	D 33 students

Students were instructed on the first exam that they complete it in any order they wished. On the second exam students were instructed to complete the exam in the same order as they completed the first. Once they completed the same questions as on the first exam, they were free to continue to work on additional questions.

By measuring the gains (if any) in the number of test questions attempted and/or completed for configurations A and D, where users performed both tests on the same machines, short-term learning effects could be isolated and measured. Short-term learning effects are those improvements in personal productivity associated with the repetition of similar work. In this experiment, both tests were similar resulting in similar if not identical application tasks to be performed. The time required for the student to complete the same task on the second test would necessarily take less time, as they were already familiar with the required operations to complete the similar task from the first exam.

Once the short term gains are isolated they can be subtracted from the configuration B students, those who took the first test on the 133 MHz machine and the second test on the 200MHz machine, identifying the amount of productivity improvement (if any) associated purely with the processor speed.

Configuration C permits the evaluation of any negative effects (decreases in productivity) that may be associated with moving to a lesser technology.

---