Methods for Evaluating IT Investment

 

Chapter 2 - Optimal Purchase Decision Criteria for Information Technology
Managing IT in Government, Business & Communities
by Gerry Gingrich (ed)
Idea Group Publishing 2003

	Brought to you by Team-Fly

Due to the importance and complexity of the decision, many methods have been used to evaluate IT investment alternatives. Some methods place emphasis on the cost-benefit analysis, using mainly return on investment, discounted cash flow, or added economic value as metrics. Typical examples include Expanded Net Present Value (Scarso, 1996) and Economic Value Added (EVA) (Mills, Rowbotham, and Robertson, 1998; Stewart, 1991). More recently, methods based on real option theory have been suggested to take into consideration important features of IT projects, such as flexibility and irreversibility of IT investments (Benaroch and Kauffman, 1999; Kim and Sanders, 2002; Li and Johnson, 2002).

Other methods address the limitations of human information-processing capability in making complex decisions. The underlying strategy is to "divide and conquer" the complexity of the problems - that is, reduce the complexity by dividing problems down to non-confounding characteristics so each characteristic can be tackled individually. This approach can be found in Fisher's separation theorem (Copeland and Weston, 1983; Harvey, 1997; Martin, Cox, and MacMinn, 1988) in Finance. It breaks down an investment decision to that of risk (individuals' subjective preferences) and return (objective market criterion). An important implication here, then, is that the objective part of investment decisions can be delegated to managers, and the investor's individual preferences can still be considered.

Several methods based on the "divide and conquer" strategy have also been used to solve the complex problem of the IT purchasing decision. Examples include the Qualitative Factor Analysis (Brown and Gibson, 1971) and the Analytical Hierarchy Process (AHP) (Saaty, 1994; Arbel and Seidmann, 1984). The AHP is particularly useful in multi-criteria decisions where it is difficult to compare alternatives against multiple criteria and to determine weights for different criteria. The AHP provides a systematic approach to assigning weights to criteria by algorithmically calculating scales resulting from pair-wise comparison, when the decision-maker has distinctive and consistent preference for different criteria of the decision. There are other weighted analysis methods that can be used that are more direct and easier to use by the general public. The one we will focus on in this paper is the Qualitative Factor Analysis.

Qualitative Factor Analysis (Factor Analysis for short) is used in Operations Management for comparing the desirability of multiple locations on an economic basis, particularly focusing on the relevant costs that vary from one location to another (Heizer and Render, 2001; Monks, 1977). Especially when there are no dominant or clear economic criteria available for quick decision, Factor Analysis injects values into a decision-making structure in a relatively formalized manner. Laudon and Laudon (2001) use a similar approach in making IT implementation decisions. Their scoring model gives alternatives a single score based on the extent to which the alternatives meet selected objectives. For example, a firm must decide among three office automation systems: (1) an IBM AS/400 client/server system with proprietary software, (2) a UNIX-based client server system using an Oracle database, and (3) a Windows NT client/server system using Lotus Notes. Using Laudon's scoring model approach, the development of a comparison for these three systems would include the following steps:

1. Determine the criteria to be applied to the problem. Establishing criteria that is agreeable to those responsible for decision making is often the most difficult aspect of this approach.

2. Decision makers should then determine the relative weight of each decision criterion.

3. Using a 1-to-5 scale (lowest to highest), determine the judgments of decision makers on the relative merits of each criterion for each alternative. For example, a score of 1 for the criterion "cost of the initial purchase" for the AS/400 system indicates that this system will be low in meeting that criterion when compared to the other systems being considered.

Using the above scoring model in Table 1 as an example, in this case the Windows NT option appears to be the preferred office automation system. It may be necessary to cycle through the scoring model several times, changing criteria and weights to determine how sensitive the outcome is to various changes in criteria (Laudon and Laudon, 2001).

Table 1: Laudon and Laudon's Example of System Selections

Criterion	Weight	AS/400		UNIX		Windows NT
		Scale	Score	Scale	Score	Scale	Score
Percentage of user needs met	0.40	2	0.8	3	1.2	4	1.6
Cost of the initial purchase	0.20	1	0.2	3	0.6	4	0.8
Financing	0.10	1	0.1	3	0.3	4	0.4
Ease of maintenance	0.10	2	0.2	3	0.3	4	0.4
Chances of success	0.20	3	0.6	4	0.8	4	0.8
Final score			1.9		3.2		4.0
Scale: 1 = low; 5 = high

This approach can also be applied to evaluate software solutions. Yerxa (1999) evaluated Netscape Enterprise Server 4.0, Microsoft Internet Information Server 4.0, and Apache Software's Apache Server for Network Computing. Using performance, development, configuration, management, platform support, and stability as decision criteria, his results are summarized in Table 2.

Table 2: Network Computing's Evaluation of Web Servers

Feature	Weight	Netscape Enterprise Server 4.0		Microsoft Internet Information Server 4.0		Apache Software Apache Server 1.3.9
Performance	0.30	4	1.20	5	1.50	3	0.90
Development	0.20	3.8	0.76	4.2	0.84	4.2	0.84
Configuration	0.15	5	0.75	4	0.60	4	0.60
Management	0.15	4	0.60	5	0.75	4	0.60
Platform support	0.10	5	0.50	2	0.20	5	0.50
Stability	0.10	4	0.40	2	0.20	5	0.50
Final score			4.21		4.09		3.94
Scale: 1 = low; 5 = high

Recognizing that different environments have their unique needs, Network Computing has developed a customized Java applet, called Interactive Report Card, on its Web site to allow readers to customize the results. For example, for some Web sites the reliability may be more important than performance, and with this change the conclusion obviously would be different. These managers then may assign higher weight to reliability instead.

While both models illustrated in this section are helpful to decision makers, they either ignore the cost or include the cost of the system as one of the factors to produce one single value. While one single value makes decision making easier, it masks the interactions of cost and benefit that can be better viewed in a two-dimensional chart. In the next section we will introduce a procedure to incorporate a modified factor rating methodology to evaluate PCs.

	Brought to you by Team-Fly