Large-Sample Empirical Evidence on IT, Organization, and Productivity


Large-Sample Empirical Evidence on IT, Organization, and Productivity

The case study literature offers many examples of strong links between IT and investments in complementary organizational practices. However, to reveal general trends and to quantify the overall impact, we must examine these effects across a wide range of firms and industries. In this section we explore the results from largesample statistical analyses. First, we examine studies on the direct relationship between IT investment and business value. We then consider studies that measured organizational factors and their correlation with IT use, as well as the few initial studies that have linked this relationship to productivity increases.

IT and Productivity

Much of the early research on the relationship between technology and productivity used economy-level or sector-level data and found little evidence of a relationship. For example, Roach (1987) found that while computer investment per white-collar worker in the service sector rose several hundred percent from 1977 to 1989, output per worker, as conventionally measured, did not increase discernibly. In several papers, Morrison and Berndt examined Bureau of Economic Analysis data for manufacturing industries at the two-digit SIC level and found that the gross marginal product of "high tech capital" (including computers) was less than its cost and that in many industries these supposedly labor-saving investments were associated with an increase in labor demand (Berndt and Morrison 1995, Morrison 1996). Robert Solow (1987) summarized this kind of pattern in his well-known remark: "[Y]ou can see the computer age everywhere except in the productivity statistics".

However, by the early 1990s, analyses at the firm level were beginning to find evidence that computers had a substantial effect on firms' productivity levels. Using data from more than 300 large firms over the period 1988–1992, Brynjolfsson and Hitt (1995, 1996) and Lichtenberg (1995) estimated production functions that use the firm's output (or value-added) as the dependent variable and use ordinary capital, IT capital, ordinary labor, IT labor, and a variety of dummy variables for time, industry, and firm.[2] The pattern of these relationships is summarized in figure 4.1, which compares firm-level IT investment with multifactor productivity (excluding computers) for the firms in the Brynjolfsson and Hitt (1995) dataset. There is a clear positive relationship, but also a great deal of individual variation in firms' success with IT.

click to expand
Figure 4.1: Productivity versus IT Stock (capital plus capitalized labor) for Large Firms (1988–1992) adjusted for industry

Estimates of the average annual contribution of computer capital to total output generally exceed $0.60 per dollar of capital stock, depending on the analysis and specification (Brynjolfsson and Hitt 1995, 1996; Lichtenberg 1995; Dewan and Min 1997). These estimates are statistically different from zero, and in most cases significantly exceed the expected rate of return of about $0.42 (the Jorgensonian rental price of computers—see Brynjolfsson and Hitt 2000). This suggests either abnormally high returns to investors or the existence of unmeasured costs or barriers to investment. Similarly, most estimates of the contribution of information systems labor to output exceed $1 (and are as high as $6) for every $1 of labor costs.

Several researchers have also examined the returns to IT using data on the use of various technologies rather than the size of the investment. Greenan and Mairesse (1996) matched data on French firms and workers to measure the relationship between a firm's productivity and the fraction of its employees who report using a personal computer at work. Their estimates of computers' contribution to output are consistent with earlier estimates of the computer's output elasticity.

Other micro-level studies have focused on the use of computerized manufacturing technologies. Kelley (1994) found that the most productive metal-working plants use computer-controlled machinery. Black and Lynch (1996) found that plants where a larger percentage of employees use computers are more productive in a sample containing multiple industries. Computerization has also been found to increase productivity in government activities both at the process level, such as package sorting at the post office or toll collection (Muhkopadhyay, Surendra, and Srinivasan 1997) and at higher levels of aggregation (Lehr and Lichtenberg 1998).

Taken collectively, these studies suggest that IT is associated with substantial increases in output. Questions remain about the mechanisms and direction of causality in these studies. Perhaps instead of IT causing greater output, "good firms" or average firms with unexpectedly high sales disproportionately spend their windfall on computers. For example, while Doms, Dunne, and Troske (1997) found that plants using more advanced manufacturing technologies had higher productivity and wages, they also found that this was commonly the case even before the technologies were introduced.

Efforts to disentangle causality have been limited by the lack of good instrumental variables for factor investment at the firm level. However, attempts to correct for this bias using available instrumental variables typically increase the estimated coefficients on IT even further (for example, Brynjolfsson and Hitt 1996, 2000).Thus, it appears that reverse causality is not driving the results: Firms with an unexpected increase in free cash flow invest in other factors, such as labor, before they change their spending on IT. Nonetheless, there appears to be a fair amount of causality in both directions—certain organizational characteristics make IT adoption more likely and vice versa.

The firm-level productivity studies can shed some light on the relationship between IT and organizational restructuring. For example, productivity studies consistently find that the output elasticities of computers exceed their (measured) input shares. One explanation for this finding is that the output elasticities for IT are about right, but the productivity studies are underestimating the input quantities because they neglect the role of unmeasured complementary investments. Dividing the output of the whole set of complements by only the factor share of IT will imply disproportionately high rates of return for IT.[3]

A variety of other evidence suggests that hidden assets play an important role in the relationship between IT and productivity. Brynjolfsson and Hitt (1995) estimated a firm fixed effects productivity model. This method can be interpreted as dividing firm-level IT benefits into two parts: One part is due to variation in firms' IT investments over time, the other to firm characteristics. Brynjolfsson and Hitt found that in the firm fixed effects model, the coefficient on IT was about 50 percent lower, compared to the results of an ordinary least squares regression, while the coefficients on the other factors, capital and labor, changed only slightly. This change suggests that unmeasured and slowly changing organizational practices (the "fixed effect") significantly affect the returns to IT investment.

Another indirect implication from the productivity studies comes from evidence that effects of IT are substantially larger when measured over longer time periods. Brynjolfsson and Hitt (2000) examined the effects of IT on productivity growth rather than productivity levels, which had been the emphasis in most previous work, using data that included more than 600 firms over the period 1987 to 1994. When one-year differences in IT are compared to one-year differences in firm productivity, the measured benefits of computers are approximately equal to their measured costs. However, the measured benefits rise by a factor of two to eight as longer time periods are considered, depending on the econometric specification used. One interpretation of these results is that short-term returns represent the direct effects of IT investment, while the longer-term returns represent the effects of IT when combined with related investments in organizational change. Further analysis, based on earlier results by Schankermann (1981) in the R&D context, suggested that these omitted factors were not simply IT investments that were erroneously misclassified as capital or labor. Instead, to be consistent with the econometric results, the omitted factors had to have been accumulated in ways that would not appear on the current balance sheet. Firm-specific human capital and "organizational capital" are two examples of omitted inputs that would fit this description.[4]

A final perspective on the value of these organizational complements to IT can be found using financial market data, drawing on the literature on Tobin's q. This approach measures the rate of return of an asset indirectly, based on comparing the stock market value of the firm to the replacement value of the various capital assets it owns. Typically, Tobin's q has been employed to measure the relative value of observable assets such as R&D or physical plant. However, as suggested by Hall (1999, 1999b),Tobin's q can also be viewed as providing a measure of the total quantity of capital, including the value of "technology, organization, business practices, and other produced elements of successful modern corporation". Using an approach along these lines, Brynjolfsson and Yang (1997) found that while $1 of ordinary capital is valued at approximately $1 by the financial markets, $1 of IT capital appears to be correlated with between $5 and $20 of additional stock market value for Fortune 1000 firms using data spanning 1987 to 1994. Since these results largely apply to large, established firms rather than new high-tech startups, and since they predate most of the massive increase in market valuations for technology stocks in the late 1990s, these results are not likely to be sensitive to the possibility of a recent "high-tech stock bubble".

A more likely explanation for these results is that IT capital is disproportionately associated with other intangible assets like the costs of developing new software, populating a database, implementing a new business process, acquiring a more highly skilled staff, or undergoing a major organizational transformation, all of which go uncounted on a firm's balance sheet. In this interpretation, for every dollar of IT capital, the typical firm has also accumulated between $4 and $19 in additional intangible assets. A related explanation is that firms must occur substantial "adjustment costs" before IT is effective. These adjustment costs drive a wedge between the value of a computer resting on the loading dock and one that is fully integrated into the organization.

The evidence from the productivity and the Tobin's q analyses provides some insights into the properties of IT-related intangible assets, even if we cannot measure these assets directly. Such assets are large, potentially several multiples of the measured IT investment. They are unmeasured in the sense that they do not appear as a capital asset or as other components of firm input, although they do appear to be unique characteristics of particular firms as opposed to industry effects. Finally, they have more effect in the long term than the short term, suggesting that multiple years of adaptation and investment are required before their influence is maximized.

Direct Measurement of the Interrelationship between IT and Organization

Some studies have attempted to measure organizational complements directly, and to show either that they are correlated with IT investment, or that firms that combine complementary factors have better economic performance. Finding correlations between IT and organizational change, or between these factors and measures of economic performance, is not sufficient to prove that these practices are complements, unless a full structural model specifies the production relationships and demand drivers for each factor. Athey and Stern (1997) discuss issues in the empirical assessment of complementarity relationships. However, after empirically evaluating possible alternative explanations and combining correlations with performance analyses, complementarities are often the most plausible explanation for observed relationships between IT, organizational factors, and economic performance.

The first set of studies in this area focuses on correlations between use of IT and extent of organizational change. An important finding is that IT investment is greater in organizations that are decentralized and have a greater level of demand for human capital. For example, Bresnahan, Brynjolfsson, and Hitt (2000) surveyed approximately 400 large firms to obtain information on aspects of organizational structure like allocation of decision rights, workforce composition, and investments in human capital. They found that greater levels of IT are associated with increased delegation of authority to individuals and teams, greater levels of skill and education in the workforce, and greater emphasis on pre-employment screening for education and training. In addition, they find that these work practices are correlated with each other, suggesting that they are part of a complementary work system.[5]

Research on jobs within specific industries has begun to explore the mechanisms within organizations that create these complementarities. Drawing on a case study on the automobile repair industry, Levy, Beamish, Murnane, and Autor (2000) argue that computers are most likely to substitute for jobs that rely on rule-based decision making while complementing non-procedural cognitive tasks. In banking, researchers have found that many of the skill, wage, and other organizational effects of computers depend on the extent to which firms couple computer investment with organizational redesign and other managerial decisions (Hunter, Bernhardt, Hughes, and Skuratowitz 2000; Murnane, Levy, and Autor 1999). Researchers focusing at the establishment level have also found complementarities between existing technology infrastructure and firm work practices to be a key determinant of the firm's ability to incorporate new technologies (Bresnahan and Greenstein 1997); this also suggests a pattern of mutual causation between computer investment and organization.

A variety of industry-level studies also shows a strong connection between investment in high technology equipment and the demand for skilled, educated workers (Berndt, Morrison, and Rosenblum 1992; Berman, Bound, and Griliches 1994; Autor, Katz, and Krueger 1998). Again, these findings are consistent with the idea that increasing use of computers is associated with a greater demand for human capital.

Several researchers have also considered the effect of IT on macro-organizational structures. They have typically found that greater levels of investment in IT are associated with smaller firms and less vertical integration. Brynjolfsson, Malone, Gurbaxani, and Kambil (1994) found that increases in the level of IT capital in an economic sector were associated with a decline in average firm size in that sector, consistent with IT leading to a reduction in vertical integration. Hitt (1999), examining the relationship between a firm's IT capital stock and direct measures of its vertical integration, arrived at similar conclusions. These results corroborate earlier case analyses and theoretical arguments that suggested that IT would be associated with a decrease in vertical integration because it lowers the costs of coordinating externally with suppliers (Malone, Yates, and Benjamin 1987; Gurbaxani and Whang 1991; Clemons and Row 1992).

One difficulty in interpreting the literature on correlations between IT and organizational change is that some managers may be predisposed to try every new idea and some managers may be averse to trying anything new at all. In such a world, IT and a "modern" work organization might be correlated in firms because of the temperament of management, not because they are economic complements. To rule out this sort of spurious correlation, it is useful to bring measures of productivity and economic performance into the analysis. If combining IT and organizational restructuring is economically justified, then firms that adopt these practices as a system should outperform those that fail to combine IT investment with appropriate organizational structures.

In fact, firms that adopt decentralized organizational structures and work structures do appear to have a higher contribution of IT to productivity (Bresnahan, Brynjolfsson, and Hitt 2000). For example, for firms that are more decentralized than the median firm (as measured by individual organizational practices and by an index of such practices), have, on average, a 13 percent greater IT elasticity and a 10 percent greater investment in IT than the median firm. Firms that are in the top half of both IT investment and decentralization are on average 5 percent more productive than firms that are above average only in IT investment or only in decentralization.

Similar results also appear when economic performance is measured as stock market valuation. Firms in the top third of decentralization have a 6 percent higher market value after controlling for all other measured assets; this is consistent with the theory that organizational decentralization behaves like an intangible asset. Moreover, the stock market value of a dollar of IT capital is between $2 and $5 greater in decentralized firms than in centralized firms (per standard deviation of the decentralization measure), and this relationship is particularly striking for firms that are simultaneously extensive users of IT and highly decentralized as shown in figure 4.2 (Brynjolfsson, Hitt, and Yang 2000).

click to expand
Figure 4.2: Market Value as a function of IT and Work Organization This graph was produced by non-parametric local regression models using data from Brynjolfsson, Hitt, and Yang (2000). Note: I represents computer capital, org represents a measure of decentralization and mv is market value.

The weight of the firm-level evidence shows that a combination of investment in technology and changes in organizations and work practices facilitated by these technologies contributes to firms' productivity growth and market value. However, much work remains to be done in categorizing and measuring the relevant changes in organizations and work practices, and relating them to IT and productivity.

[2]These studies assumed a standard form (Cobb-Douglas) for the production function, and measured the variables in logarithms. In general, using different functional forms, such as the transcendental logarithmic (translog) production function, has little effect on the measurement of output elasticities.

[3]Hitt (1996) and Brynjolfsson and Hitt (2000) present a formal analysis of this issue.

[4]Part of the difference in coefficients between short and long difference specifications could also be explained by measurement error (which tends to average out somewhat over longer time periods). Such errors-in-variables can bias down coefficients based on short differences, but the size of the change is too large to be attributed solely to this effect (Brynjolfsson and Hitt 2000).

[5]Kelley (1994) found that the use of programmable manufacturing equipment is correlated with several aspects of human resource practices.