8.3 Research and Education

8.3 Research and Education

The government plays an important role in the software industry through the public funding of education and research, supplementing private investments in both arenas.

8.3.1 Workforce

The success of individual software suppliers and the ability of end-user organizations to successfully provision and use software crucially depend on an educated workforce that can capture user requirements, develop software, and provision and operate the infrastructure and applications (NRC 2001). Government and industry share responsibility in ensuring an adequately educated and adequately large workforce to maintain a healthy software industry.

The workforce in IT can be divided into two major categories: those who are primarily engaged in hands-on activities in the technology itself, such as specification, design, testing, and maintenance of software (the technology category; see section 4.2), and those who are primarily engaged in the uses and applications of technologies developed by others, such as conceptualization and analysis, provisioning, and operation (the user category; see section 5.1). The characteristics of these two categories are quite different.

Example In the United States it is estimated that about 5 million out of a total workforce of 140 million are in IT occupations, about equally divided between the two categories (NRC 2001). Estimates of the user category are the more difficult because there is no clear definition or distinction between these IT workers and the users of IT.

Two essential requirements of the workforce are education and training. Working in IT requires conceptual abilities, knowledge of theoretical IT constructs and frameworks, and applied technical skills. Conceptual and theoretical knowledge is usually acquired in a formal educational setting, whereas many skills can be acquired in vocational training programs or through on-the-job experience. Even the general population of IT users, which includes a large fraction of the population, requires some level of literacy or fluency in IT (NRC 1999a; 2002), a need generally unsatisfied in today's general education curricula in secondary schools and colleges.

The software industry has grown rapidly, and technologies change rapidly. Both growth and change raise concerns about the availability of appropriately educated workers. Many have argued that there is a shortage of appropriately educated workers, although this is difficult to substantiate. Clearly the labor market is tight, with salaries rising more quickly and with more job turnover compared to the general labor market. (The market for IT professionals is cyclical, however, and dropped off considerably in the recession of 2001-2002.)

Example In the United States the Current Population Study finds that salaries of IT workers in the technology category are rising between 3.8 and 4.5 percent annually (exclusive of noncash forms of compensation, such as stock options) as compared with 3.2 percent for workers as a whole (NRC 2001). This more rapid rise signals a general tightness in the labor market. This workforce has also expanded rapidly, growing by 60 to 75 percent in an eight-year period starting in 1991. This rapid growth suggests that many are entering the workforce through on-the-job experience rather than formal education, and thus raises questions about the qualifications of the workforce.

There are two ways to acquire an educated workforce in a particular country like the United States: educate your own populace or import workers educated elsewhere. Both mechanisms have been important to the United States, and government initiatives are crucial to both. Foreign educated workers have been a significant and growing source of labor in the United States.

Example In the United States the H-1B visa has allowed an increasing number of foreigners to work temporarily for up to six years. Foreign-born individuals represent about 17 percent of IT workers in the technology category (compared with about 10 percent of the U.S. population), and as much as 10 out of the 17 percent may be temporary workers under the H-1B visa program. Under considerable pressure from IT employers, but also taking into account the effect on opportunities and wages of American workers, the American Competitiveness and Workforce Improvement Act of 1998 increased the annual quota of H-1B visas from 65,000 to 115,000 per year, and established an IT technical training grant program using fees collected in issuing the visas. Some other developed countries, notably Germany, have discussed importing more foreign workers.

Of course, foreign workers can be utilized in place rather than brought into another country, and this is also a growing trend.

Example India has developed a multibillion dollar software industry, over half its revenues based on exports ("Software in India" 1996). A major enabler is substantial investment in education (India is also a major source of H-1B visa applicants in the United States), together with a large expatriate population working overseas that can shuttle business to India or return in managerial roles, and good English-speaking skills in the educated populace. Relatively little revenue is based on original software applications, in part because of the lack of a large domestic market for software (Balasubramanyam and Balasubramanyam 1997). Rather, the dominant activity is "body shopping," where well-specified modules are outsourced to Indian companies for implementation, and another revenue source is customizing software to specific end-user needs (Abraham, Ahlawat, and Ahlawat 1998).

With a rapidly improving infrastructure supporting international development efforts in software, a globalization of the software industry appears inevitable. That infrastructure includes not only networking but also increasingly sophisticated software collaboration and project management tools. As IT expands in many countries, they will develop an indigenous software industry to support local needs, and that will lead to export markets as well. Concurrent development of software—splitting project teams into different time zones so that the development can proceed twenty-four hours a day and reduce completion times accordingly—is also a major opportunity. A growing market for components should also assist the international market.

The characteristics of the IT workforce differ substantially from the general population in both racial and ethnic diversity, gender, and age distribution, and this is of considerable concern. White and Asian males are overrepresented, as are younger age groups, and clearly redressing this diversity would increase the supply of workers. This is an area where greater government involvement in training could be helpful.

8.3.2 Technology Transfer

Research in software is a good investment if it results in fundamental understanding that aids the education of current or future generations of students, or if it results in outcomes that are ultimately applied to commercial uses. Over the long term, investments in research as well as education are an important predictor of success in the software industry.

Technology transfer—moving research outcomes to commercial realization—is an integral part of research planning and investments. A book by Donald Stokes (1997) has galvanized a lot of thinking, both helping to explain some of the recent trends in industrial research and stimulating further changes. Before Stokes, thinking was dominated by the linear model (see figure 8.4). The development, provisioning, and operation phases were previously discussed in section 5.1. In the linear model, research is a separate activity that precedes development, one that is divided into basic and applied. Basic research embarks on the unknown, enlarging the realm of the possible, and applied research elaborates and applies the known in new ways. Thus, basic research focuses on the expansion of human knowledge, and applied research attempts to find new uses for existing knowledge. Development, on the other hand, directly adapts research findings into commercial products.

Figure 8.4: Two models of technology transfer.

Research policy in the United States has been dominated for five decades by the linear model as promulgated by Vannevar Bush (the government's science adviser in the 1940s). Bush asserted that "applied research invariably drives out pure" and therefore basic research must be completely isolated from considerations of use. The two-dimensional Stokes model, on the other hand, recognizes a third category of research in addition to pure (Bohr's quadrant) and applied (Edison's quadrant), which he calls Pasteur's quadrant. In this quadrant, considerations of use are a motivator, but the acquisition of new fundamental knowledge in pursuit of these new uses is also appreciated and encouraged.

Example It is an oversimplification to attribute any research purely to one quadrant, yet most research activities can be identified predominantly with one of Stokes's quadrants. The pursuit of mathematical models for computational complexity and fundamental limits to our ability to compute certain algorithms (major topics of computer science theory) have the flavor of Bohr's quadrant. As predicted by the linear model, such research finds useful application; for example, it has proven fundamental to the understanding of cryptography, which is based on hard computational problems. The UNIX operating system is arguably an outcome of Edison's quadrant research, since it was first prototyped by researchers at Bell Laboratories to serve as a platform for their other software research and not as a research objective in itself. An example of Pasteur's quadrant research is the seeking of stronger security protocols, which has also led directly to fundamental insights into protocols and algorithms (such as zero-knowledge proofs and protocol design and verification schemes).

The observation of the existence and importance of Pasteur's quadrant research leads to a revised dynamic model of technology transfer (see figure 8.5).

Figure 8.5: A dynamic model of technology transfer based on the Stokes (1997) two-dimensional model.

Example Investigations into the use of speech in user interfaces are an example of use-inspired basic research in software. While this research is inspired by the promise of more intuitive and natural interfaces that mimic they way people interact with one another, it also results in numerous basic insights into speech. Improved accuracy in speech recognition has led to better understanding of the physiology of speech production and new statistical techniques (so-called hidden Markov models). Automated understanding (not just recognition) of speech has led to many linguistic insights. Research into the interactions between humans and complex software systems has led to new insights in cognitive psychology.

While many top industrial research laboratories once believed in the linear model, today the trend is toward Pasteur's quadrant research, which has proven an effective way to enhance the returns on research investment to the sponsoring organizations. This increasingly leaves a gap in Bohr's quadrant research, a gap that can and should be filled with publicly funded research.

Today industrial researchers are usually encouraged to keep in close touch with product business units, and sometimes even forced to obtain a portion of their funding directly from these units. It is also common for researchers to switch from research to development organizations and back, an effective means of technology transfer and also an effective way to sensitize researchers to the opportunities and challenges of use.

Example IBM Research has been a leader in connecting its researchers directly with customers, with the objective of forming a more direct connection between research directions and market opportunities and challenges.^[7] Its First-of-a-Kind program creates direct partnerships between promising technologies and customers who may be able to benefit. It has created three Industry Solutions Laboratories in the United States, Europe, and Japan to display research outcomes to visiting customers and enhance the dialogue between customers and researchers. Another example is the IBM Institute for Advanced Commerce, which sponsors conferences and university partnerships to initiate new interdisciplinary research on the effect of IT on commerce.

While direct contributions to corporate success are a major motivation for funding research, there are other benefits as well. The top-notch scientists and engineers that a research laboratory can attract benefit a corporation in many ways, such as reviewing development projects, providing new product ideas, and providing advice on the direction of technology to top executives. Also, internal researchers will maintain contacts with the broader industrial and public research activities and bring them into the company.

8.3.3 Publicly Supported Research

While industrial research expenditures are large and growing, publicly supported research (largely in universities and government laboratories) plays an essential role in the software industry. This is not primarily a result of industrial research laboratories' emphasis on use-inspired research, because a great deal of publicly funded research on software is use-inspired as well. Rather, it is primarily a matter of time horizon, where industrially funded research emphasizes outcomes that can enhance revenues in the relatively short term, and publicly funded research can pursue long-term objectives.

Industry expenditures are generally reported for the aggregate of research and development, and it is important to note that most of these expenditures are directly in support of product development, not research (although the demarcation is fuzzy, particularly in Pasteur's quadrant).

Example Annual federal spending in the United States on research in computing was estimated at about $1 billion as of the mid-1990s, compared to industry research (not including development) expenditures of about $1 billion to $1.5 billion (NRC 1999a). Government funding has grown steadily over past decades, whereas industrial funding is quite volatile and in fact decreased dramatically during the 1990s' presumably because of an increasingly competitive industry environment. Government funded about 70 percent of total university research on computing.

Government support of research is crucial to the development of a highly educated work force in computing, directly supporting research assistantships for over half the graduate students at the best universities (NRC 1999b). Economists have also identified reasons why competitive markets and price mechanisms are not as effective in producing and distributing knowledge (a primary outcome of research) as they are for tangible goods, primarily because knowledge has many characteristics of public goods (NRC 1999b). Particularly important in the case of computing research is that the payoffs ascribed to major technology shifts (in contrast to ongoing incremental change) are uncertain and far in the future.

Example A study in 1995 examined the effect of major government-funded research in the 1960s over the intervening years (NRC 1995). In a number of cases, research led to $1 billion or greater commercial revenues, but the delay between the early research and major commercial application was in every case one to three decades. These cases included time-sharing (leading to minicomputers and today's client-server computing), computer graphics, the Internet, desktop workstations, graphical user interfaces, reduced instruction set computer architectures, computer-aided design of integrated circuits, parallel disk arrays, and parallel computing. A more recent study confirmed a continuation of this phenomenon, examples including relational databases, artificial intelligence, and virtual reality (NRC 1999b).

Historically, government-sponsored research has played an important role in the evolution of the computing industry. The long time horizon of commercial exploitation of many fundamental advances makes it difficult to justify these types of research by a return-on-investment analysis, even though intuition suggests (and history confirms) that some portion of such research eventually has major commercial payoffs. Patents are not fully effective in protecting research outcomes, and their two decades of protection is ineffective for these long-term benefits. With the rapid turnover of workers, and the disclosure of an invention's possibilities through the act of using it, trade secrets are difficult to preserve in the long term. Thus, long-time horizon research outcomes are relatively easily appropriated by those who did not contribute to the research (other companies or venture capitalists). Where effective, patents create a monopoly and trade secrets limit wide availability of knowledge, whereas publicly supported research can be widely disseminated and used and thus maximizes the benefit of research expenditures to society as a whole.

^[7]For further details on IBM Research's initiatives for working with customers, see <http://www.research.ibm.com/about/work_customers.shtml>. The IBM Institute for Advanced Commerce is described at <http://www.research.ibm.com/iac/>.