Chapter 14: Implementing a Software Research Plan

  • The Law of the Hammer: Give a child a hammer and everything in his or her environment becomes a nail.

  • The Law of C++: Give a programmer access to C++ and everything in his or her environment becomes a C++ application.

14.1 What is Software Research?

Much of what we know about software development is purely conjectural. The literature abounds with new software processes that promise miracles, new software development methodologies that border on the extreme, or new language metaphors whose object is unclear. There is little or no empirical research to support any of these new approaches. That, in and of itself, is not bad. What is most depressing is that no one appears to be seeking validation of these claims. The validation process has at its foundation empirical research.

In a typical modern hardware development organization, approximately 10 percent of the development budget is typically devoted to basic research. This is research that is not designed to produce an immediate product. An example of such a research facility was the long-standing example of AT&T Bell Laboratories. There were many people employed within this organization that were doing research on theoretical areas that were far removed from the mechanics of a production line. The intellectual property that resulted from this investment was truly astonishing. It produced, among many other things, the transistor that, in turn, fundamentally changed the world in which we live.

Curiously, it is difficult, if not impossible, to point to a similar organization devoted to basic research in the area of software engineering (computer science, informatics, etc.). It is even stranger, in that a lion's share of development cost of any new product with embedded software is the software itself. Most organizations have the attitude that software is just something that you add in when the product has been built. It is odd to hear from development organizations that are using embedded software systems that the cost of software development has now exceeded that of hardware development. These same organizations are not listening to their own message. They continue to devote substantial resources to hardware research and zero resources to software research.

We start our software research program on the premise that little or nothing is known about software development methodology. We are on a quest to create the foundation for this knowledge within our own organization. We can learn little from the work that others have done in the area because of the lack of measurement standards. A group might report, for example, that a typical developer in their organization could produce 20 C statements per day. We learn nothing from this data because we simply do not know what a statement means to the people reporting the result. The 20 statements might well include an average of 10 comment statements, for example. If we were to use these data to insist that our developers produce 20 executable statements per day based on the results of this report, then we would be seeking to achieve a level of productivity twice that of the reporting organization.

The best way to kill an incipient software research plan is to form a committee to study the process. If there is to be a software research effort, it will have to have its impetus at the top of the corporate food chain. There must be commitment at the highest level in the company. Software research should be a line item in the operating budget of every department. Audit systems should be put in place to ensure that the money budgeted actually winds up in the hands of the people doing the research.

14.1.1 How We Got This Way

It is clear that there has been very little empirical research in what we call computer science. In fact, computer science is really a very odd term. There has been really little empirical science in computer science. We have much theory; that is because most of the early founders in the field came from mathematics. In fact, most computer science programs had their origins in departments of mathematics. A typical mathematician has had little or no training in the scientific method. There is no reason to have this preparation. The concept of empirical validation of theoretical constructs is about as far from a mathematician's thinking process as you can get. It is completely understandable, then, that there has been little impetus in computer science curricula to teach students about scientific methodology. The first serious chemistry course that a chemistry major will take is called quantitative methods. In this course, students learn how to measure things for laboratory research. Psychology students are exposed early in their academic careers to a course in the design of experiments. A careful analysis of computer science curricula throughout the United States will not reveal a single course in either measurement or the design of experiments. It would be unrealistic, therefore, to presume that people with computer science backgrounds could begin to formulate an empirically based research program. They will have received no training in the conduct of scientific inquiry, nor will they have the statistical preparation to assist them in this regard.

There is an emerging discipline of software engineering. It is now relatively common to see software engineering curricula at the undergraduate and graduate levels in universities throughout the world. Unfortunately, students in these curricula are not exposed to the notions of the basic engineering discipline. At the beginning of every term at every university that supports a mechanical engineering program, you will see prospective engineers out around campus with their survey instruments, trying to find the locations of various points of interest throughout the university. This is a most important course. Students learn how to measure and they also learn a great deal about accuracy and precision. This knowledge is vital to their success as mechanical engineers. Measurement and the discipline associated with measurement are fundamental principles of the engineering discipline. However, there are no similar measurement courses for software engineering undergraduates. Whatever the discipline of software engineering might be, it is not an engineering discipline.

For the moment, then, the software development industry will have to play a leading role in empirical software research. In essence, these organizations will have to develop their own training programs to teach computer scientists to do science and software engineers to do engineering. Some software development organizations have gone so far as to recruit only mechanical, computer, or electrical engineers for their software engineering staff. It is far more cost effective to teach a mechanical engineer the essentials of software development than it is to try to teach a software engineer about an engineering discipline. A typical mechanical engineer will have four to five years of training in what it means to be an engineer. A typical new software engineer will have had no training in any engineering discipline.

14.1.2 Empirical Research

Empirical research is a scientific means of investigating the validity of theoretical research. It is also the driving force behind the discipline of engineering. In the field of mathematics, there is only theory. The notion that theory might in some sense have practical value is preposterous. Theory exists for theory's sake. Practical relevance was far from White-head and Russell's thinking when they did the Principia Mathematica. Mathematicians, and consequently computer scientists, are theoreticians and that is that. It is not reasonable, therefore, to expect that the typical computer scientist will have the necessary training or interest to do empirical research.

A real case could be made that considerable investment has been made in empirical research programs in software, particularly in the aerospace industry. There is a great difference, however, between empirical research in an application domain and empirical research on software development. The current work in speech recognition and prosody is a very good example of empirical research in an application area. In this context, there has been a very substantial research investment on the part of quite a number of companies. There is great interest in speech processing, voice recognition, and speaker intonations. A considerable amount of research money is now pouring into this area. Unfortunately, these research dollars are being invested in the application domain, and not in the software domain. There is very little interest in the nature of software development to do speech recognition. Very soon we will have speech recognition technology that will be able to recognize the speaker and make an assessment of his mental attitude. We will have little information, however, about how to develop software for this application. It will probably be written in the universal language, C++.

The defining element of empirical software research is the controlled experiment. In the controlled experiment environment, potential sources of variation are first identified. Some of these sources of variation will be controlled so that they do not impact the sources of variation that will be manipulated by the experiment. If, for example, we wanted to compare programming language A with programming language B for use in an application C, begin by identifying one or more criterion variables such as complexity, maintainability, or reliability that will serve as the basis for comparison. We would then seek to identify sources of variation not related to the criterion variable(s) that would influence the outcome of the experiment. One such source of variation might be programmer experience with language B. We might have a development staff that is completely familiar with language B but has little or no familiarity with language A. In this case, we would expect that language B will turn out to be the language of choice, not because it is intrinsically better for application C, but because our programming staff does not really have the same degree of expertise with language A that they have with language B.

The focus of this experiment is on the language comparison, not on the application. We seem to be quite willing to invest research money in solving application-specific problems but not on software-specific questions. Most of the programming languages in common use today were simply hacked out by a small group of individuals. They are then deployed in safety-critical or high-availability applications. It is not possible to produce reliable code with a language system that is, in and of itself, unreliable.



Software Engineering Measurement
Software Engineering Measurement
ISBN: 0849315034
EAN: 2147483647
Year: 2003
Pages: 139

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net