1.3 Measurement Across the Life-Cycle Model

Software measurement activities can and should occur throughout the software life cycle. Software requirements are very good leading indicators of the software development cost. Software design measurements will permit evaluation of design alternatives. Measuring the test activities will give great insight into the adequacy of the test process and the expected time to produce a software product that can meet certain quality standards.

We do not have unlimited resources to invest at every stage in the software life cycle. We are constrained by staff hours, hardware resources, and staff capabilities. A good measurement program will permit the optimal allocation of these limited resources to produce the best possible product within the timeframe and with the resources available.

Perhaps the most important contribution that can be made by an effective software measurement program is that a historical database of software development experience can be established. The best estimator of future software development costs is historical data. The best estimator of software reliability and quality is also data from previous software development projects.

1.3.1 Operational Requirements Specification

The requirements specification process is vital to the success of a software development organization. One of the principal products that emerges during this phase should be a cost estimate of the software development project. Unfortunately, it is very easy to underestimate the effort necessary to develop a new software product. That, in turn, means that it is very easy for software development organizations to lose their shirts in underbudgeted and underfunded development projects.

It is clear that the costs of a project will be directly related to the complexity of the project. This complexity clearly begins to emerge during the requirements engineering process. What is missing is a clear-cut method of effectively estimating project complexity from the requirements themselves. In most organizations, however, there is a wealth of historical data from previous projects. The nice thing about a finished product is that it is a done deal. At the conclusion of a project, most of the managers turn their attention to the new threat to their corporate future, the new XYZ project. And this is where the problem lies.

When a project is finished, it should be mined for its historical value. At the end of the project, we will know exactly how much it cost to develop. If we have kept good records, we know where the major cost centers were in the development and where the major concentrations of effort were. In particular, we would know how much time was devoted to design, to coding, to testing, to documentation, to training, etc. We would also have at our disposal a complete requirements document for the entire project. With the historical data from our most recent project and others that preceded it, we can learn to measure and quantify specific requirements attributes. ^[3] We can then use these measurements to develop predictive models for software costing. The essence of scientific software engineering is that it allows us to learn from the past and then use this information to create a better future.

In most software development organizations, the path from requirements to design to code seems to be a one-way street. The requirements process is merely a way to prime the code development pump. Once we begin to develop the actual code from a preliminary sketch of the requirements, the requirements are never revisited. We will learn that the software requirements specification process is vital to the understanding of the system that we are developing, as well as its maintainability.

If we are to build and maintain a requirements system, we must learn to partition this specification process into three distinct compartments. The first compartment is the user's view of the system; this we will call the operational specification. The interesting thing about software design and development is that there are no real constraints placed on us for the virtual environments that we may create for the user. We can take a basic accounts receivable system and build a nice, warm, cozy, user-friendly system, or we can build a very ornery, cold, user-hostile system. The operational requirements will articulate the operations that the user will perform to interact with the software. They deal only with what the system will do — not how it will be done.

One of the most important concepts that will emerge from the requirements specification process is the notion of requirements traceability. Each operational specification will map to specific code modules. Each code module can be traced to specific operations. We will be able to devise a system that will permit the requirements specification to be kept current with the evolving code base.

1.3.2 Design

Having a good clear idea about what the system we are going to build is to do for the customer, we can then set about to talk about how we are going to implement this system. This leads directly to the development of the high-level design of the system and a very precise description of the functionality of the system. Each user operation will be implemented by a very precisely defined set of system functionalities. Each functionality will be employed in one or more operations. There is a precisely defined mapping between the operational specifications and the functional specifications.

Knowing how we are going to build the system in the high-level design specification, we then have to decompose each functionality into one or module specifications. This constitutes the low-level design stage. Each module specification will detail completely how its piece of functionality will be achieved. It will contain the algorithm necessary to implement its task, and it will completely specify all the data to be used in this process. At this stage, we will be able to implement our first automated measurement systems. We will be able to measure and evaluate different design alternatives. We can understand the nature of the system that we are building before it is actually committed to source code.

During the design process, the overall structure of the program begins to emerge. Its basic functionalities are parsed into software modules. These software modules are then structured into a call graph that will detail the interactions among the modules. The data elements used within the modules are made explicit, as is the data being exchanged among modules on the module interface. In essence then, many salient features or attributes of the evolving program are made explicit during the design process.

The key feature of the design process will center on the notion of requirements traceability. The fundamental purpose of the software design process is to map user operational requirements to system functional requirements to software code elements. During the design process, the structure of the program with regard to control flow complexity and data complexity will clearly emerge. These program characteristics will be remarkably easy to identify and measure. If we are sufficiently careful during this phase, we will be able to model, with some precision, the operational performance characteristics of the system we are designing.

It is quite possible, indeed very easy, to design a system that cannot be tested or maintained. It is vital that we learn to identify those design characteristics that will lead to testable and maintainable software.

1.3.3 Coding

Coding is perhaps the easiest phase of the entire life cycle to measure and assess. For this reason, the literature is replete with measurement studies. We can watch programmers do their work and measure the time it takes to code. We can measure the products of their efforts in volume of code generated. We can measure the total number of problems (faults) they have created in their programming efforts. Unfortunately, as we will see, much of what we have learned about this process is, at best, misleading and, at its worst, dead wrong.

At the coding phase, the code has yet to have a life of its own. Our focus on this phase will initially be on the static properties of the code itself. We will learn to enumerate things about the code, such as the number of lines of source code, the number of statements, etc.

1.3.4 Test

Just as the coding process lends itself to measurement activities, so does the test activity. We can easily count the number of problems found when running the code. We can also measure the test effort in terms of human resources spent on this task.

During the test phase, our focus will shift to measuring aspects of the running code. In this sense we will learn to measure code dynamics, exactly what the code is doing when it is executing. Each test activity will have its own outcome. It will exercise a subset of program modules. Some tests will be good tests; they will expose potential problems in the code. Some tests will be bad; they will exercise code that has already been exercised or they will touch only those code modules not likely to contain faults.

We must learn to establish reasonable goals for the test process. With modern software systems, the number of different states that these systems can get into is impossibly large. We must accept the fact that we cannot find and remove all of the faults that a system might have. What we do care about is that the system we build will not fail when it is deployed. Different users will use our systems in different ways. Different uses of the software will, in turn, exercise different system features and consequently different code modules. Therefore, a vital part of the test process will be to determine how our customers will use the system. Once we have a good idea about how it will be used, we can then devote our test effort to ensuring that the system will perform reliably within this framework.

1.3.5 Maintenance

In the academic preparation of future computer scientists and software engineers, there is a tendency to ignore the maintenance phase altogether. In fact, it is very easy for these students to get the impression from their training that this is the information technology (IT) equivalent of janitorial service. Nothing could be further from the truth. For modern software projects, more than 80 percent of the total life-cycle costs will be allocated to this phase. It is also clear from this figure that the vast majority of jobs in this commercial sector will also be in software maintenance.

Because the focus of most IT development managers is strictly on the development side, there is little interest in what happens to a software product after it is placed into the customer's hands for the first time. Because software development technology is primarily based on the principle of craftsmanship and not on the basic principles of manufacturing technology, the vast majority of delivered software falls far short of any standards currently applied to manufactured goods. These software projects are fraught with flaws. They have been lovingly (or not so lovingly) crafted by a handful of developers who are the only ones who really understand the product functionality. When future problems arise with these systems, they require a phenomenal amount of human effort to fix. Central to this problem is the very simple notion of requirements traceability. We must learn to deduce the functionality of a software system by looking at the code. This is quite the wrong way to develop software. The structure and the functionality of the system should be specified outside the code. This, we will come to learn, is the essence of software maintainability. Software requirements traceability and maintainability are one and the same thing.

^[3]Munson, J.C. and Coulter, N.S., "An Investigation of Program Specification Complexity," Proceedings of the ACM Southeastern Regional Conference, April 1988, p. 590–595.