6.9 Summary

As programs have increased in length several orders of magnitude in the past three decades, the problems associated with measuring these programs have also increased several orders of magnitude. Furthermore, these systems experience a very large number of changes during their development and early deployment. Not all changes will have the same relative impact on the code in terms of its overall complexity. Some changes, by their very nature, will be innocuous, such as the introduction of comment statements. Other changes will make substantial changes to the basic architecture of a program module. Even the simplest measurements taken on each program module have a way of creating an enormous data management problem in the measurement of evolving systems. The key to the success of the measurement problem is to reduce the size of the problem with which we are working.

Some effort has been devoted to organizing the numerous metrics into various taxonomic categories in an attempt to understand the nature of the underlying complexity domains. These taxonomic structures, however, do not reflect the actual variation of the metrics when they are potentially applied to the development of measurement models. The technique we have developed is based on the variability of each metric in conjunction with other metrics in a set to show the structure of the complexity domains and ultimately represent this complexity with a single numerical value called the fault index (FI) using a statistical procedure known as principal component analysis (PCA).

The potential success of using complexity metrics is predicated on the ability of these metrics to describe all aspects of program variability. Problems have arisen in the past in the application of complexity metrics to predictive models for software development in that different program modules would have substantially different values of these metrics. In response to the need to compare programs one to another based on their complexity, we have developed a realistic methodology to determine the FI that will reflect the contribution of each program module to the total complexity of a software system. As a methodology, any relevant set of complexity metrics can be used to derive the FI measure. Typically, a working set of metrics would include metrics representing each of the complexity domains that we have determined to be directly related to our criterion measure - software faults.

The HALMET metric tool for the HAL/S language has been refined through three major revisions to generate a working set of software complexity metrics that serve to explain approximately 85 percent of the variation in the software faults in each program module as measured by Discrepancy Reports filed against that module. The driving force in the construction of this measurement tool is to construct suitable surrogate measures for software faults. These surrogate measures will vary in direct relation to the embedded software faults. While we would like to be able to measure software faults directly, these are known only to Nature. It is possible, however, to construct software measures that will vary directly with software faults. Orthogonal domain metrics and a single measure, FI, will be constructed to serve as such surrogates. If a change has been made to a system that results in a major increase in our fault surrogate, FI, then we will come to understand that this change will likely introduce a number of new faults proportional to the change.

There are many different aspects of a program that can be measured. Each of these attributes represents a different aspect of program complexity. Programs differ in the number of statements they contain. They differ in the control complexity arena; and they also differ in the data structures they use. There are literally hundreds of different complexity attributes that can be measured. From a software maintenance perspective, not all of these are of equal interest to us. The metrics we will choose to use will be selected because of their demonstrated relationship to faults.