6.3 Sources of Variation

6.3 Sources of Variation

As we measure software attributes, we can characterize in two different ways the number that we get from each program module that we measure. First, we can look at measures of central tendency such as mean, median, or mode. Unfortunately, there is very little information in these statistics. We could know, for example, that the average Exec for a program module is 1000 executable statements. If we were to know this statistic and the fact that module A had 1010 executable statements in it, we will have to learn little about module A in relation to the rest of the program modules. This information is contained in the measures of dispersion, typically the second moment about the mean or variance. To understand the relationship between module A and all the other program modules, we will need information about the variation among the program modules. If the standard deviation is found to be 5, for example, then module A is fully two standard deviations from the mean. It is large in relation to the other program modules. If, on the other hand, the standard deviation is found to be 20, then we learn that module A is relatively close to the average of all program modules in executable statements. If the variance were 0, then all program modules would have 1000 statements. There would be no information in the knowledge that a module was comprised of 1000 statements. The main point here is that the information content on the measure is dependent on its variance.

There are many possible sources of the observed variance in program statements. First, there is the intrinsic variation. Different program modules simply have different numbers of statements in them. Some modules are large and others are small, depending on the module functionality. Second, there are systematic sources of variation that are not intrinsically related to the size of the program module. In this category, we have errors of measurement. If we have people counting statements, they can simply miscount statements. This will induce a source of variation that is related to errors in measurement. There can also be systematic variance induced by differences in people. If there are ambiguities in the standards for defining statements, different people will enumerate statements differently. There will be a source of variance, then, directly related to constant differences among the enumerators. These are summarized in the Venn diagram of Exhibit 1.

Exhibit 1: Sources of Variation

If we are measuring both the executable statements and lines of code of a program module, these metrics clearly have a lot in common. They are both measures of the size of a program. As such, they share an element of common variation, as is shown in Exhibit 2. We call this element of shared variance the covariance, whereas, the variance term is derived from the second moment about the mean as follows:

Exhibit 2: Common Variation

The covariance term is derived from the cross-product of two variables and their simultaneous variation as follows:

If x_i and y_i differ from their respective means in the same way for the same program module, then their element of shared variation is very large. If, on the other hand, knowledge of x does not yield much information about the corresponding value of y for the same module, then their shared variation is very small.

Each attribute that we are measuring has its own mean and variance. It is very difficult to impossible to learn anything about a program module of 75 lines of code whose statement count is 53 by looking at these raw values. If we adjust each for the effect of its own mean and standard deviation, then we can look at these adjusted values and know something about the size of the program module in question. To do this we will convert the raw metric scores to z-scores as follows: . Now each of the new z-scores will have a mean of 0 and a standard deviation of 1. Now, if we know that the average lines of code for a module is 85 with a standard deviation of 10 and that the average statement count is 62 with a standard deviation of 7, we can convert the raw scores of 75 LOC and 53 Exec to their corresponding z-scores as follows:

Z_LOC = (75 - 85) / 10 = -1.0

Z_Exec = (53 - 62 / 7 = -1.3.

It is now apparent that the program module in question is significantly smaller in both lines of code and executable statements than other program modules. That the z-scores are negative indicates that the values are less than the mean. That they are less than or equal to -1.0 indicates that they are at least one standard deviation below the average module for the system. In that they now have the same underlying distribution, it would make statistical sense to add them together or find their average.

Now that we have adjusted for the effect of the mean and the standard deviation in the creation of z-scores for the lines of code and executable statement metrics, we can now talk about their common variance in terms of the Pearson product moment correlation statistic as follows:

This new statistic is a measure of the common variation between the variables x and y. It is based on the standardized z-scores. This means that -1.0 ≤ r_xy ≤ 1.0. This new statistic is a measure of the bivariate relationship between two variables x and y.

It would be useful, at this point, to examine some real data with this newly found understanding of variation. Exhibit 3 depicts the basic set of size and control metrics discussed in Chapter 5 for the Space Shuttle Primary Avionics Software System (PASS) Guidance and Navigational Control subsystem. This system was written in a programming language called HAL/S. To collect these data, we wrote a metric analyzer that conformed to the rules articulated in Chapter 5. The means and standard deviations of the various metrics are shown in Exhibit 3. On the surface there are some rather surprising values in this table. For example, the average program module has more than 7300 paths and there is considerable variation in this value. Also, there is substantial variation in the total operator and operand counts among the program modules. Of greater interest to us than the simple variation of each variable is the degree of interrelationship among these variables. To this end, the correlation coefficients for the bivariate relationships among all of the metrics is shown in Exhibit 4. We can see, for example, that there is a strong relationship between Exec and LOC (0.86). That is not surprising. There is also a strong relationship between Nodes and Edges; again, there are no surprises in this. What we are looking for is a pattern of communality among the variables. There seems to be a cluster of relationships among the size metrics and another among the flowgraph metrics. It turns out that these metrics are all related to one another simultaneously. Their individual bivariate relationships simply do not reveal how all of these metrics interact simultaneously. There is a multivariate relationship among them. We will use a multivariate statistical technique called principal components analysis (PCA) to explore these multivariate relationships. ^[3]

Exhibit 3: Metric Statistics for PASS

Metric	Mean	Standard Deviation
η1	15.5	13.3
η2	50.4	70.0
N1	299.6	593.5
N2	154.4	289.5
Exec	69.7	130.7
Loc	138.3	229.3
Nodes	34.3	54.5
Edges	43.9	74.4
Paths	7300.8	16667.9
Cycles	1.1	2.9
Maxpath	30.3	25.2
Avepath	46.8	42.5

Exhibit 4: Correlation Coefficients for PASS Metrics

	η1	η2	N1	N2	Exec	LOC	Nodes	Edges	Paths	Cycles	Maxpath
η2	0.78
N1	0.62	0.85
N2	0.64	0.89	0.99
Exec	0.64	0.89	0.97	0.96
LOC	0.61	0.79	0.84	0.83	0.86
Nodes	0.58	0.66	0.67	0.65	0.71	0.73
Edges	0.56	0.65	0.66	0.64	0.70	0.70	0.99
Paths	0.47	0.56	0.50	0.50	0.55	0.53	0.66	0.66
Cycles	0.39	0.45	0.39	0.40	0.42	0.39	0.63	0.65	0.49
Maxpath	0.60	0.65	0.60	0.60	0.64	0.65	0.87	0.87	0.70	0.82
Avepath	0.55	0.63	0.58	0.58	0.63	0.64	0.86	0.86	0.67	0.83	0.99

^[3]Jackson, J.E., A User's Guide to Principal Components, John Wiley & Sons, New York, 1991.