6.6 Discovering New Sources of Variation

6.6 Discovering New Sources of Variation

One of the fundamental benefits of the PCA technique is that it allows us to identify exactly what it is we are measuring. That is, we can identify relevant sources of variation in our working set of metrics. To date, there are 12 metrics in this working set. It is not clear, at this point, just why we have so many metrics that all seem to measure either size or control characteristics of program modules. Although there are six different measures of program size, they will each contribute meaningfully to our understanding of a criterion measure. We will discuss this further in the next chapter. What we would like to do next is to see if we can expand our knowledge in what we are measuring.

In Chapter 5, a new metric, the data structures metric DS, was introduced. This was an outgrowth of a series of studies to identify new sources of variation, particularly in the area of data structures complexity. The need to measure data structures complexity was driven by an observation that a very large number of program discrepancy reports (DRs) involved programmer misuse or misunderstanding of particular data structures in the HAL/S programming language for the Space Shuttle PASS. We developed several candidate data structures metrics, each of which was believed to measure the observed variability in data structures complexity. Sadly, most of these metrics turned out to be new measures of program size, as is generally the case with unvalidated metrics. One metric, the DS metric presented in Chapter 5, was distinctly different from the other measures of data structures complexity. It behaved in a very different manner from the other 12 measures. The results of the varimax rotation of the PCA with the new DS metric are shown in Exhibit 11. In this exhibit we can see that the DS metric is distinctly correlated with only one of the orthogonal domains. Further, it is really the only metric so correlated. It identifies a new source of variation distinctly different from other measures of size and control complexity. Thus, the DS metric has met the first of several criteria for a valid metric; it has measured a new and distinct source of variation. In Chapter 7 we conduct the next level of validation. We want to establish in this next step that the DS metric will help us explain the observed variation in a criterion measure.

Exhibit 11: PCA with the DS Metric

start example

Metric

Size

Control

Data Structures

η1

0.67

0.36

-0.25

η2

0.87

0.36

-0.03

N1

0.92

0.28

0.08

N2

0.93

0.28

0.07

Exec

0.91

0.39

0.07

LOC

0.82

0.38

0.05

Nodes

0.49

0.80

-0.00

Edges

0.47

0.81

0.01

Paths

0.36

0.66

0.01

Cycles

0.17

0.86

0.03

Maxpath

0.37

0.91

-0.00

Avepath

0.35

0.91

0.01

DS

0.05

0.03

0.98

% Variance

40.6

36.8

8.0

end example

In passing, it is interesting to note the tenuous relationship between the unique operator count and the data structures metric. We see that η1 appears to be inversely related to the DS metric. This reminds us of a book by Nicklaus Wirth entitled Programs = Algorithms + Data Structures that was popular in the 1970s. There are many ways to solve the same problem. We can, in essence, trade off data structures complexity for algorithmic complexity. This is most certainly the case with the use of pointers in the C programming environment. The use of pointers in these applications will certainly reduce the data structures complexity. There is an enormous cost to pay for this trade-off. The algorithmic complexity of a program will increase as a result of this design decision. We will see this in a concomitant increase in η1 and η3.

Let us return to the discrepancy reports for the PASS data and examine them further. PASS is a family of embedded real-time programs working together. Program synchronization (or lack thereof) is a significant source of errors made by the programmers. It would be very useful to identify measures of interprocess and intraprocess communication to be able to isolate these two sources of potential variation among program modules. To this end, a variety of candidate metrics were created. A subset of these measures were a measure in interprocess communication (SIGNALS) and intraprocess communication (SETS, RESETS, and CANCELS). These are simple enumeration of these HAL/S commands in the individual program modules.

As was the case for the data structures metric, data was collected for these new program measures. The results of the PCA with these metrics and the established 13 metrics are shown in Exhibit 12. In this new case, we can identify two new sources of variation corresponding to the role that new metrics are playing in inter- and intraprocess communication.

Exhibit 12: PCA with Process Communication Metrics

start example

Metric

Size

Control

Data Structures

Intraprocess

Interprocess

η1

0.65

0.36

-0.25

0.13

0.10

η2

0.85

0.36

-0.03

0.14

0.03

N1

0.91

0.28

0.08

0.18

0.02

N2

0.91

0.27

0.07

0.19

0.04

Exec

0.90

0.34

0.07

0.17

-0.01

LOC

0.77

0.37

0.05

0.30

-0.02

Nodes

0.47

0.79

0.00

0.15

-0.02

Edges

0.45

0.80

0.00

0.13

-0.02

Paths

0.39

0.67

0.02

-0.04

0.03

Cycles

0.11

0.86

0.03

0.06

0.07

Maxpath

0.35

0.90

0.00

0.17

0.06

Avepath

0.32

0.90

0.01

0.19

0.06

DS

0.05

0.03

0.98

0.02

0.02

Sets

0.25

0.16

0.05

0.80

-0.07

Resets

0.09

0.17

-0.01

0.82

-0.09

Signals

0.03

0.09

0.02

0.03

0.96

Cancels

0.21

-0.02

-0.03

0.63

0.27

% Variance

30.2

28.2

6.1

12.0

6.1

end example

Both the DS metric and the new process synchronization metrics represent steps in understanding the different sources of variation in program complexity. Some of this knowledge will prove useful and some will not. The driving force for discovery of new sources of variation is to identify those measurable program attributes that will allow us to improve our prediction of measures of software quality. That is, new measures must also vary in accordance with measures of software quality such as software faults. I can measure many distinct and different attributes of students in my class. Among these attributes, I could find, for example, a distinct source of variation in eye color, as measured by the wavelengths of light reflected from a student's iris. The knowledge gained from this measurement, however, would only be useful to me if it were related to a student's future performance in my class.



Software Engineering Measurement
Software Engineering Measurement
ISBN: 0849315034
EAN: 2147483647
Year: 2003
Pages: 139

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net