Chapter 11: The Measurement of Software Testing Activity

11.1 Introduction

Sometimes there is ambiguity in the goals and objectives of the software testing process. For our purposes, we would like to remove this ambiguity. The objective of the test process will be to certify a software system for a range of behaviors thought to be representative of a typical user's operation profile. In this guise, the software should perform reliably as long as it is operated within the limits of behavior that we have certified. Our real objective in this process is to be able to guarantee the reliable operation of our software. The test process will be used to achieve this objective.

We are rapidly coming to the close of the Wild West days of software development. The corporate survivors in the new world of software development will be those that can certify and guarantee the safe and reliable operation of their software. Sound engineering practice is essentially the only way that a guarantee of reliable software operation can be made. The one thing that we have learned over the years is that it is not possible to use the software test process to achieve reliability. By the time the system arrives in the tester's hands, the damage has been done. Bad software process cannot be redeemed during testing. That will not be an objective of the testing process. Rather, the testing process will be used to certify that the software will perform reliably within the context of a well-defined operating environment.

Each test activity will exercise one or more functionalities. This will, in turn, cause subsets of code modules to be executed. Not all modules are equal. Some are potentially fault prone while others are not so fault prone. The astonishing level of complexity of modern software will guarantee that we will not have the resources to identify all possible faults and fix them. Further, as we have seen, an attempt to fix one fault might well introduce additional faults. We would like, then, to have the ability to marshal our resources so that we can have the greatest impact for the very limited resources that will be available to the test activity. The test activity is perhaps the most resource-poor activity in the entire software development cycle. We need to know how to get the biggest bang for the limited test dollars.

The results of each test activity can be quantified. It is possible to evaluate that test activity in terms of the benefit derived from it. It would be pure folly to devote test resources to an activity that provides little or no exposure to latent faults in the system. The test measurement enterprise will allow us to quantify and evaluate each test. This knowledge will guide us in our future test activities.

In Chapter 8 we saw that an evolving software system is very difficult to manage, measure, and describe. A family of software modules may lay quiescent in the system for many build cycles, having received little or no attention from developers. As a result, the reliability of these modules will probably increase over time. We will learn to accept the reliability of this family and devote our test resources to other, more active code. All of a sudden, the attention of the developers may well turn to this code backwater. In the face of heavy change activity, our code family that was once reliable may now begin to inherit more and more faults. This code family must now be promoted to be a center of attention for future test activity. In that the code base is not static, there can be no static test plan. A test plan must be a living document that changes as the code base changes.



Software Engineering Measurement
Software Engineering Measurement
ISBN: 0849315034
EAN: 2147483647
Year: 2003
Pages: 139

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net