In-Process Metrics for Software Testing

In Chapter 9 we discussed quality management models with examples of in-process metrics and reports. The models cover both the front-end design and coding activities and the back-end testing phases of development. The focus of the in-process data and reports , however, are geared toward the design review and code inspection data, although testing data is included. This chapter provides a more detailed discussion of the in-process metrics from the testing perspective. [1] These metrics have been used in the IBM Rochester software development laboratory for some years with continual evolution and improvement, so there is ample implementation experience with them. This is important because although there are numerous metrics for software testing, and new ones being proposed frequently, relatively few are supported by sufficient experiences of industry implementation to demonstrate their usefulness . For each metric, we discuss its purpose, data, interpretation, and use, and provide a graphic example based on real-life data. Then we discuss in-process quality management vis-  -vis these metrics and revisit the metrics framework, the effort/outcome model , again with sufficient details on testing- related metrics. Then we discuss some possible metrics for a special test scenario, acceptance test with regard to vendor-developed code, based on the experiences from the IBM 2000 Sydney Olympics project by Bassin and associates (2002). Before we conclude the chapter, we discuss the pertinent question: How do you know your product is good enough to ship?

[1] This chapter is a modified version of a white paper written for the IBM corporate-wide Software Test Community Leaders (STCL) group , which was published as "In-process Metrics for Software Testing," in IBM Systems Journal ,Vol. 40, No.1, February 2001, by S. H. Kan, J. Parrish, and D. Manlove. Copyright 2001 International Business Machines Corporation. Permission to reprint obtained from IBM Systems Journal .

Because the examples are based on IBM Rochester's experiences, it would be useful to outline IBM Rochester's software test process as the context, for those who are interested. The accompanying box provides a brief description.