Test Compression Factor

As the example and other cases in the literature illustrate (for example, Misra, 1983; Putnam and Myers, 1992), a fair degree of accuracy to project the remaining number of defects can be achieved by software reliability models, based on testing data. This approach works especially well if the project is large, where defect arrivals tend not to fluctuate much; if the system is not for safety-critical missions; and if environment-specific factors are taken into account when choosing a model. For safety-critical systems, the requirements for reliability and, therefore, for reliability models, are much more stringent.

Even though the projection of the total number of defects (or defect rates) may be reasonably accurate, it does not mean that one can extend the model density curve from the testing phase to the maintenance phase (customer usage) directly. The defect arrival patterns of the two phases may be quite different, especially for commercial projects. During testing the sole purpose is to find and remove defects; test cases are maximized for defect detection, therefore, the number of defect arrivals during testing is usually higher. In contrast, in customers' applications it takes time to encounter defects ”when the applications hit usual scenarios. Therefore, defect arrivals may tend to spread. Such a difference between testing-defect density and field-defect density is called the compression factor . The value of the compression factor varies, depending on the testing environments and the customer usage profiles. It is expected to be larger when the test strategy is based on partition and limit testing and smaller for random testing or customer-environment testing. In the assessment by Elbert and associates (1992) on three large-scale commercial projects, the com-pression factor was 5 for two projects and 30 for a third. For projects that have extensive customer beta tests, models based on the beta test data may be able to extrapolate to the field use phase.

Figure 8.4 shows a real-life example of compression of defect density between testing and initial data from the field. For the upper panel, the upper curve represents the extrapolated cumulative defect rate based on testing data. The lower curve is the actual cumulative field defect rate. Although the points of the two curves at four years in the life of product are close, the upper curve has a much faster buildup rate. The difference is even more drastic in the lower panel, in which the two defect density curves are contrasted. The extrapolated curve based on testing data is front loaded and declines much faster. In vivid contrast, the actual field defect arrival is much more spread out. It even follows a different density pattern (a delayed S or a Rayleigh-like pattern).

Figure 8.4. Compression Illustration ”Cumulative and Density Curves

graphics/08fig04.gif

What Is Software Quality?

Software Development Process Models

Fundamentals of Measurement Theory

Software Quality Metrics Overview

Applying the Seven Basic Quality Tools in Software Development

Defect Removal Effectiveness

The Rayleigh Model

Exponential Distribution and Reliability Growth Models

Quality Management Models

In-Process Metrics for Software Testing

Complexity Metrics and Models

Metrics and Lessons Learned for Object-Oriented Projects

Availability Metrics

Measuring and Analyzing Customer Satisfaction

Conducting In-Process Quality Assessments

Conducting Software Project Assessments

Dos and Donts of Software Process Improvement

Using Function Point Metrics to Measure Software Process Improvements

Concluding Remarks

A Project Assessment Questionnaire



Metrics and Models in Software Quality Engineering
Metrics and Models in Software Quality Engineering (2nd Edition)
ISBN: 0201729156
EAN: 2147483647
Year: 2001
Pages: 176

Flylib.com © 2008-2020.
If you may any questions please contact us: flylib@qtcs.net