Analysis concepts

5.1 Analysis concepts

Defect analysis is the discipline of measuring software development and use. The measures can come from surveys, defect data, schedule and budget plans and records, system characteristics, help line usage, and so on. Each organization will discover and determine the sources of measurements best suited to their own needs and situations.

Metrics are an essential part of any software quality system. They are relationships between measures that turn the measurement data into applicable, quality management information. However, they are too often merely an exercise in collecting numbers without developing any useful information. The role of the quality assurance practitioner is to bring to decision-making, action-capable managers the information they need to beneficially affect the software development process, and metrics are only useful when they contribute to that information base.

5.1.1 Measures

A number, such as 6 or 1,000, is not a metric. It is merely a number, usually with no information value. When a dimension is added to the number, such as lines of code (LOC) or critical errors (CE), a measure is created that is more descriptive than a number by itself but still without significant utility. Six CEs and 1,000 LOCs are both measures, but they hold no information until they are related to something.

5.1.2 Metrics

In this book, a metric is defined as the ratio of, or relationship between, two measures. Thus, a defect density of 6 CEs per 1,000 LOCs (KLOC) begins to take on the characteristic of information. Ratios of metrics finally reach real information status. For example, one could compare a defect density of 6 CE/KLOC before institution of software inspections to a defect density of 2 CE/KLOC after inspections begin.

Defect metrics are, not surprisingly, those metrics composed of measures dealing with defects. The measures might include number of software system failures, number of calls made to the help line, time spent recoding after a failure, and cost of lost sales after a bad press review based on too many errors found by the reviewer. The list could go on and on. A typical defect metric is number of problem reports closed versus the number of new problem reports recorded.

Nondefect metrics are just that; metrics that are not based on defects. Budget overruns, schedule shortfall, size of the software system in lines of code or function points, module complexity, cost of computer time, and the like are representative of nondefect measures. Nondefect measures are combined to develop nondefect metrics. An example of a nondefect metric might be LOC developed per person-month.

5.1.3 Product analysis

Product analysis is the first area for most organizations to begin to measure. Error frequency, software product size, development cost, number of tests run successfully, and the like are often the kinds of things measured in the beginning. Most of these product metrics can be developed directly from the software trouble reports (STR) and project planning documentation. Product metrics deal with the attributes of the product, defect characteristics, and other data about the product.

Using product metrics, the software quality control practitioner can locate error-prone areas of the code, design weaknesses, testing flaws, and so on. These metrics also help the quality assurance practitioner identify efforts that can beneficially affect the specific product being analyzed. In the longer run, product analysis will build up a body of information that can be applied to process-oriented analyses.

5.1.4 Process analysis

It is the goal of the software quality assurance practitioner to improve the process by which the various software products are produced. Process improvement models and process assessment models are becoming important throughout the software industry. International standard ISO 9000 is a family of quality standards that require stable and improving processes. The SEI's CMMI is a model for assessing the ongoing improvement in the software development process of an organization. Another international effort, ISO 15504, is under way and it, too, is a process assessment and improvement approach. These and other efforts are based on the concept of process identification, definition, and continuous improvement.

Process understanding and improvement depend heavily on the results of product analysis. As each product is reviewed, tested, operated, and maintained, a history of its defects and changes can be kept. By itself, the record of analysis of defect and nondefect data for a single product will not be especially useful on a process basis. It is the continuing analysis of multiple products' data that leads to the body of information that describes the process. At this point, the software quality assurance practitioner can begin to describe the process and its behavior.

Analysis of defect detection methods and their results can give insight into their value. Comparing when and how defects are detected as opposed to when they were introduced into the software products is one view of that process. Comparing budget and schedule actual values with estimates reflects on the estimation and management processes. Analysis of defect types gives insight into the development process strengths and weaknesses. Tracking the levels of the costs of the quality program versus its effects provides information about the quality process. Once an understanding of the behavior of the process is achieved, intentional modification to the process will result in identifiable changes in the products it produces. In this way, the software quality assurance practitioner is able to suggest beneficial process changes based on data and information rather than on guesses.

For example, if defect records show that walk-throughs are finding more defects than testing, the quality practitioner may do further research to determine if the walk-throughs are finding most of the defects and that fewer exist for testing to uncover, or if the testing process is not sufficiently robust.



Practical Guide to Software Quality Management
Practical Guide to Software Quality Management (Artech House Computing Library)
ISBN: 1580535275
EAN: 2147483647
Year: 2002
Pages: 137
Authors: John W. Horch

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net