Philip Crosby, in Quality Is Free,[3] argues that the cost of quality (CoQ) is "the expense of nonconformancethe cost of doing things wrong." Some call it the "cost of poor quality" (CoPQ) or the "price of nonconformance" (PoNC). Determining CoQ is important, because it reveals how much you expend on prevention, appraisal, inspection, discovering, analyzing, scrapping, and correcting defects, faults, and errors. Software process maturity and quality levels lag hardware by a factor of 20 to 1. The cost of poor quality (CoPQ) related to software bugs, development cost overruns, botched implementation, and canceled projects is astronomical. A recent study puts one aspect of CoPQ, canceled projects, in excess of $300 billion in the United States alone.[4] Cost of software quality (CoSQ) can be defined as the total cost of conformance and nonconformance to the customer's quality requirements. It consists of various direct and indirect expenses incurred when preventing, appraising, testing, discovering, analyzing, and fixing software faults, including maintenance. As we discuss in Chapters 1 and 2, some 40% of software development cost is spent on testing alone to remove errors and ensure adequate quality. Further, some 80 to 90% of the total software life-cycle cost (LCC) goes into software maintenance to fix, adapt, and expand the delivered program to meet users' changing and growing needs.[5] Benefits of Cost-of-Quality AnalysisIdentifying, understanding, and managing the cost of software quality are crucial components of a DFTS initiative for the following reasons:[6]
Traditional cost-evaluation metrics, generally associated with manufactured products, have focused on internal costs of rework, rejects, and returns; therefore, they have focused just on "failure-related activities."[7] They emphasize inspection and cost minimization by taking corrective measures following inspection. Although the need for corrective measures cannot be eliminated, an enlightened approach emphasizes preventive interventions so that corrective measures are unneeded in the first place, or can be minimized. It should be emphasized that quality can be "free" only if the cost of quality improvement tasks is less than the consequent cost of not undertaking such tasks. This is possible only if adequate quality tasks are actually performed and are targeted at failure reduction. Such reduction can be dramatically enhanced by employing ways and means to move failure prevention, detection, and removal activities closer to the front end of the development processto the upstream stages of requirements planning, concept development, and design. Only in this way can the cost of performing quality tasks become less than the cost of not performing quality tasks.[8] Evaluating the costs of not performing quality tasks is always challenging, because so many intangible factors are related to customer expectations and competitive factors that are in play in a particular industry. Often, conservative estimates are the only way to come up with a workable figure. This issue is discussed further when we cover Taguchi's Quality Loss Function (QLF) later in this chapter. Two sets of measures are involved in cost benefits related to quality tasks: the costs associated with quality tasks, and the cost savings from performing quality tasks. Cost of Quality TasksFigure 4.2 shows the relationships between the cost components of various quality tasks. They can be summarized as follows:
Figure 4.2. Cost Breakdown of Software Quality TasksIt's important to understand this cost structure and its implications for overall quality and product cost for three reasons:
The following example illustrates the points we want to make:
The objective should be to reverse this process. The majority of quality costs should be incurred at prevention, some in appraisal, perhaps a few in upstream and downstream internal failures, and virtually none in external failure. Thus, the software developer should first attempt to reduce external failure costs to zero by investing in preventive and appraisal activities to design bug-free software in the first place. Then he or she should discover sources of design and coding failure, if any, and take corrective actions consequently. As quality improves, failure costs decrease, and the amount of appraisal can be reduced with the shift of emphasis to prevention activities.[10] Determining cost of quality is a first step in identifying and removing the root causes of failure. Classification of Cost of Software QualityThe profile of software quality costs differs from that of manufactured products. The costs reflect the manpower intensity, service orientation, and more intense customer interaction of a software development process. Because much of the work is intellectual, it lacks the visibility of a manufacturing process. The required visibility must be "created" by appropriate documentation. Just like in other service organizations, user interaction, or the lack thereof, is a crucial element of overall costs. CoSQ can be classified into five major categories:
Prevention CostsThis category includes various costs associated with equipment, systems, and manpower associated with preventive upstream activities to ensure that nonconforming software products are not designed in the first place. These costs can be broken down as follows:
Appraisal CostsThis category includes various costs associated with activities carried out to ensure that the software designed and developed meets customer requirements. This includes salaries, equipment, measurement and analysis, customer interaction, and equipment costs. The activities involved are
Upstream Internal Failure CostsIncludes all costs related to manpower and equipment incurred in detecting and correcting defects and faults associated with requirements identification, design, and coding. This involves self-debugging your own unit and components. All this may also require interacting with the customer and user domains and carrying out risk analysis. The formal testing by the independent test team is done in the following stage and is identified as part of downstream internal failure costs. Failure costs escalate after self-debugging, because it involves the following steps:
It helps a great deal if self-debugging can detect and fix faults. Downstream Internal Failure CostsThis category includes all costs related to manpower and equipment incurred in detecting and correcting defects and faults associated with review, test and evaluation, and verification and validation. Testing at this stage involves an independent test team. As just mentioned, the internal failure costs in the downstream stages may be much higher than in the previous stage. These activities also require interacting with the customer and user environments and carrying out risk analysis. External Failure CostsHere the costs may be even higher than those incurred at the two internal failure stages just discussed. The faults in this case are found by the customers and not by self-debugging or by the internal test engineer. The whole cycle of fault fixing has to be undertaken. The costs include all manpower as well as equipment-related costs associated with detecting and correcting faults and complaints following release to customers. This category also includes
As stated earlier, the cost of software quality can be minimized dramatically by taking preventive measures, and corrective measures, if needed, upstream. (Table 4.2 lists essential activities at various stages.) The investments made in upstream preventive measures provide a much higher payoff than development models that rely on corrective measures downstream, or worse, corrective measures following failure during customer usage. External failures involve numerous unforeseen and intangible costs. In extreme cases, the potential damage may cripple the business. In addition to a sound preventive system, provisions must be made for costs related to insurance, product liability, and legal defense. The best strategy is, of course, prevention.
Cost-reduction potentials are realized only if quality-related activities are based on clear analysis and understanding of root causes of faults, which a CoSQ analysis helps reveal. A CoSQ quality matrix points out quality problems and associated costs, as shown in Table 4.3. It is important not to get into a futile discussion about a particular item's cost categorization. Certain items could be considered either a development activity or a quality-related one. It is best to exclude debatable items in CoSQ reporting. Similarly, a cost item may belong to one category or another, depending on a particular perspective. However, major cost items are not difficult to categorize. In all such cases, it is best to be consistent so that improvements and opportunities for improvements are clearly identified.
CoSQ reporting is the first step in identifying the root causes that need to be addressed by the software developer and outsourcing firms if they are involved. Various tools such as process mapping, cause-and-effect diagrams, and brainstorming can be used to identify the root causes (see Chapter 6). Establishing a CoSQ Reporting SystemThe first requirement of a CoSQ program is to establish a reporting system that identifies costs incurred due to poor quality. Establishing such a reporting system is a major initiative and must be carefully planned. Evans and Lindsay[11] have summarized a 12-step implementation plan based on the recommendations of the Institute of Management Accountants:
Payback from Investment in QualityQuality initiatives are actually investments in future benefits and cost savings. They are like investments in preventive health care with a view to enjoying the benefits of good health (the quality). They also prevent catastrophic health-care expenses (costs) that may have to be incurred if prevention is ignored. The benefits or savings to be realized are not always apparent. For, example although certain internal cost savings are identifiable, costs of external failures associated with regulation, litigation, bad publicity, loss of customer or market share, and bankruptcies are difficult to estimate. Equally, the impact of quality on future revenue and business growth is difficult to estimate. Dollar estimates are just thatestimates. Being obsessed with absolutely measurable costs and benefits is counterproductive to any quality initiative. This mind-set of absolute accuracy and measuring everything often becomes a major irritant. If past records are available, they should be used. But it is impossible to get all the data on costs and benefits. It is best to take a practical approach toward these matters: make a conservative estimate of costs and benefits, and take appropriate measures upstream to prevent external failures. As a CoSQ system is established and up and running, such data is available more readily. The payback from investment in various quality tasks can be expressed as follows:
The Return on Quality Investments (ROQI) is calculated as follows:
Discounted Cash Flow ApproachROQI normally is discounted using Net Present Value (NPV)/Internal Rate of Return (IRR) analysis. Another important Discounted Cash Flow (DCF) metric is the payback period. ROQI with discounted return is
Value of CoSQ AnalysisQuality initiatives very often emanate from quality professionals. Management may not be fully aware of the benefits to be realized. A good CoSQ analysis invariably reveals surprising, even shocking, facts to upper management. They are likely to understand and respond to such revelations because CoSQ is expressed in the language they understand bestdollars. It uncovers the cost of not paying attention to quality and helps identify activities that need to be addressed. A CoSQ metric program thus plays a critical role in successful execution of a DFTS initiative from both quality and cost perspectives. It may be added that normally an appropriate quality task saves more than it costs. At times, a quality task may cost more than it saves, but it may satisfy a critical customer requirement. Customer requirements and not cost alone should be the determining factor in initiating a quality task. Equally, customer requirements may necessitate cost, quality, and schedule optimization, a basic precept of DFTS. The important thing about DFTS is that it helps you understand customer requirements throughout the software life cycle. Such understanding of various customer requirements provides the need and opportunity for robust optimization. We discuss this in Chapters 16 and 17. Pitfalls of a CoSQ ProgramA CoSQ initiative is too critical to be allowed to go astray. It is therefore important to be aware of the following potential pitfalls:
|