Detailed Benefits of Improved Software Practices

An in-depth study of 13 organizations by the Software Engineering Institute found that the typical (median) organization engaged in systematic improvement experienced a productivity gain of 35 percent per year, schedule reduction of 19 percent per year, and reduction in post-release defect reports of 39 percent per year. These gains provided the underpinnings for the overall returns on investment. These results are summarized in Table 13-1.

The gains experienced by the best organizations were even better. The organization with the strongest productivity gains improved 58 percent per year over a 4-year period, for a total compounded gain of more than 500 percent. The best schedule reduction was 23 percent per year for 6 years, for a total compounded schedule reduction of 91 percent. The best long-term quality improvement was a reduction in post-release defect reports of 39 percent per year for 9 years, for a total compounded defect reduction of 99 percent. Two organizations achieved short-term defect reductions of 70 percent or more in less than 2 years.

Table 13-1. Results of software process improvement efforts^[8]

FACTOR	MEDIAN IMPROVEMENT	BEST SUSTAINED IMPROVEMENT
Productivity	35%/year	58%/year
Schedule	19%/year	23%/year
Post-release defect reports	39%/year	39%/year[*]
Business value of organizational improvement	500 percent	880 percent

^[*] The "median" and "sustained" values here are the same because the highest defect-reduction results were short-term and not counted as "sustained" improvements.

Organizations that are hooked on code-and-fix development tend to think there is a tradeoff between low defect count and productivity. But as I explained in Chapter 2, much of the cost on a code-and-fix project arises from unplanned defect-correction work. The results in Table 13-1 confirm that for most organizations no tradeoff exists between higher productivity and better quality. Organizations that focus on preventing defects also get shorter schedules and higher productivity.

As a percentage, the number of companies that have engaged in systematically improving their software practices is small. In raw numbers, hundreds of organizations have engaged in systematic improvement, and many have reported their results in professional journals, conference proceedings, and other publications. Table 13-2 summarizes the returns on investment reported by 20 organizations.

Table 13-2. Examples of software improvement ROI^[9]

ORGANIZATION	RESULTS
BDN International	ROI 300%
Boeing Information Systems	Estimates within 20%, $5.5 million saved in 1 year, ROI 775%
Computer Sciences Corporation	65% reduction in error rates
General Dynamics Decision Systems	70% reduction in rework; 94% defect rate reduction; 2.9 x productivity gain
Harris ISD DPL	90% defect rate reduction; 2.5 x productivity gain
Hewlett-Packard SESD	ROI 900%
Hughes	$2 million annual reduction in cost overruns, ROI 500%
IBM Toronto	90% reduction in delivered defects, 80% reduction in rework
Motorola GED	2 3 x productivity improvement, 2 7 x cycle time reduction, ROI 677%
Philips	ROI 750%
Raytheon	ROI 770%
Schlumberger	4 x reduction in beta test bugs
Siemens	90% reduction in released defects
Telcordia	Defects 1/10 industry average, customer satisfaction increased from 60% to 91% over 4 years
Texas Instruments Systems Group	90% reduction in delivered defects
Thomson CSF	ROI 360%
U.S. Navy	ROI 410%
USAF Ogden Air Logistics Center	ROI 1,900%
USAF Oklahoma City Air Logistics Center	ROI 635%
USAF Tinker Air Force Base	ROI 600%