2.3. Some Number Stories
Most of the IT managers and executives I know have an intuitive appreciation for process, at least in general. What they most often ask me when I am making the case for them to formalize a process program in their shops is, "What's the return?" They want to know what kind of ROI they can expect from the investment they know they'll have to make. And it's not that they need proof. It's that the people around themperhaps a CEO or a CFOare trained to support decisions built around those kinds of concrete terms.
The fact is most of us in the process improvement field can't guarantee an ROI figure. We can anticipate a positive ROI. We know how to shape a program to generate an ROI. But each IT shop is different. Each has its own focus, its own culture, its own specialties, its own distinct customer base. All those variables make it tough to predict an ROI.
Most of this chapter will focus on what I call the qualitative benefits of process. And by qualitative, I don't mean "soft." Power steering gives you better control of an automobile, but how much more over manual depends on the driver. Likewise, process can give you better control over projects. That's a safe assumption. How much depends on what you build.
John Brodman once conducted an empirical study of the qualitative traits of process improvement and noted that many soft benefits were often overlooked. These included improved morale by the developers, less need for overtime, smoother day-to-day operations and communications, and increased respect from the customer base (Brodman 1995).
But I do appreciate the need to link process with hard improvement data. The best way I know to do this is by reference. Let's take a look at some companies that have made the commitment to process improvement, studied what they were doing, and then reported their results.
I think the best understanding to take away from these brief cases is that process improvement does have a track history of delivering tangible benefits. And these benefits have a very strong potential to deliver impressive returns for the organization that undertakes the effort seriously.
Process improvement at Schlumberger helped the company meet its on-time delivery goals. At program inception, the company was meeting schedule dates about 50 percent of the time. Less than three years after implementing a program that focused on process definition, project planning, and project tracking, on-time delivery rose to 99 percent. Extensions to the program then helped the company reduce post-release defect rates from 25 percent of all defects down to 10 percent. Schlumberger also reported productivity increases between 2X and 3X, and noted an ROI value for the program of 6:1 (Curtis 1995).
Raytheon has had similar success with its process management program. Its goal was, in part, to reduce the amount of rework it had to deal with in its projects. Before the program, rework typically accounted for up to 41 percent of a project's budget. Raytheon invested about half a million dollars in a program to address this issue, and in the first full year of its use, realized a savings on rework of $4.48 million, along with a 2X productivity increase (as might be expected). After just under five years, the savings were documented at $15.8 million, and rework accounted for only about 11 percent of the project budget (Dion 1993).
2.3.3. Tenzer-Spring Dynamics
Tenzer initiated a process program to help the company better manage its development costs. Hard costs of internal projects were slimming the company's profit margins. Net margins that were anticipated in the range of 12 to 19 percent were being realized at closer to 9-to-14 percent levels. The company invested about $400,000 into an enterprise program to define processes around project planning, project monitoring, and requirements control. After two years of implementation, the company documented a reduction in project overruns of just under $3 million. Development costs dropped by 55 percent, cycle time dropped by 38 percent, and post-release defects dropped by 74 percent (Lildman 2001).
2.3.4. Behrben International
Behrben directed its process improvements at a very specific issue: requirements validation. The company was dealing with two issues. First the lack of precision in requirements definition was resulting in many iterative cycles of validation. The effort of these cycles had a cascading effect on project schedules. Schedule slippage was endemic across project groups. After Behrben standardized a methodology for requirements definition and control, activities began to improve. Validation cycles dropped from an average of 34 to 17, a 50 percent reduction. As a result, delivery dates began to improve also. On-time delivery of software increased from 51 percent of the projects to 94 percent in less than 19 months.
2.3.5. Hughes Aircraft
Hughes Aircraft invested in a process initiative that focused on project and team support areas: training, peer reviews, Process Group formation, and quality assurance. Hughes also implemented quantitative measures to track how well these areas impacted overall operations. From an investment of $445,000, the company reported savings of $2 million annually. Managers also reported softer benefits realized from the program: improved quality of work life in the business environment, fewer overtime hours, and increased retention. Additionally, management attributed this program, in part, to a rise in the company's marketplace image (Humphrey 1993).
2.3.6. Boeing Space Transportation Systems
Boeing Space Transportation Systems focused one aspect of its process improvement efforts on cycle time and defect detection, two interrelated productivity gates. The company addressed this with a formalized peer-review program. Initially, about 70 percent of defects were discovered during verification activities, with an additional 19 percent being found in validation. After the peer reviews began, a significant number of the defects were eliminated before even reaching verification. This improved the cycle times by 50 percent and dropped the amount of rework that needed to be done by 31 percent. The reviews also had the added benefit of increasing design effort efficiencies by 25 percent (4 percent of total development). This 4 percent increase in overall project efficiency provided for the 31 percent reduction in rework (Yamamura and Wigle 1997).
Hewlett-Packard worked with two of its divisions to focus process improvement around the benefits of reuse. The company calculated that its ability to identify and reuse proven systems code and components was crucial to achieving productivity and quality objectives. After the reuse program went into effect, one group experienced a 51 percent reduction in code defects with a 57 percent increase in productivity. The other group experienced a 24 percent defect reduction, with a 40 percent increase in productivity. One group within HP that had been committed to reuse for 10 years reported gross (period) program costs of about $1 million with a savings of $4.1 million. Another group, involved in reuse for eight years, reported gross program costs at $2.6 million with savings of $5.6 million (Lim 1994).
2.3.8. An SEI Study
Finally, Carnegie Mellon University studied a group of technology companies that had been committed to process improvement an average of three years. The results showed a range of positive indicators. Project costs were reduced from 5 percent to as much as 83 percent. On-time delivery was improved from 15 percent to as high as 95 percent. Productivity increases were realized from 11 percent to 60 percent. And all companies showed a positive return on investment. ROIs ranged from 2:1 to 13:1.
These are all success stories. And one of the reasons they have been publicly released is no doubt because they are success stories.
As compelling as these cases are, they don't seal the case for process improvement and process programs. But they go a long way to helping us see what kinds of tangible and measurable improvements well-designed processes can bring to an IT organization.
The error at the other extreme would be to ignore those numbers, to treat them as belonging to a different industry set. People might say, "That's fine for Boeing, but we are not Boeing; we're a small shop." Or "Schlumberger's industry is totally different from ourswhat works in that industry won't work in ours." And so on.
Often I get those responses from the same people who asked me to provide the ROI data in the first place. I think that attitude goes back to one of orientation. Many IT managers are not adverse to process. They just lack the appreciation for how it can be implemented in almost any shop.
Along that line, I've noticed six pretty common myths that are often cited as reasons why managers choose not to consider making process improvement a focus in their shops. We'll take a look at these in the next section.