Chapter 4: Where Does Estimation Error Come From?

There's no point in being exact about something if you don't even know what you're talking about.
—John von Neumann

Overview

A University of Washington Computer Science Department project was in serious estimation trouble. The project was months late and $20.5 million over budget. The causes ranged from design problems and miscommunications to last-minute changes and numerous errors. The university argued that the plans for the project weren't adequate. But this wasn't an ordinary software project. In fact, it wasn't a software project at all; it was the creation of the university's new Computer Science and Engineering Building (Sanchez 1998).

Software estimation presents challenges because estimation itself presents challenges. The Seattle Mariners' new baseball stadium was estimated in 1995 to cost $250 million. It was finally completed in 1999 at a cost of $517 million—an estimation error of more than 100% (Withers 1999). The most massive cost overrun in recent times was probably Boston's Big Dig highway construction project. Originally estimated to cost $2.6 billion, costs eventually totaled about $15 billion—an estimation error of more than 400% (Associated Press 2003).

Of course, the software world has its own dramatic estimation problems. The Irish Personnel, Payroll and Related Systems (PPARS) system was cancelled after it overran its €8.8 million system by €140 million (The Irish Times 2005). The FBI's Virtual Case File (VCF) project was shelved in March 2005 after costing $170 million and delivering only one-tenth of its planned capability (Arnone 2005). The software contractor for VCF complained that the FBI went through 5 different CIOs and 10 different project managers, not to mention 36 contract changes (Knorr 2005). Background chaos like that is not unusual in projects that have experienced estimation problems.

A chapter on sources of estimation error might just as well be titled "Classic Mistakes in Software Estimation." Merely avoiding the problems identified in this chapter will get you halfway to creating accurate estimates.

Estimation error creeps into estimates from four generic sources:

• Inaccurate information about the project being estimated

• Inaccurate information about the capabilities of the organization that will perform the project

• Too much chaos in the project to support accurate estimation (that is, trying to estimate a moving target)

• Inaccuracies arising from the estimation process itself

This chapter describes each source of estimation error in detail.