4.8 Off-The-Cuff Estimates

Project teams are sometimes trapped by off-the-cuff estimates. Your boss asks, for example, "How long would it take to implement print preview on the Gigacorp Web site?" You say, "I don't know. I think it might take about a week. I'll check into it." You go off to your desk, look at the design and code for the program you were asked about, notice a few things you'd forgotten when you talked to your manager, add up the changes, and decide that it would take about five weeks. You hurry over to your manager's office to update your first estimate, but the manager is in a meeting. Later that day, you catch up with your manager, and before you can open your mouth, your manager says, "Since it seemed like a small project, I went ahead and asked for approval for the print-preview function at the budget meeting this afternoon. The rest of the budget committee was excited about the new feature and can't wait to see it next week. Can you start working on it today?"

I've found that the safest policy is not to give off-the-cuff estimates. Lederer and Prasad found that intuition and guessing in software project estimates were both correlated with cost and schedule overruns (Lederer and Prasad 1992). I've collected data on off-the-cuff estimates from 24 groups of estimators. Figure 4-8 shows the average error of estimates in these 24 groups of estimators for off-the-cuff estimates versus estimates that go through a group review process.

image from book
Figure 4-8: Average error from off-the-cuff estimates vs. reviewed estimates.

The average off-the-cuff estimate has a mean magnitude of relative error (MMRE ^[1]) of 67%, whereas the average reviewed estimate has an error of only 30%—less than half. (These are not software estimates, so the percentage errors shouldn't be applied literally to software project estimates.)

One of the errors people commit when estimating solely from personal memory is that they compare the new project to their memory of how long a past project took, or how much effort it required. Unfortunately, people sometimes remember their estimate for the past project rather than the actual outcome of the past project. If they use their past estimate as the basis for a new estimate, and the past project's actual outcome was that it overran its estimate, guess what? The estimator has just calibrated a project overrun into the estimate for the new project.

While Lederer and Prasad found that guessing and intuition were positively correlated with project overruns, they also found that the use of "documented facts" was negatively correlated with project overruns. In other words, there is a world of difference between giving your boss an off-the-cuff answer versus saying, "I can't give you an answer off the top of my head, but let me go back to my desk, check a few notes, and get back to you in 15 minutes. Would that be OK?"

While this is a simple point, off-the-cuff estimation is one of the most common errors that project teams make (Lederer and Prasad 1992, Jørgensen 1997, Kitchenham et al. 2002). Avoiding off-the-cuff estimates is one of the most important points in this book.

Tip #22

Don't give off-the-cuff estimates. Even a 15-minute estimate will be more accurate.

What if your boss calls on a cell phone and insists on getting an estimate right now? Consider your performance on the estimation quiz in Chapter 2, "How Good an Estimator Are You?" Did you get 8 to 10 answers correct? If not, what are the odds that the off-the-cuff answer you give your boss on the cell phone—even an estimate padded for uncertainty—will give you a 90% chance of including the correct answer?

^[1]MMRE is equal to AbsoluteValue [(ActualResult — EstimatedResult) / ActualResult].