10.3 Hazards of Adding Up Best Case and Worst Case Estimates


10.3 Hazards of Adding Up Best Case and Worst Case Estimates

Have you ever had the following experience? You put together a detailed task list. You carefully estimate each of the tasks on the list, thinking, "We can pull this off if we try hard enough." After you go through meticulous planning, you work hard on the first task and deliver it on time. The second task turns up some unexpected problems, but you work late and get it done on schedule. The third task turns up a few more problems, and you leave it unfinished at the end of the day, thinking you'll polish it off the next morning. By the end of the next day, you've barely finished that task, and haven't yet started the task you were supposed to do that day. By the end of the week, you're more than a full task behind schedule.

How did that happen? Were your estimates wrong, or did you just not perform very well?

Warning: Math Ahead!

The answer lies in some of the statistical subtleties involved in combining individual estimates. Statistical subtleties? Yes, for better or worse, this is an area in which we must dig into the mathematics a little to understand how to avoid common problems associated with building up an estimate from decomposed task or feature estimates.

What Went Wrong?

To see what happened in the preceding scenario, let's take another look at the case study from the beginning of the chapter. The team in the case study produced an accurate estimate. But the accuracy of their single-point estimates was unusual. A more common attempt to produce an estimate by decomposition would not produce the estimates listed in Table 10-1; it would be much more likely to produce estimates such as those shown in Table 10-4.

Table 10-4: Example of More Typical, Error-Prone Attempt to Estimate by Decomposition

Feature

Estimated Staff Weeks to Complete

Actual Effort

Feature 1

1.6

3.0

Feature 2

1.8

2.5

Feature 3

2.0

1.5

Feature 4

0.8

2.5

Feature 5

3.8

4.5

Feature 6

3.8

4.5

Feature 7

2.2

3.0

Feature 8

0.8

1.5

Feature 9

1.6

2.5

Feature 10

1.6

3.5

TOTAL

20.0

29.0

In this example, the accuracy of the 20-staff-week estimate obtained through a simple summation of decomposed, single-point estimates is actually worse than the aggregate estimate of 22 staff weeks that you provided earlier in the case study. How can this be?

The root cause is a combination of the "90% confident" problem that that was discussed in Chapter 1 ("What Is an 'Estimate'?") and the optimism problem discussed in Chapter 4 ("Where Does Estimation Error Come From?"). When developers are asked to provide single-point estimates, they often unconsciously present Best Case estimates. Let's say that each of the individual Best Case estimates is 25% likely, meaning that you have only a 25% chance of doing as well or better than the estimate. The odds of delivering any individual task according to a Best Case estimate are not great: only 1 in 4 (25%). But the odds of delivering all the tasks are vanishingly small. To deliver both the first task and the second task on time, you have to beat 1 in 4 odds for the first task and 1 in 4 odds for the second task. Statistically, those odds are multiplied together, so the odds of completing both tasks on time is only 1 in 16. To complete all 10 tasks on time you have to multiply the 1/4s 10 times, which gives you odds of only about 1 in 1,000,000, or 0.000095%. The odds of 1 in 4 might not seem so bad at the individual task level, but the combined odds kill software schedules. The statistics of combining a set of Worst Case estimates work similarly.

These statistical anomalies are another reason to create Best Case, Worst Case, Most Likely Case, and Expected Case estimates, as described in Chapter 9, "Individual Expert Judgment." Table 10-5 shows how that might work out if the developers who produced the estimates in Table 10-4 were asked to produce Best Case, Worst Case, and Most Likely Case estimates, and if the Expected Case estimates were computed from those.

Table 10-5: Example of Estimation by Decomposition Using Best Case, Expected Case, and Worst Case Estimates

Weeks to Complete

Feature

Best Case (25% Likely)

Most Likely Case

Worst Case (75% Likely)

Expected Case (50% Likely)

Feature 1

1.6

2.0

3.0

2.10

Feature 2

1.8

2.5

4.0

2.63

Feature 3

2.0

3.0

4.2

3.03

Feature 4

0.8

1.2

1.6

1.20

Feature 5

3.8

4.5

5.2

4.50

Feature 6

3.8

5.0

6.0

4.97

Feature 7

2.2

2.4

3.4

2.53

Feature 8

0.8

1.2

2.2

1.30

Feature 9

1.6

2.5

3.0

2.43

Feature 10

1.6

4.0

6.0

3.93

TOTAL

20.0

28.3

38.6

28.62

As usual, it turns out that the developers' single-point estimates in Table 10-4 were actually their Best Case estimates.




Software Estimation. Demystifying the Black Art
Software Estimation: Demystifying the Black Art (Best Practices (Microsoft))
ISBN: 0735605351
EAN: 2147483647
Year: 2004
Pages: 212

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net