Chapter 15: Use of Multiple Approaches

Overview

Applicability of Techniques in This Chapter
	Use of Multiple Estimation Approaches
What's estimated	Size, Effort, Schedule, Features
Size of project	S M L
Development stage	Early-Late
Iterative or sequential	Both
Accuracy possible	High

No single estimation technique is perfect, so using multiple approaches is useful in many contexts. The most sophisticated commercial software producers tend to use at least three different estimating approaches and then look for convergence or spread among their estimates. Convergence among the estimates tells you that you probably have a good estimate. Spread tells you that there are probably factors you have overlooked and need to understand better. This technique applies equally to estimates of size, effort, schedule, and features.

My first personal exposure to this idea was in creating the estimate for the first edition of my book Code Complete (McConnell 1993). I had spent about 2 years doing background research for the book, writing prototype chapters, and preparing in other ways to write the book. Throughout this 2-year period, I had the idea that I was writing a 250 to 300-page book. That idea didn't come from any analytical exercise—it was just a length that I had gotten stuck in my head.

Since I hadn't published a book before, I thought I should present a proposal to the publisher that made it look like I might be capable of actually finishing the book. So as I was nearing the completion of my proposal for the book, I created my first estimate by decomposition. I went through the detailed outline I'd planned for the book and estimated the length of each section individually. Table 15-1 shows what that estimate looked like.

Table 15-1: Estimated Draft Pages in Code Complete Using Expert Judgment with Decomposition

Chapter	Estimate #1: Original Whole-Book "Gut Feel" Estimate	Estimate #2: Expert Judgment with Decomposition
Preface	-	4
Welcome	-	5
Metaphors	-	11
Prerequisites	-	52
…	…	…
Character	-	20
Review of themes	-	20
TOTAL	250–300	802

Up to this point, I had essentially used two estimation techniques: gut instinct, which led to a 250 to 300-page estimate, and expert judgment with decomposition, which lead to an 802-page estimate. There was enough spread between these two estimates that I needed to understand why the estimates differed so much.

I was attached to my 250-page preconception of the book's length, so I thought, "That 802-page estimate can't possibly be right. I must have made an error in my estimate." I decided that I would reestimate the book a second time and get the "correct" estimate.

For the third estimate, I took the number of pages in each of the prototype chapters I'd written and I divided those page counts by the number of points in the outlines for those chapters. I had a ratio of 1.64 pages per outline point. I then went through my detailed outline for the whole book and counted the number of outline points per chapter. I multiplied those by 1.64. Table 15-2 shows the estimate obtained by using this method.

Table 15-2: Estimated Draft Pages in Code Complete Using Outline Points and Historical Data

Chapter	Estimate #1: Original Whole-Book "Gut Feel" Estimate	Estimate #2: Expert Judgment with Decomposition	Estimate #3: Outline Points and Historical Data
Preface	-	4	4
Welcome	-	5	5
Metaphors	-	11	11
Prerequisites	-	52	52
…	…	…	…
Life-cycle models	-	20	16
Review of themes	-	20	21
TOTAL	250–300	802	759

The third estimate of 759 pages was within 5% of the second estimate of 802 pages. Because of the convergence of those two estimates, I had a pretty clear picture that I wasn't writing a 250 to 300-page book, as I had thought for 2 years. I was writing a 750 to 800-page book.

My experience was representative of a more general finding: Estimation accuracy improves when results from multiple estimators or results from multiple estimation techniques are combined (Jørgensen 2002, Tockey 2004).

Tip #66 

Use multiple estimation techniques, and look for convergence or spread among the results.

The software parallels of my book-related estimating are straightforward. People form ideas about possible project costs, durations, and features that are based on nothing in particular. They will keep those preconceived ideas until someone presents them with enough data to dislodge their preconceptions. Without data, I would not have believed that the scope of my project was three times as large as I had thought. With a little bit of data, I initially still needed more data before I was convinced.

Another parallel to software is that it's better to get bad news early than late. I adjusted my expectations about the size of my project at proposal time. I could have chosen to stop at that time. But I looked at the project scope and decided it was worth doing anyway, which gave me a more realistic view of the schedule I was planning and the commitment I was making.

The fact that two completely different approaches had produced similar estimates increased my confidence in those estimates. In software, be sure to use different kinds of estimation techniques to create your different estimates. For example, you might use estimation by proxy, expert judgment, estimation by analogy, and a software estimation tool.

Once I accepted the convergence of the estimates, I was suddenly able to see how other data confirmed that ballpark size as well. One of my prototype chapters was 72 pages long. That would have made up 29% of a 250-page book. I never believed that I was 29% done with the project just because I had completed that prototype chapter. Indeed, at the gut-instinct level I knew that chapter was no more than about 10% of the book. After my eyes were opened by the two convergent estimates, I realized that the length of the prototype confirmed that the real scope of the book was in the 750 to 800-page range, not the 250 to 300-page range.

Project-specific data usually provides the most accurate estimate. I ended up with 749 draft pages, which was only 10 pages (1%) different from the outline points estimate that had been created using historical data from the same project.

My original gut-instinct estimate was based on looking at other books in the 250 to 300-page range. Because I hadn't written a book before, I made what turned out to be a naïve assumption that I would need to write 250 to 300 pages to end up with a 250 to 300-page book. But page count from manuscript to published book expands because of blank pages between chapters, blank pages between book parts, the table of contents, perhaps a list of figures, the index, and other front and back matter. Page count is also affected by the choice of typeface, margins, line leading, and so on. All these elements are obvious once someone points them out, but as a rookie author it was easy for me to forget to account for them in my estimate. Even in my decomposed estimate I committed the classic estimating mistake of doing a good job of estimating the things I knew about but forgetting to estimate certain significant parts of the project.

Finally, my second two estimates converged to within about 5%. I didn't know it at the time, but that turns out to be a good target for convergence in general. (If you're in the wide part of the Cone of Uncertainty, you will sometimes need to settle for less convergence.).

Tip #67 

If different estimation techniques produce different results, try to find the factors that are making the results different. Continue reestimating until the different techniques produce results that converge to within about 5%.

In a more software-specific context, I was later asked to estimate a project for one of my clients. The crosses in Figure 15-1 show the individual estimates I created for that project. The size of each cross represents the confidence I had in each estimate. The triangle shows my "most accurate estimate" of 75 staff months and 12 calendar months. Although the crosses look somewhat dispersed, the estimates in which I had the most confidence all converged to within 5% of the "most-accurate estimate." The square shows my client's business target of 25 staff months and 5 calendar months. In this particular case, the client chose to proceed on the basis of the business target of 5 calendar months and 25 staff months. This was unfortunate. The project was ultimately delivered after 14 calendar months and about 80 staff months' effort, and the team delivered far less functionality than originally planned.

Figure 15-1: An example of multiple estimates for a software project.

Tip #68 

If multiple estimates agree and the business target disagrees, trust the estimates.