Causes of Estimation Error

As revealed by Jones (1994) and many other experts, both IT and business project teams have an extremely poor track record in the area of estimation. However, unlike many other experts, we believe that IT people, in particular, are very good at estimating. Other factors just confuse the estimation process. In fact, it is our experience that the factors covered in the following sections are the most significant causes of estimation error.

Misestimating the Scope

This is such an obvious problem that it is amazing it still is a problem in business and IT projects. As shown in Figure 14.2, Barry Boehm's (1989) research shows that the average estimation error in estimates made before the project's scope and objectives are clarified is +400% to -400%. Yet many IT and business experts are constantly expected to provide estimates at the early stages of projects before any detailed understanding of the project's scope and objectives is gained .

Figure 14.2. Project estimation points

Misestimating the Stakeholders' Effort

This has already been covered throughout this book when examining the role of stakeholders in projects. The process of estimation involves you making assumptions about your critical stakeholders' commitment (time and effort). For example, you may have gained an estimate and commitment from another project manager of five days to modify an interrelated system from which your project is going to get vital data. However, the related project manager is pressured by another stakeholder to deliver a different change. As a result, the system interface takes 15 days instead of five days. Your project is 10 days behind schedule, your estimates look bad, and you look bad.

Most estimation errors occur before you estimate.

Misunderstanding the Quality

This is one of the major estimation errors made in projects. Without the negotiation of a formal quality agreement (Chapter 10), you are making assumptions that have a serious impact on your estimates. For example, in making your estimates, you have assumed that your critical stakeholders do not require full system documentation. However, halfway through your project, you get a new requirement that full system documentation, including online tutorials, is required. Your original estimate is now wrong by at least 100%.

Miscalculating the Project Risk

Miscalculating or missing a major high-risk factor can result is massive estimation errors. As we discussed in Chapter 12, project risk assessment examines over 70 factors that can cause problems in your project and contribute to estimation errors. In developing your original estimate, you and your team assume that you will have access to a room in which you can all work together. However, that room is not available and your team is split across two building levels. You can add more than 30% to your original estimates.

Estimation Is Politics

The process of estimation is also heavily influenced by a series of well- understood and private political "games" such as The Low Bid, where a project manager or sponsor believes that the organization will not be prepared to fund the project. The Low Bid game involves a deliberate lowering of the estimates to get the project approved and started. The whole game works when the organization is not prepared to stop the project once the real costs are revealed. This and many other estimation games are discussed on our Web site (www.Thomsett.com.au).

Forgetting Tasks

We covered this in Chapter 13. Forgetting a task that will take 40 days of elapsed time causes 40 days of estimation error when you discover the task ( generally at the last moment).

Misunderstanding Your People

Extensive research by Jones (1994), Boehm (1989), and others has shown conclusively that the skills and experience of your team and your stakeholders are among the biggest factors in productivity and estimates of productivity. When making your initial estimates, you've assumed that you will have a skilled business analyst with at least two years ' experience in the business area that your project is focused on. Instead, you are allocated a new recruit straight out of college. Your estimates are now seriously wrong.

In effect, as shown in Figure 14.3, these factors can be seen as a series of filters that clean up your estimates as they are applied. In effect, as you are undertaking the RAP process described in this book, you're improving your estimates as a by-product.

Figure 14.3. The funnel of increasing accuracy

Estimation Overview and Principles

Before we examine the estimation process, we need to do some linguistic housecleaning.

Getting Our Language Right

Whenever the term estimate ^[2] is used in business and IT projects, we are using it to represent one of three completely different concepts, as described next .

^[2] Other alternatives to the word estimate include "gut feel," "wild ass guess," "wet finger in the air," "making it up," and so on.

Guess

A guess is an uninformed prediction. For example, ask 20 people "How many jelly beans are in the world now?" The answers will vary dramatically because people don't know ”they are guessing. Statisticians will tell you that when guessing is happening, the more people you ask, the more random the answers become! When you ask your team "How long will Task A take?" and the answers are 10 days, 12 weeks, and one year, your team is guessing.

Guesstimate

A guesstimate is an informed and expert prediction based on informal or undocumented experience. This is the most common form of estimation in IT and business projects. It is also called expert judgment. Although purists dismiss guesstimation as a poor alternative, our experience is that guesstimation is more accurate than the standard estimation techniques (provided the issues covered in this chapter are addressed). The technique of wide- band Delphi covered in this chapter is a powerful guesstimation technique suitable for all business and IT projects. With relevant experts, when you ask, "How long will Task A take?" and the answers are 10 days, 12 days, and 20 days, your team is guesstimating.

Estimate

An estimate is an informed and expert prediction based on formal or documented experience, measures, or metrics. For engineering, IT, and construction projects there are a number of estimating techniques based on either static (simple variables and total effort results) or dynamic (complex variables and effort/duration results) calculations. In IT, the widely adopted function point technique (and its variations) is an example of a static technique. Barry Boehm's COCOMO 1 and 2 is an example of a dynamic technique. The problems with these estimation techniques are that they require organization-specific metrics and they are not suitable for business and infrastructure projects. In addition, these techniques are more suited for conventional software projects, not e-projects. Most important, they address only the technology-specific components of the business project.

Project Estimation Points

The estimation process is an incremental one. In all industries it is understood that the accuracy of estimates improves as the project progresses. As shown in Figure 14.2, the accuracy of IT and business estimates can be dramatically improved during the initial project development phases. It should be noted that, whereas in other industries such as building and construction the accuracy of estimates at the concept/feasibility study phase can be on the order of plus or minus 50%, ^[3] extensive research by experts such as Barry Boehm (1989) has shown that larger estimate variances are experienced in IT projects.

^[3] It should also be noted that for high-risk building projects such as the Sydney Opera House and the New Australian Parliament House, the building industry experiences estimation errors of +500% and greater.

It is important to recognize that the estimates you make early in the system development life cycle are probably incorrect and, by using sensitivity analysis (discussed later), project sponsors, and steering committee members should gain some idea of the potential range of estimation accuracy.