Software Metrics: Best Practices for Successful IT Management

---

4.12 ESTIMATION BY ANALOGY

The last two techniques I want to identify as potential candidates for a cost estimation strategy can be applied at any time during the development lifecycle. The first is estimation by analogy.

If you ask a group of software developers how they estimate, this will be the technique most will claim to use, except the honest ones who usually answer by sticking a wet finger in the air. Estimation by analogy is a simple approach that asks you to look at your project in terms of what it is expected to deliver, what is the function of the product, and how you expect to achieve that, not in design terms but in terms of the development environment. Then you simply find a project that has already been done that is similar to yours and you use data from the completed project to help you estimate for yours.

For most organizations it is reasonably easy to find a project that was similar in terms of functionality. It is also very easy to forget about the soft factors or the development environment. It is no good finding a perfect functional match where the completed project was done using a top flight team using tried and tested techniques and then expecting to deliver in the same time and at the same cost using a team consisting of mainly new graduates using tools never employed before.

Now while these concepts are simple there is one major sticking point as far as estimation by analogy is concerned. The technique depends upon having an accessible history file of projects. But, most organizations, as I have said earlier, fail to keep much data on current projects, let alone past work. Given this situation you should concentrate on establishing that history file. It does not need to be complex. Record data such as size, actual effort, actual duration, type of project through a simple classification scheme, team size and composition. You can add other elements to your records such as effort per phase, but remember, it is better to get something simple going than to have a detailed procedure that is never used.

Also remember that you will seldom find a perfect match for your new project so again expert opinion is needed to guide judgment. In a maintenance environment where new releases occur regularly, building up a usable history file does not take long.

The final component I wish to discuss are the generalized cost models and the associated tools that were discussed earlier. The tools can be used at any time during the development lifecycle provided that you have a size estimate with which to drive them. This estimate can come from any of the other techniques described so far and, in this way, the tools can provide a good confidence check provided that they have been correctly calibrated.

The other great benefit that can come from using these types of tool is that they can get project managers thinking about the different factors that can affect projects. One organization, a large UK clearing bank, has integrated an estimation tool with Delphi sessions using the tool to stimulate discussion with interesting and beneficial results.

Figure 4.1 shows how the various techniques fit against a typical development lifecycle. There is overlap and it is quite deliberate as you will realize from our earlier discussions regarding the basic principles of estimation.

Figure 4.1: Estimation Template

So, we have a variety of bricks but we really need to put them firmly within a template by considering the channels of communication between bricks and the development lifecycle and the management of that lifecycle. Figure 4.1 presents one such view and is offered as an estimation template that could be tailored to your organization.

This diagram shows a set of bricks or estimating techniques associated with certain lifecycle phases, for just the "development" arm of the V but not for testing, together with a link from the project manager to the estimation support group. Notice in the expansion of the estimation support group's role in Figure 4.2, there is a commitment to training, checking and to the development of local models. This model is presented as a strategy for a typical software engineering organization of today, and will apply, I suspect, for some years. It is not presented as the final solution. We will only have fully solved the estimation process when organizations tie effective estimation to the project management process. This will take effort and time but will provide rich dividends in terms of customer satisfaction.

Figure 4.2: Role of the Support Group

Figure 4.3 shows a typical process diagram of a first-stage estimation process as implemented in a number of organizations. Notice the use of Function Point Analysis to size individual work requests and the final project. This two-stage sizing is used because economies of scale can sometimes be realized at the project level. FPA is also used to drive the negotiation with the customer, sometimes in conjunction with cost estimates against individual work requests.

Figure 4.3: Typical Process Diagram

Finally, there is one element of estimation that has not been discussed: how do we monitor the effectiveness of the process? In essence this is simply a comparison of estimates made at specific lifecycle points to actual results within projects. In practice this data needs to be collected, analyzed and presented to management to demonstrate an improvement over time. Again, this will not happen automatically and this responsibility should fall to the estimation support group from the very start.

I have included three suggestions that may help get you started as far as presenting this type of effectiveness measure is concerned. All three charts are simply based on collection of actuals versus estimates. The first, Figure 4.4, plots the estimate against the actual so that the distance from the 45-degree line indicates the level of estimate accuracy. A plot on the line means that the estimator got it spot on!

Figure 4.4: Monitoring and Feedback (Actual vs. Estimates)

Figure 4.5 shows four estimates taken during the course of two separate development projects. An actual line is included and the variance against the actual is the apparent. As this is based on real data, notice the way the estimation process has become more sensible by the time we are working on the third project. This should be compared against the wild variances experienced on the first project.

Figure 4.5: Monitoring and Feedback (Estimate vs. Effort)

Figure 4.6 takes the absolute variance, that is ignoring the fact that it may be an over or underestimate, between the estimate and the actual and plots this for a number of projects. In this case we are using the initial estimates and we have data from four projects that have used an estimation strategy. This type of diagram enables the benefits of such an approach to be represented to managers and to customers.

Figure 4.6: Monitoring and Feedback (Variation Actual Against Initial Estimate)

---