4.3 SOME BASIC PRINCIPLES

< Day Day Up >

Given that cost estimation is important from a business viewpoint then we need to consider how effective cost estimation can be obtained. To do this we need to consider, as a first step or starting point, some basic rules or principles of estimation. Most of these are common sense but I make no apology for stating them here as they are so often forgotten or ignored. Most of these principles are also generally accepted within the world of cost estimation. That is to say that if you attend a conference on cost estimation or one of the user groups that exist you will hear the same points made there as I will make here. Notice the use of the word "most" two sentences back. I have taken the liberty of sliding one or two of my own "principles," based on experience, in with the others. Those of you who have been to cost estimation conferences can have fun spotting the interlopers.

The first point I would like to make is more a statement of fact than a principle. It is simply to point out that life is a harsh but fair task mistress. If you give nothing you should expect nothing. So why is it that many of us expect to be able to use some magic approach that enables accurate cost estimation at the very start of the initiation stage when we have practically no information or understanding of the work to be done?

Imagine a similar situation in another environment. "Good morning sir," says the friendly travel agent, "and what can I do for you?"

At this point you should imagine a John Cleese or Peter Sellers character. "Oh hello. I would like to know how much it would cost for a two week, foreign holiday please."

"Of course sir, let me just check the price chart, yes that would be three hundred and twenty two pounds."

"Guaranteed price?" which is simply the equivalent of fixed price.

"Of course sir."

Away goes our customer wondering how this agent can manage to send him to Hong Kong, Thailand, Bali and Australia all for three hundred pounds!

Now we all know that this situation is ludicrous but it happens every day in the IT world. In fact you can almost guarantee that, as you read this, someone, somewhere is making a promise to deliver on a system requirement that they know very little about.

Of course, there is an easy answer to this problem. Organizations that initially commit only to a requirements definition activity, often at fixed price, and who, as the final part of this work, carry out an estimation process that leads to a formal quotation for doing the actual implementation often find that they operate more effectively. It is easy to understand why. Requirements definition can be managed effectively on a fixed price basis because it can be constrained within a set of identified viewpoints. The implementation estimate is then only performed when the requirement is well understood. Is it strange that this leads to more accurate estimates?

There are two points to bear in mind about this approach. First, it does not guarantee success or accuracy. Requirements can change, things can go wrong on the project or you may simply come up with the wrong estimate. The chances of getting it wrong, however, are reduced.

Second, you can find a great deal of resistance to this approach. Interestingly this resistance tends to come from the IT professionals rather than the customers. Customers quite like the idea of getting things when you said they would!

Of course, this approach is not ideal for every environment. In a fixed price, competitive scenario you may have to bid for the whole job. If you are close to the prospective customer though, you could ask them why they were doing it in this way. You may get an interesting reply like, "because that's the way we've always done it, but now you come to mention it..."

The next point concerns the presentation of estimates and again impinges upon the maturity of our industry. An estimate is an estimate, not an actual. Obviously this is true but what does it imply? The implication is that an estimate will contain a degree of error. As such, estimates should not be presented as absolute figures. Now, we all know that when a project manager presents a cost estimate as, for example, twelve engineering months, what is actually being said is that the most likely estimate for this work is twelve engineering months give or take x percent. It is the x that causes the problems.

The project manager may be thinking in terms of give or take three engineering months but the more senior manager, or worse the customer, may be thinking in terms of give or take one month, or no give!

A cornerstone of our industry is communication. We should take our own medicine and start to communicate estimates clearly as bounded intervals. We should also realize that such intervals can be severely skewed. For instance an estimate that is presented as, "this work will cost at least six engineering months effort but the most likely estimate is eight, mind you we can definitely do within fourteen."

"Note that because of various risks associated with this project, consideration of the worst-case scenario produces an estimate of fourteen engineering months" should be perfectly acceptable. I was amazed to find that this kind of bid estimate is even being adopted in one fixed price arena. The idea is that the customer and the supplier communicate more honestly and share benefits if the project is delivered early, while sharing risk and cost if the project is delivered late, up to some predetermined limit. Much as I would like to, I cannot name the organizations involved because the supplier sees this as a major competitive advantage. I wonder why?

Again, you may find resistance to this idea of presenting estimates as bounded intervals but, yet again, it is only a recognition and application of common sense.

The next point concerns what should be estimated. Size and cost are valid candidates for estimation. Duration, which is often estimated, should, in fact, be planned. Size is driven by the functional requirement and cost is driven by the functional requirement, possibly expressed in terms of size, and the development environment including the productivity rate that applies within that environment. Duration is very heavily influenced by the organizational environment. Duration depends on so many factors that are outside of the functional requirement that it can rarely be estimated effectively. This is one of the most common complaints from estimators. Even teams that are very good at estimating cost or size find it difficult to estimate duration — which in many ways makes sense.

If I size a decorating job up as having to paint four walls and I know that I can paint two walls in three hours, effectively an evenings work, am I right to say that the duration of that job will be six hours or two evenings? This is true only if I use the term "duration" to mean cost, as many planners do. To my mind, duration is more to do with calendar time than with cost. For this decorating job, I may plan to do two walls on Monday evening but then I am away from home Tuesday, I am out on Wednesday and there is a film I want to see on Thursday. If I plan to finish the job on Friday then the duration of the decorating project will be five days. This is because I have planned it that way, not because of an estimate. This is certainly how my wife sees it when she, as the customer in this case, finds that she cannot use that room for five days!

The same is true of a software development or enhancement project. I may estimate that it will take nine engineering months, with bounds of course, but the duration it will take depends upon its priority, how I ramp up staff, who I put onto the project and many other factors.

This may be one reason why cost estimation tools tend to produce acceptable cost estimates, after calibration, but tend to perform less effectively for duration estimates.

Next I would like to consider our view of the estimates themselves. How often have you come across a project that is missing milestones or about which there are serious concerns, only to find that they are still working with the very first set of estimates that were made? Estimates should be seen as dynamic entities and they should be reassessed whenever a significant milestone is completed, in other words when we have gained significantly more information or when the requirements we are seeking to satisfy change.

Yet again this is simply the application of common sense. For most projects the estimates that the project team are working to should be reassessed at the end of the design and implementation or coding stages as a minimum. This assumes that you used the approach of estimating development after you did the requirements definition. For short term projects of up to six months duration it is probably more practical to assess the estimates at the end of each month because of the degree of phase overlap common on smaller projects, re-estimating as necessary.

Long term projects of more than two years duration, (from initiation to first delivery), will probably have a greater number of milestones identified. Typically these will include the end of requirements specification, high level, intermediate and low level design together with delivery to the test group of major functional items or subsystems. Estimates should be reassessed at each of these points.

These are simply ideas about suitable points in the development lifecycle at which re-estimation makes sense. The main point to understand is that re-estimation is not only acceptable but it should be seen as desirable because you will know a great deal more about the project and the work involved at the end of the design stage than you did at the project's initiation. When you re-estimate will depend upon your own environment and development methodology, but you should ensure that you have clearly defined points within the project plan at which re-estimation will occur.

The other trigger that should cause you to re-estimate is when the requirements change significantly. How big a change is "significant" is a moot point but most project managers know when this type of volatility is going to cause them problems.

If you are using a sizing technique like Function Point Analysis or even estimated Lines of Code then I would suggest, as a rule of thumb, that a ten percent change in the size of the project as a result of requirements change should trigger the re-estimation process.

Incidentally, have you noticed how requirements changes always make the project grow rather than shrink? Remember, one of the aims of re-estimation is to ensure that probable delays are identified and discussed with the customer early so that they do not come as a total shock. At the very least, the project manager should be aware of these potential problems and not have to report a six month delay, one week prior to delivery.

Now what about the estimators themselves? How do they fit in to the estimation process? I will have more to say about this later but, for now, I would like to make two points. First, the techniques that are made available to estimators, such as cost models, are there to assist the estimator. They do not remove the responsibility for the estimates from the estimator.

Second, estimators should use the techniques in a sensible way. If two or more techniques are applicable at a given point in the development lifecycle then two or more techniques should be used. There is a very simple reason for this. Estimation is always going to be something of an art and, at the end of the day, it relies upon the judgment of one or more individuals. To assist in this judgmental activity the maximum amount of information should be made available. As each technique could provide more information (or at least a different view of the same information), it makes sense to use as many techniques as is practical. At the very least, an estimator should not rely upon one technique to produce an estimate.

You should also be aware of an interesting phenomena in the area we are discussing. When it comes to making an estimate people tend to produce optimistic predictions. I am tempted to say that people always underestimate but there is one case where this is not true, namely when estimating productivity. Of course, this does not apply only to software projects.

My wife has the endearing habit of always expecting me to finish a DIY job in half the time I estimate it will take, and I always underestimate anyway, at least as far as DIY is concerned!

You also come across this when you ask people how long it will take to get from one place to another. An ex-colleague of mine, when asked how long it would take him to get from his house to an airport, quoted a time that would require a mean speed of one hundred and fifty five miles an hour. But as he said, he was not going to be traveling during the rush hour! Another ex- colleague arranged to meet me at the end of a meeting that "...would take no more than two hours so I will see you at 2pm." At four thirty I was still waiting for him to finish that meeting. And I think he put his finger on it when he did finally emerge, "yes I remember now," he said, "we only ever actually finished at two o'clock on one occasion."

People tend to remember good times or successes and try to ignore memories of failures or problems. This is why, when you ask someone to estimate, say, the cost of a job they will give you an answer based on the best case scenario. The organization that asks four project managers for an estimate then doubles or triples the average result would probably be most likely to survive in a fixed price environment! It is worth remembering that this phenomena exists next time you have to accept or produce an estimate.

< Day Day Up >