Dealing with Uncertainty in the Model

As noted previously, cells that are gray filled are ones where the values entered are highly subjective and vary depending on your company, industry, and so on. Here are some ideas for dealing with these uncertainties in the model.

When originally building this model, I found that a useful approach to using it was to talk with people who had been using the tool and have them provide values for these areas of uncertainty. In that way, they could see if the values they are willing to buy into resulted in a good benefit to cost ratio. This may not result in a definitive benefit to cost study, but it certainly is useful for getting grassroots acceptance of the tool (assuming that their values result in benefits greater than cost, which has always been the case when I've tried this).

A variation on this idea is to assemble a group in your company and apply a technique like Wideband Delphi to determine reasonable values for areas of uncertainty in the model. Wideband Delphi is a group problem-solving technique that is often applied to project schedule and effort estimation that allows a group to converge on an answer that is better than any individual would have come up with alone.^[12]

^[12] Wideband Delphi is explained in more detail in Chapter 1, "An Introduction to QFD: Driving Vision Vertically Through the Project."

Another common technique for dealing with uncertainties in numeric-based models is to build a Min and Max version of the model. These values could come, for example, from your interviews with the staff members who are using the tool. If you want to get a bit more sophisticated, you can even run the model through a tool like @Risk, a Monte Carlo simulation add-in tool for Excel. This provides a probability distribution function of the benefit to cost ratio: a report that tells you the likelihood of a range of possible benefit to cost ratios based on the uncertainties in your model. It may sound complicated, but it's actually very straightforward, and add-ins for Excel are relatively inexpensive.

Finally, it's always a good idea to run up estimates based on different models. Leffingwell (2003) provides an alternate model based on project cost and benefits due to reduction in requirements errors. It would be a good cross-check on the results of this model after both are calibrated to the values that make sense for your company and industry.