(Mis)Using Metrics Data

The best use of data such as that described in these sections is simply to provide insight into what is happening (or what happened) on a project. Using this data for future project cost and schedule prediction can be tricky and misleading. Be careful of hidden assumptions that the data represents. If you average the number of requirements over the amount of time and number of resources consumed, the result is a number that is applicable only to that specific project. Future projects will involve different customers, different project personnel, newer or different technologies, and a different environment, as well as a different application to be developed. Also, the amount of time needed to implement and test individual requirements varies by orders of magnitude. (It's like asking Boeing how many pounds of airplane were built in a given day!) Individually, the data has little meaning. However, taken in aggregate, the data is more meaningful.

When you use historical data such as this for project estimation, the data should be used as a starting point only. Then, you can make adjustments to account for all the aspects that will be different on the new project (and many things will be different!).

Exercise caution when using charts and metrics to justify making decisions or adjustments on a project. I am reminded of something a physician friend once told me: "Treat the patient, not the X-ray." Don't make changes on a project solely because a trend or metric doesn't indicate what you anticipated. In other words, the data shows you something is happening. You must investigate further to decide how to properly react to the data or whether to take action at all.