13.4 Institutionalizing Measurement Process Improvement

Once a commitment has been made to build a good software development measurement system, the initial focus should be on building an effective measurement process. Around this measurement process we will systematically add new measurement tools, enhance the software measurement database, build a larger SQL framework to add value to the data that are constantly flowing into the database, and increase the exposure of the development staff to the contents of the measurement system through a training program.

There must next be a regular process created to improve the measurement process itself. The purpose of this process improvement process is to review and analyze the measurement process. The criterion measures for the measurement process are that the measurement process must:

Be transparent
Be timely
Be cost effective
Produce accurate and precise data
Produce useful information
- For project planning purposes
- For managing the software development process
- For future project planning

Once we understand just precisely how we are going to use the information from the measurement process, then we can institute a process to ensure that its objectives are met.

Perhaps the best way to conceptualize the measurement process improvement process is for us to think of the improvement occurring in a set of discrete steps. We will think of each of these potential improvement steps as an experiment. First we will formulate a hypothesis that suggests that our understanding of a single criterion measure would be enhanced if we were to make a change to the measurement process. That, after all, is why we are measuring things in the first place. We do not wish to make any change to the process that will not somehow make it better. We will not be able to make this assertion unless we have a controlled experiment to show the effect of change. We definitely want to steer clear of the Hawthorn effect. The essence of the Hawthorn effect is that change activity of just about any type will induce a short-term positive effect on the process that is being altered. Thereafter, the process will return to normal.

As a side note in the institutionalization of the change process, it is worth noting that the best source of potential improvements in the process will come from the people who are using the information provided by the measurement database. Far too often, experts are brought in to suggest changes to a process with which they are unfamiliar. The real experts are the consumers of the information.

The second step in this process will subject the experimental design to a critical review process of a team of people familiar with sound experimental design principles, software measurement processes in general, and the target software process. The purpose of this panel review process is to ensure the scientific and commercial integrity of the proposed changes to the measurement system.

The third step is to institute the proposed process change in a controlled environment. This will serve as a good pilot project for instituting the change on a more massive scale should the process change prove advantageous. Data collection during this trial phase will be subject to the rigors of a scientific experiment. Data will be collected for the new and improved process as well as the old and unimproved measurement process.

Finally, the process improvement must demonstrate significantly better understanding of the selected process criterion measure. If, for example, we wanted a more cost-effective measurement system, then a successful experimental outcome would be that the new measurement process would produce the same flow of information at reduced expense.