9.2 Configuration Management as Supplier of Measurements

Configuration management is an outstanding source for measurements for metrics connected to other processes. Both metadata and change control data for configuration items may be used to extract information. It's therefore advisable, when defining metadata and change control data, to have a good grasp of the requirements from all processes regarding metrics. Likewise, it's important to define reports so these data can be presented as useful information. Those in charge of the processes are responsible for expressing their requirements. The person responsible for configuration management may, of course, also contribute suggestions.

Below are a few examples of how configuration management data may be the basis for the collection of measurements. Only the imagination and the usability set the limits for the measurements, but there is no reason to collect measurements if nobody is going to analyze them and act in accordance with the results.

Configuration management data may be used to collect or calculate measurements in the following categories: estimation, event frequencies, and effectiveness of quality assurance activities. The latter two may be made as specific as required in view of available data.

Estimation

Estimated resources for implementing changes may be part of the information on a change request. The number of change requests may itself give rise to estimates for implementation and approval of changes. To improve accuracy, estimates should be compared with actual data after the activities are completed.

Estimates may also, with reservations regarding their reliability, be used for activities in the company in general. In such cases, estimates for implementing and approving changes should be taken into accountsomething all too rarely done.

Event Frequencies

Events may be analyzed according to frequency, such as appearances over time or for specific items. To give an example, a person responsible for testing may calculate the frequency of already identified failures. This could enable prediction of the test course and possibly a decision about when to stop testing.

Somebody may also be interested in knowing the distribution of events, such as in relation to configuration item type, producer, or time of production. Do we create more faults on Mondays? Just after a long holiday? In large items or small ones?

Likewise, it may be interesting to analyze events in relation to who observed the event or who made and approved related changes. Is one user better at finding failures in the system? It might be interesting to discover why.

Effectiveness of Quality Assurance Activities

Quality assurance activities may be reviews, inspections, and various types of tests, such as module or system tests. The effectiveness of these activities may be calculated at any time as the relation between events found during each activity and the total number of events that could have been found.

To give an example: four faults are found during a module test. After three months in production, a defect is identified, resulting in the isolation of one more fault. This gives an effectiveness for this module test of 4/5, or 80%. It may be argued that it takes quite a while before effectiveness can be calculated, but this is nevertheless a metric much used in the testing field.

Ideas for Process Improvement

If analyzed correctly, all measurements may be used as inspiration for process improvement. The more precise the measurements, the better they are as a basis for process improvement. Measurements connected to clearly defined configuration items provide the best possibility for analysis. Furthermore, they may stimulate ideas as to how faults may be avoided in the future. Much literature is available on the definition of metrics and the collection and analysis of measurements. CMMI gives many examples to determine the status of activities for processes defined in the model.