The Concept of Measurement

Why do we measure? We measure primarily to gain control of a project and therefore to manage it. We measure to evaluate how close or far we are from the plan's objectives in terms of completion, quality, and compliance with requirements. We also measure so that we can plan better a new project's effort, cost, and quality based on experience. Finally, we measure to evaluate the effects of changes and assess how we have improved over time the key aspects of the process's performance (see Chapter 17).

Measuring key aspects of a project adds a non-negligible cost, so we do not measure something simply because we can. We must set precise goals for a measurement effort and collect only measurements that allow us to satisfy these goals. There are two kinds of goals: ^[4]

^[4] Kevin Pulford, Annie Kuntzmann-Combelles, and Stephen Shirlaw, A Quantitative Approach to Software Management ”The ami Handbook . Reading, MA: Addison-Wesley, 1995.

• Knowledge goals

  These goals are expressed by the use of verbs such as evaluate, predict, and monitor. They express a desire to understand your development process better. For example, you may want to assess product quality, obtain data to predict testing effort, or monitor test coverage or requirements changes.

• Change or achievement goals

  These goals are expressed by the use of verbs such as increase, reduce, improve, and achieve. They express an interest in seeing how things change or improve over time, from one iteration to another, and from one project to another.

The following are examples of goals that might be set in a software development effort:

• Monitor progress relative to the plan.

• Improve customer satisfaction.

• Improve productivity.

• Improve predictability.

• Increase reuse.

These general management goals do not translate readily into measurements. We must translate them into subgoals (or action goals), which identify the actions that project members must take to achieve the goal. We also must make sure that the people involved understand the benefits.

For example, the goal "Improve customer satisfaction" would break down into the following action goals:

• Define customer satisfaction.

• Measure customer satisfaction over several releases.

• Verify that satisfaction improves .

The goal "Improve productivity" would include these subgoals:

• Measure effort.

• Measure progress.

• Calculate productivity over several iterations or projects.

• Compare the results.

Some of the subgoals (but not all of them) would require that you collect measurements. For example, "Measure customer satisfaction" can be derived from the following:

• A customer survey (in which the customers would give marks for different support aspects)

• The number and severity of calls to a customer-support hotline

What Is a Measurement?

There are two kinds of measurement:

1. A measure is a concrete numeric attribute of an entity (e.g., a number, a percentage, a ratio). For example, project effort expressed in person-months is one measure of a project's size. To calculate this measure you would need to sum all the time-sheet bookings for the project.

2. A primitive measure is an item of raw data that is used to calculate a measure. In the preceding example, the time-sheet bookings are the primitive measures. A primitive measure typically exists in a database but is not interpreted in isolation.

Each measure comprises one or more collected measurements. Consequently, each primitive measure must be clearly identified, and its collection procedure must be defined.

Measures to support change or achievement goals are often first-derivative over time (or iterations or project). We are interested in a trend and not in the absolute value of the measure. If our goal is "Improve quality," we must check that the residual level of known defects diminishes over time.