Why Does Fit Need a Criterion?

When you have a requirement for the product to carry out some function or to have some property, the testing activity must demonstrate the product does, indeed, perform that function or possess the desired property. To carry out such tests, the requirement must have a benchmark such that the testers can compare the delivered product with the original requirement. The benchmark is the fit criteriona quantification of the product that demonstrates the standard the product must reach.

"The idea is for each requirement to have a quality measure that makes it possible to divide all solutions to the requirement into two classes: those for which we agree that they fit the requirement and those for which we agree that they do not fit the requirement."

Source: Christopher Alexander, Notes on the Synthesis of Form

You should also consider the builders of the product. It stands to reason that once they know the criterion for the product's acceptance, they will build to that standard. If they are told their product will be used underwater, and the acceptance criterion is that the product must operate for as long as 24 hours at a depth of 15 meters, then they are unlikely to build the product out of anything except waterproof materials (see Figure 9.1).

Figure 9.1.

If they know the performance criterion the product must meet before it is accepted, the builders will naturally build a product to meet that criterion.

Possibly the hardest part of testing a requirement against an agreed-upon measurement is defining the appropriate measurement for the requirement. If your stakeholder asks you for a product that is "nice," then you must find some way of measuring its niceness. This measurement must, of course, be agreed upon with your stakeholder. You really have no idea what he means by "nice." But once you and your stakeholder reach agreement on how to measure it, the builders can construct the right product, and the testers can demonstrate that it is the right productit fits the requirement.

So how do you find a measurement for "nice"? Suppose that after some interrogation, your stakeholder tells you that as far as he is concerned, "a nice product is liked by the staff members intended to use it." A little more probing reveals that "liked by the staff members" means they take to the product instinctively, and don't hesitate before using it.

This explanation gives us something to measure: the duration of hesitation before using the product, the time elapsed before complete adoption of it, or the user satisfaction level after a period of usage (the last criterion can be measured by staff survey, for example). Of course, you have to reach agreement with your client that your proposed measurement does, in fact, measure the sought-after quality. However, once you do reach an agreement, you have a satisfactory fit criterion for a "nice product." If your stakeholder had a different meaning for "nice," then you would simply derive a different fit criterion.

Keep in mind that any requirement can be measured. All you have to do is find a suitable scale.