Fit Criteria for Nonfunctional Requirements

A nonfunctional requirement is a quality that the product must have, such as usability, look and feel, performance, and so on. The fit criterion is a measure of that quality.

Some of the nonfunctional requirements may at first seem difficult to quantify. Ultimately, however, it is possible to measure all of them. If you cannot quantify and measure a requirement, it is not really a requirement. It might be several requirements written as one, or it might be incomplete, ill considered, irrational, haphazard, or just plain not a requirement. Either you can quantify it or you delete it.

If you can't measure a requirement, it is not really a requirement.

Let's look at some examples.

Description: The product shall be user-friendly.

This requirement at first seems vague, ambiguous, and not at all amenable to measurement. However, you can find a way to measure it. Start with the client: Can he tell you more precisely what he means by "user-friendly"? This is where the rationale comes in. For example, is the product meant to be easy to learn, easy to use, attractive and inviting, or some other meaning of "user-friendly"?

Even vague, ambiguous requirements can be measured.

Suppose your client clarifies his intention toward "user-friendly" by saying, "I want my users to be able to learn how to use the product quickly." This rationale suggests a scale of measurementthe time taken to master given tasks. You can measure it by identifying a set of standard tasks and measuring the learning time needed before being able to complete them successfully. Alternatively, you can specify an amount of training time allowed before being able to use the product to a certain standard. Another measurement of how successful a user is at knowing the product could be the number of calls to a help desk or references to online help.

A suggested fit criterion for this "user-friendly" requirement is this:

Fit Criterion: New users shall be able to add a road, change a road, and delete a road within 30 minutes of their first attempt at using the product.

Note that this fit criterion measures this particular meaning of "user-friendly." It does not necessarily apply to all instances of the client asking for a "user-friendly" product.

Earlier in this chapter you saw an example of a requirement that specified a "nice" product; the client said that "nice" meant the staff liked it. You can measure "like": If the staff likes the product, they will use it. You can measure how quickly they start using it, how much they use it, or how soon word gets around that the product is good and users encourage one another to use it. All of these criteria quantify the client's desire that the staff like the product and use it (see Figure 9.2).

Figure 9.2.

You can measure your users' liking for the product by surveying their work practices before and after the product is introduced, by measuring how long it takes them to start using the product once it is available, or by surveying them after a period of use to ascertain their liking for the product.

You could write a fit criterion like this:

Fit Criterion: Within three months of introducing the product, 60 percent of the users shall be using it to carry out the agreed-upon work. From those users, the product shall receive a 75 percent or more approval rating.

Note how you clarify the requirement when you add the fit criterion. By negotiating a measurement, you transform the requirement from a vague and somewhat ambiguous intention into a fully formed, testable requirement. You will find that it is usually not possible to get the complete, measurable requirement in the first instance. It is unlikely that your stakeholders will express themselves in such precise terms. We suggest you go with the flow. Don't slow down your requirements-gathering processes to make the requirement measurable, but rather get the stakeholders' intentionyou write this as the descriptionand the rationale. Then analyze your understanding, write your own best interpretation of the fit criterion, and improve it by asking your stakeholder whether it is an accurate measurement of the requirement.

Product Failure?

The fit criterion might be determined by asking your stakeholder, "What would you consider a failure to meet this requirement?" Suppose you have this requirement:

Description: The product must produce the road de-icing schedule in an acceptable time.

Clearly, the scale of measurement here is time. Your client can tell you how much time he thinks would constitute a failure. For example, if the engineer has to wait for more than 15 seconds (or whatever) for the schedule, the client may consider the product unacceptable. Thus you have the following fit criterion, with suitable business tolerances applied:

Fit Criterion: The road de-icing schedule shall be available to the engineer within 15 seconds from when he makes his request for 90 percent of the times that it is produced. It shall never take longer than 20 seconds.

There will be times, however, when you discover there is no agreement on a quality measure, and hence there can be no fit criterion. In these circumstances, it is possible that the original requirement is actually several requirementseach requirement has its own measurementor the requirement is so vague, and its intention so unrealistic, that it is not possible to know whether it has been satisfied. For example, no fit criterion is possible for this requirement: "I want a product my grandmother would have liked had she been alive today."

Subjective Tests

Some requirements have to be tested using subjective tests. For example, if a cultural requirement for a product to be used in the public domain is "not offensive to any group," then the fit criterion must be along these lines:

Fit Criterion: The product shall not be offensive to 85 percent of a test panel representing the makeup of the people likely to come in contact with the product. No more than 10 percent of the interest groups represented in the panel shall feel offended.

The business tolerance here allows for the fact that you cannot count on 100 percent of humans passing any test. In this case the business tolerances shield the product from the fringe-lunatic extreme views, while at the same time allowing "offensive" to be measured.

Although fit criteria are measures of the product's performance, it is usually cost-effective to test prototypes built specifically for the purpose.

It is likely you would use a prototype or simulation, and not the delivered product itself, for this kind of testing. Although fit criteria are measures of the product's performance, it is usually cost-effective to test prototypes built specifically for the purpose.

Numbers used in fit criteria are not arbitrary. Suppose you have a fit criterion of "reduce the time to perform [some task] by 25 percent of the current time." This means that the current time must be known and documented, not just guessed at. The reason for the target of a 25 percent reduction must be well understood, and agreed to by the client. Ideally, the reasoning behind wanting 25 percentand not 20 percent or 30 percentis backed by empirical data taken from a study of the business.

Look and Feel Requirements

Look and feel requirements specify the spirit, mood, or style of the product's appearance and behavior.

Many companies require their products to be in company colors. The rationale for this requirement is either adherence to branding standards or a desire to enhance customer recognition. These two are slightly different, however.

Let's look at branding standards first. The product either conforms to the standards or it doesn't. Your organization has a person, or department, who are responsible for the branding standards. Thus the fit criterion should specify the target standard and state who or what is to certify the product's compliance.

Fit Criterion: The product shall be certified as complying with this year's corporate branding standards by the head of marketing.

Where the rationale is customer recognition, we suggest a fit criterion along these lines:

Fit criterion: Sixty percent of the target audience will recognize the product as belonging to the corporation within five seconds of encountering it for the first time.

Look and feel requirements may start out as "touchy-feely" statements of intent. However, by questioning, by repeatedly asking for a rationale, and by looking for measurable aspects, you will always find suitable fit criteria for them. Even something as vague as "cool" can be measured. How? By asking why it has to be cool, thereby finding the stakeholder's rationale for the requirement. Suppose the client says the product has to be "cool" to appeal to the target market of high school students. A fit criterion in this situation could be a rate of acceptance by a representative panel of high school students. Or perhaps it is the product selected by 75 percent of the panel when shown five sample prototypes. Or perhaps, when shown a simulation of the product, 85 percent of the representative high school panel say they would buy it.

Look and feel requirements may start out as "touchyfeely" statements of intent, but you will always find suitable fit criteria for them.

Usability and Humanity Requirements

Usability and humanity requirements specify the product's convenience and fitness for use by its intended users. Products are usually required to be easy to use, easy to learn, able to be used by certain types of users, and so on. To write the fit criterion for each of these requirements, you must find a measurement scale that quantifies the objective of the requirement.

Let's look at some examples.

Description: The product shall be intuitive.

To measure "intuitive," you must consider the people to whom the product must be intuitive. In the IceBreaker example, you are told the users/actors are the road engineers; they have an engineering degree and meteorological experience.

Rationale: The engineers must find it easy and intuitive; otherwise, they will not use it.

Now that you know this rationale, "intuitive" takes on a different meaning.

Fit Criterion: A road engineer shall be able to produce a correct de-icing forecast within ten minutes of encountering the product for the first time without reference to any out-of-product help.

Sometimes "intuitive" really means "easy to learn." In this case, you ask how much time can be spent in training, and the resulting fit criterion might look like this:

Fit Criterion: Nine out of ten road engineers shall be able to successfully complete [list of selected tasks] after one day's training.

Fit criteria for usability requirements might also quantify the time allowed for given tasks, the error rates allowed (quantifying ease of use), the satisfaction rating awarded by the users, ratings given by usability laboratories, and so on. Look for the real meaning of the requirement, and confirm it by having your stakeholders agree that your proposed fit criterion is the correct measurement of the meaning

It might be appropriate for your usability requirements to have a fit criterion based on readability. Several readability scoring systems are availability. For example, the Flesch Reading Ease Score assigns a rating based on several factors in the text. The highest score is 100, and a suggested score is in the 6070 range. The Flesch-Kincaid Grade Level Score rates text against the U.S. grade school levels of comprehension. For example, a score of 8.0 means that an eighth grader can understand the material. Suggested scores for a standard document are in the 7.08.0 range.

Look for the real meaning of the requirement, and confirm it by having your stakeholders agree that your proposed fit criterion is the correct measurement of the meaning.

Accessibility requirements specify how easy it should be for people with common disabilities to access the product. This could result in a fit criterion like this:

The product shall be certified to be in compliance with the Americans with Disabilities Act.

It may also be necessary to specify the relevant parts of the act and perhaps the body responsible for certifying compliance.

Performance Requirements

Performance requirements deal with the speed, accuracy, capacity, availability, reliability, scalability, and similar characteristics of the product. Most of the time, the nature of the performance requirements will suggest a measurement scale. Let us look at some examples.

Suppose you have this requirement:

Description: The response shall be fast enough to avoid interrupting the user's flow of thought.

The word "fast" indicates you should measure time. Here is a suggested fit criterion:

Fit Criterion: The response time shall be no more than 1.5 seconds for 95 percent of responses, and no more than 4 seconds for the remainder.

Similarly, the fit criterion for an availability requirement might be written as follows:

Fit Criterion: In the first three months of operation, the product shall be available for 98 percent of the time between 8 a.m. and 8 p.m.

A fit criterion may be shown as a range. This is particularly appropriate for performance requirements. For example:

Fit Criterion: The product shall allow for 3,000 downloads per hour, although 5,000 per hour is preferred.

The point of using a range is to deter the developers from constructing a product that could be overly expensive, and to make the best design trade-offs to fit the budget and design constraints.

As most performance requirements are themselves quantified, it should be fairly straightforward to write appropriate fit criteria. If the requirement is given to you in correctly quantified terms, then the fit criterion and the requirement are the same: Write one or the other.

If the requirement is given to you in correctly quantified terms, then the fit criterion and the requirement are the same.

Operational Requirements

Operational requirements specify the environment in which the product will operate. In some cases the product has to be used in adverse or unusual conditions. Recall our example from the IceBreaker product:

Description: The product shall be used in and around trucks at night, and during rainstorms, snow, and freezing conditions.

The fit criterion for an operational requirement is a quantification of the successful usage in the required environment. For the preceding (somewhat unusual) operational requirement, the fit criterion quantifies the enabling of the operator to achieve specified tasks and the ability of the product to with-stand the conditions. For example:

Fit Criterion: The operator shall successfully complete [list of tasks] within [time allowed] in a simulation of a five-year storm (this is an accepted quantification of meteorological conditions) and the product shall function correctly after 24 hours exposure.

Operational conditions may also specify that the product must coexist with partner, or collaborating, systems. The fit criterion in this case will cite the specification of the partner system or the way to communicate with the partner:

Fit Criterion: The interfaces to the Rosa Weather Station shall be certified as complying with the National Transportation Communication ITS Protocol (NTC/IP).

This criterion is testableby engineers from NTCand points the product's builders toward a known and accepted standard.

Maintainability Requirements

Maintainability requirements specify the expectations about the maintenance of the product. Usually the fit criteria for these requirements quantify the amount of time allowed to make certain changes. This is not to say that all maintenance changes can be anticipated, but where changes are expected, then it is possible to quantify the time allowed to adopt those changes.

Fit Criterion: New users shall be added to the system with no more than five minutes' interruption of normal service to existing users.

If you are creating a software product, and there is a requirement to port it to another computer, this is specified in the maintainability section. The fit criterion quantifies the amount of time, or effort, to satisfactorily port the software.

Security Requirements

Security requirements cover the security of the product's operation. The most obvious requirement in this category specifies who is allowed access to what parts of the product, and under what circumstances. The fit criterion could be something like this:

Description: Only engineers using category A logins shall be able to make any additions, updates, or deletions to any weather station data.

Fit Criterion: Of 1,000 additions, updates, or deletions to any weather station data, none shall be from other than category A engineer logins.

File integrity is part of security. The most common source of damage to computer files is authorized users accidentally corrupting the data. Thus at least one requirement should refer to file integrity. Its fit criterion would be something like this:

Fit Criterion: After any operation, the product's weather station data shall be identical to the temperature transmissions as recorded by the transmitting weather station.

This fit criterion says that the product's data must agree with the authority for that data. As most data is transmitted to the product from the outside in this case, from a weather stationthe transmitter must be the authority. Thus, if the product's data conforms to the authority's data, it is considered to be correct.

Cultural and Political Requirements

Cultural and political requirements, by their nature, are subjective and slightly more difficult to quantify. The fit criterion is usually based on who will certify the product's acceptability. The following examples of fit criteria should reveal their originating requirements:

Fit Criterion: The Shatnez Laboratory of Brooklyn shall certify that the product complies with the shatnez rules.^[1]

Fit Criterion: The ethics committee shall attest that the product displays no religious or political symbols, or words or symbols that could be construed as religious or political.

Fit Criterion: The marketing department shall certify that all components are American made.

^[1] We are indebted to Ethel Wittenberg for information on the Orthodox Jewish prohibition on wearing a garment made of a mixture of linen and wool.

Cultural requirements come about because different countries and different people have different customs, experiences, and outlooks. When you are specifying fit criteria for cultural requirements, remember that words used in some countries are unknown in others that speak the same language. Make sure that you are not making assumptions about how the fit criteria will be interpreted by defining your terms in section 5 of your requirements specification.

Legal Requirements

Legal requirements specify the conformance with laws. Here is a fit criterion that will apply to most legal requirements: Your client wins a court case brought by somebody who uses the product. However, fit criteria must be able to be tested in a cost-effective manner, and court cases are far too expensive to be indulged in lightly. Thus the majority of fit criteria will be along the following lines:

Fit Criterion: The legal department/company lawyers shall certify that the product complies with the [appropriate laws].

Legal requirements are also written to ensure that the product complies with cited standards. Most standards are written by organizations that either have people who certify compliance"standards lawyers"or issue guidelines as to how you can certify compliance for yourself. In either case, fit criteria can be written to specify how compliance with the standard is to be verified.