Use Cases as Sources of Test Cases

To test for conformance to requirements, we want to construct the test cases from the use cases that specify requirements. As we previously noted, we need to determine how many test cases to use from each use case and then build the test cases themselves. Although we discussed this in Chapter 4, we will provide additional details here.

Constructing Use Profiles

The construction of a use profile begins with the actors from the use case diagram. The actor profile in Figure 9.4 is for the player actor in the Brickles game. When there is a single actor, that profile should match the frequency field in the use cases. The interesting, and useful, case is when there are several actors. Seldom will all of these actors use the system in exactly the same way. The frequency field in the use case is a composite of the frequency values in the individual actor profiles.

Figure 9.4. An actor profile

In Figure 9.5, the frequency in Use Case #1 would be some type of average of the frequencies provided in the profiles of actors A and B. You might wish to weigh the frequency of actor A more heavily than that of actor B. The frequency field for each of the use cases is constructed from these actor profiles.

Figure 9.5. Actor to use case mapping

This technique is a very useful one for systems that have never been deployed. It is more accurate to estimate how each actor will use the system than to guess what the aggregate of individual uses will be. After deployment the use case frequencies can be updated based on actual data and used for regression testing.

Using Scenarios to Construct Test Cases

A use case typically contains multiple scenarios that could be converted to test cases. The use case section labeled The system responds by is the normal case and is the first source of test cases. There will be data attributes that should be analyzed in the same way as any parameter to a method. The Boundary Conditions sidebar on page 180 provides one type of analysis that can be used. Here we take a few lines to give the details of a table-driven approach.

The process for identifying specific values from the variables mentioned in a use case has four steps:

1. Identify all of the values that would be supplied by the actors contained in the use case model.

2. Identify equivalence classes of values for each input data type.

3. Construct tables that list combinations of values from the various equivalence classes.

4. Construct test cases that combine a single permutation of values with the necessary environmental constraints.

Since there are not enough inputs to our continuing example to make for an adequate example, we will consider a few variables from the personnel system example first introduced in Chapter 3. There are three variables mentioned in the use cases for that system. Each person has a name, an indication of whether the person is new to the system or has been existing, and a level of security authorization. Figure 9.6 shows the equivalence classes for these three variables. Figure 9.7 contains a column for each variable. Down the column are values from different equivalence classes for the variable. Each row of the table specifies a test.

Figure 9.6. Input value specifications for a personnel system

Figure 9.7. Input data permutations for a personnel system

The use case section Alternative Courses of Action is one of the most frequently used sections. These are less frequent but still normal cases. Also the Extends section provides successful cases in which the system achieves the use indicated in the main scenario and performs additional behaviors as well. These scenarios receive relatively less testing attention in the extending use case than in the original use case.

The Exceptions section provides those scenarios in which the use described in the normal cases cannot be achieved. These cases will be covered for the full system test; however, they will not be covered in regression test suites unless life-critical or safety-critical issues are involved.

The number of test cases that are created will depend on the contents of the frequency attribute of each use case. One technique for estimating the appropriate number is to take the number of distinct inputs and the number of equivalence classes for each input type and multiply them to get the maximum number of permutations:

 E1 * E2 * E3 *... * En 

in which E_i is the number of equivalence classes for the ith input. This gives the maximum number of test cases that could be created using this approach.

The number that will actually be created can then be limited either by the importance of the use case or by the amount of resources available. A small pilot effort or historical data can be used to determine the amount of resources required for a typical use case. Given the total number of use cases, an estimate can be made of the total effort that would be required for the full project.

The Expected Results Section of a Test Case

One of the most difficult things about testing complex systems is being able to determine the expected results from a test run. Telecommunication systems, scientific systems such as spacecraft control software, and information systems for multinational corporations are examples of systems for which test data and test results are expensive to construct. For each of the test cases derived from scenarios in the previous section, there must be a description of the results expected from the system during the test.

On a recent project we began to construct scenarios and derive test cases from them. It was only when we were ready to evaluate the test results that it became obvious that the development staff and their management had no idea what the exact system behavior should be for each of the scenarios!

A couple of techniques are useful to reduce the amount of effort required to develop expected results. The first is to construct the results incrementally just as the test cases are constructed incrementally. In this approach the test cases are written to cover some subset of a use of the system, perhaps only the data entry portion. Then, in succeeding cases the scope is broadened to cover the complete use. As the cases are broadened, the test results are extended.

This can be expanded to include iterative development of test cases. In this mode we begin by writing small test cases and gradually increase in size and complexity until the tests are realistic in terms of the production environment. In a database system, we might begin with a database containing fifty records and gradually increase until we have several thousand. The results expected at each new level must include any interactions that will occur because of the new cases. For example, in a database the presence of one record may inhibit the selection of another record that was selected in the previous test.

The second technique is to develop grand tour test cases in which the result of one test case produces the data that is input to the next test case. In this approach each test moves the test data through an entire life cycle. The resulting state of the database is then used as the input for the next test. This technique is particularly useful for life-cycle testing after low-level testing has found most of the failure-causing defects. By sequencing the test cases properly, the state after a successful execution of test case 1 establishes the database expected as the input to test case 2. The obvious problem with this approach is that a failure of test case 1 leaves the database in an unexpected state and we either cannot run test case 2 or at least must fix the database before we do.

Brickles

In Chapter 4 we began the system-level testing of Brickles by conducting a guided inspection. In preparation for that we prioritized the use cases using the frequency and criticality measures from the use cases. Figure 4.23 shows the results of that analysis. By the time the first increment is ready for code-based testing, we should reexamine this analysis to determine whether there are reasons to change the ratings for individual use cases. We will continue to use the ratings that were computed in Chapter 4.

With such a simple set of inputs there are few input parameters to analyze. Figure 9.8 shows the results of the analysis and adds additional factors such as the operating system version.

Figure 9.8. Input value specifications for Brickles

Figure 9.9 gives a subset of the permutations of those variables. Each row provides a test scenario. We will use the set of rows for a single Java operational environment, such as the Appletviewer, as the test suite for the C++ version.

Figure 9.9. Input data permutations for Brickles

In the first row, if zero pucks are lost, the expected end result would be that the game is won. However, in Brickles, if there is no movement of the paddle, all pucks should be lost (eventually). This is referred to as an infeasible test case. That is, it is a test case that is generated by some algorithm, but it is not possible. There is another infeasible case in which only three pucks are lost and there is no movement.

There is another variable that affects the operation of the game and the portion of the paddle on which the puck hits. A different computation is performed depending on whether the puck hits the paddle on the left, center, or right portion of the paddle. However, the analysis of the sequence of hits on the paddle produces a very large number of possibilities. The tester might choose to informally explore this dimension by having some hits occur in each of the regions. We will add it to the table because we are about to use an orthogonal array testing system (OATS) to reduce the number of tests needed to cover the problem.

Our table of permutations, shown in Figure 9.9, has four factors.^[1] Each factor happens to have three levels. Figure 9.10 provides the standard orthogonal array that fits the problem. Figure 9.11 shows the same array with problem-specific values substituted for the integers. These nine cases assume that each of the factors is independent of the others. Cases 1 and 4 are infeasible because if there is no movement then all the pucks will be lost. There is a dependency between these two factors, but it is not a perfect correlation. By changing the number to all, we can still use the test cases to cover some of the permutations.

^[1] See the earlier discussion of OATS in Orthogonal Array Testing on page 228 if you have forgotten the terminology.

Figure 9.10. Array for an OATS test design

Figure 9.11. Mapping of equivalence class values onto array