15.2 Test Execution

The entire testing and verification process of a domain can be automated. Each test can be set up, run, and verified automatically so that we can run tests frequently ideally with every change. We can also add new test cases as we change the models.

First, we show how to establish a unit test for a single activity in a use case. For each test vector, we need to establish the initial state of the model, accept the external signal that signifies the use case, and execute to see what happens. We then evaluate the outcome to see if it is correct.

The state of the system that must exist prior to the test is defined in terms of a set of objects, the values of their attributes, their current states, and any required associations. Examining the preconditions in Figure 15.1, we need:

• An order.

• A product to add to the order. There must be sufficient quantity on hand to fill the order.

• One or more associated selections already associated with the order. Each selection must be associated to a product.

Write action language to set up the test state of the system. It is easiest and most direct to create Product objects synchronously with arbitrary, but reasonable, attribute values, as shown in Figure 15.5.

Figure 15.5. Creating Instances of Product

There now exist two instances of the product unimaginatively named A and B.

To execute test vector 1 from Figure 15.4, we must establish the initial state. The precondition of this activity requires designating Adding Selection to Order as the initial state for the scenario, as shown in Figure 15.6. (Even though state 1, New Order, can accept the same signal, it is self-directed and will have already caused a transition to state 2.)

Figure 15.6. Establishing the Initial State

Next we identify the external signal that initiates the test case and define the parameters to it. The signal parameters include:

• A reference to an existing open order (the one required by the state of the system above).

• A reference to an existing product (again, one required by the state of the system).

• A quantity to add to the order. This value needs to be less than the quantity on-hand of the product.

So we generate the signal addSelection:

generate addSelection (productID: productB.productID, quantity: 3) to order;

Each procedure may send other signals. The verifier processes signals until there are no more signals outstanding, leaving the system in a new state that should match the postconditions in the use case.

This process creates an execution trace that can be visualized using a sequence diagram, which shows the new states, as shown in Figure 15.7.

Figure 15.7. Sequence Diagram for Add Selection Activity (Very Simple!)

Check also that attributes have the correct values, associations are correct, and that any expected outbound signals have been sent. For this test case:

• A product selection should be associated with the order and the product referenced in the initial signal.

• The quantitySelected in the product selection should match the quantity in the initial signal, plus any amount already selected.

• All derived attributes (e.g., the totalValue of the order) should have the correct value.

Result-checking against externally specified values can also be coded in the test case. In some cases we may need to record the old value of an attribute (e.g., the order's original value) to check it against a new or changed value (the order's value with the new product added to it). Figure 15.8 shows how we could check the results of the test case.

Figure 15.8. Checking the Results of the Test Case

If all the assertions are true, the test vector passes. The test case may also clean up objects and/or restore the system back to a known state so that other test cases may be run.

Test vectors can be checked automatically. We have abbreviated checking the test vector to a single (nonstandard) assert statement in the logic above. To find out more about test vector generation and injection a whole body of knowledge in itself see, for example, [1] and [2].