Section 40.4. Summary

40.4. Summary

Our example model to generate tests depends on a very simple model of the rental application. It is clear that this technique could be applied to a wide range of applications and could interleave more types of actions.

The generator makes changes that cancel out over the course of the whole test. For every rental that is made in the generated tests, we ensure that a corresponding return is made. Rentals and returns of different rental items by different clients are interleaved to make the tests more realistic and more likely to find errors.

See [Got03] for a formal introduction to testing based on symmetries. This paper also discusses techniques for reducing the generated test set without losing effectiveness. Cohen et al. [CGMC03] discuss various approaches to reducing test set size.

Other approaches to test generation may be more or less comprehensive. The more comprehensive the tests, the more comprehensive the model and the generator need to be. As with all testing, there are inevitable tradeoffs, so that limited testing resources can be used as effectively as possible.

Testing a system is based on the idea that having two or more different ideas of how a system will work may help pick up errors in either of them. It's important that the form of those approaches be different enough that we don't make the same wrong assumptions about the implementation.

However, one problem is very difficult to avoid: If there are misunderstandings about what is really required, the different approaches may simply encode the same mistakes. As Reinertsen [Rei97] points out, the major risk in design is in understanding what is needed, not how it will be provided.