Example Unit Testing Tool

We have created a unit testing tool called Test Mentor that embodies many, if not all, of the previous requirements. Although it is a commercial tool, the principles outlined as follows are general enough to be applicable anywhere.

Test Representation

We chose to provide a dual representation for tests. For developers, code is the representation of choice. For testers, we developed a representation that enables them to describe test cases without writing code.

Developer Test Representation

At the highest level, tests are composed of suites, which in turn are composed of test assets. Suites are physically represented as classes within a test hierarchy. Test assets are represented by methods.

Integration with JUnit

To support developers who are already comfortable with developing tests with JUnit, we chose to integrate the JUnit and Test Mentor frameworks rather than substantially changing the way developers work.

Changing the superclass of junit.framework.TestCase to inherit from Test Mentor's hierarchy makes all JUnit tests visible to Test Mentor without the need to change the tests themselves. Test Mentor also provides a means to register JUnit tests as Test Mentor assets available for reuse by testers.

Test Representation for Testers

The representation of a test for a tester takes one of two forms:

• An XML description of the complete test.

• A visual representation that testers edit through a specialized GUI. The visual representation is saved as Java code, although this is transparent to the user.

These representations are robust enough to represent instantiation, interaction, and validation, as described earlier.

The representation for testers adds the notion of a test step. A test step is the unit of greatest granularity and is modeled after the notion of intelligent test artifacts [Silverstein2000]. Test Mentor provides a number of step types, each providing its own specific behavior, such as the following.

• Calling methods and constructors

• Setting/getting fields

• Iterating over test data

• Referencing test assets (asset reuse)

• Sequencing other test steps

• Running a piece of Java code (script)

• Recursively extracting object state

• Validation (assertion-like behavior)

• Accessing distributed objects such as EJBs or through distribution schemes such as RMI or CORBA

Testers use the editing tools provided by Test Mentor to organize steps in a particular order and configure steps to operate on target objects.

Test steps are configured through common properties such as the following.

• Name and description (documentation)

• Expected exception (class of exception expected to be thrown by executing the step)

• Iterations (number of times to execute the step)

• Performance criterion (maximum time the step should take to execute)

• Precondition (step to execute as a precondition; similar to the setup method in JUnit)

• Postcondition (step to execute as a postcondition; similar to the teardown method in JUnit)

• Failure (step to execute if the owning step fails for any reason)

• Invariant (step to execute, which performs an assertion on some state that is assumed to be true under all circumstances)

• Maximum execution time

All of these common properties except for "name" are optional, which reduces the overhead of step configuration. Individual step types have their own configuration properties.

For example, a method step is a type of step that calls a method. A method step is configured by the method name, the receiver (usually the object under test, known as a test subject), and any parameters. In this case the tool GUI provides lists of available methods in the class or provides a way for the tester to specify a method that does not yet exist in a class that may or may not yet exist. Figure 11.1 shows the part of the tool user interface for configuring a method step.

Figure 11.1. Configuring a step to call a method

The execution framework takes care of the underlying details of assembling parameters and handling returned values.

To instantiate an object, Test Mentor provides a step similar to the previous called a constructor step. Users configure constructor steps by selecting a constructor from a list and specifying the required parameters. When a constructor step is executed, it returns an instance of the object in question.

In the case of remote objects, such as EJBs, access is provided by special steps called EJB steps that encapsulate the mechanics of remotely accessing an EJB on an application server. All the tester needs to know to test an EJB is the name that the bean has been registered under and its type (interface) as well as certain deployment-specific properties such as application server address and port. From the perspective of a tester, an EJB step performs exactly the same way as a constructor step (it returns an object) and can be used interchangeably. The tool hides the fact that what is actually returned is a stub for a remote object.

Sequences of steps are organized as tests to create, apply stimuli to, and validate the component under test. Figure 11.2 shows a simple test asset (named "Test financialInstrument.Stock") that creates an instance of a financialInstrument.Stock class, calls several methods, and then validates the state of the object.

Figure 11.2. A sequence of steps

Finally, test assets can be reused through a type of step called a shortcut step. These steps are configured by specifying the name of a target suite and an asset within it. Because assets can be parameterized, and steps can be passed as parameters, there are few limits on the way tests can be composed and reused.

Universal User Interface

One of the side benefits of the availability of a visual test composition facility is that it provides a universal user interface for interacting with arbitrary objects, as shown in Figure 11.3

Figure 11.3. End-user interface and visual test steps operating on an object under test

The visually composed test steps stand as a surrogate for a user interface. A user interface might be the final end-user interface or one created solely for the benefit of testing.

More detail about Test Mentor is beyond the scope of this chapter. To summarize, Test Mentor is an example of a tool that enables developers to create unit tests as they always have as code and a visual representation that enables testers to define unit tests without requiring programming skills, the end goal being collaboration and sharing between the two groups during early testing.