Optimizing Test Development

The JUnit test life cycle specifies setting up a test fixture before a test and tearing it down afterward. But we found that many of our functional tests depended on a large number of other objects. A test can fail if it depends on a previous test's side effects, and those effects can vary, depending on a test's success or failure. We divided the objects used by a test into three groups.

• Objects referenced but never modified. These shared objects can be set up once for all tests.

• Objects created or modified specifically for a test. These temporary objects must be created as part of each test's setup.

• Objects created, modified, or deleted during the test.

We made it a hard-and-fast rule that tests could not modify any shared objects, because to do so makes tests interdependent, which in turn makes tests much harder to maintain.

Shared Test Objects

In database testing, these shared objects would be the initial contents of the database before any testing started. How do you ensure that the shared objects are available in both in-memory and database testing modes?

Solution: In-memory Database Initializer

For in-memory testing, we define an Initializer object, whose responsibility is to create all the objects necessary to replicate the expected static database contents. The test infrastructure delegates to it to ensure that these objects are created before any tests need them.

If there is an existing legacy database, testing can be performed before the database O/R mappings are in place by manufacturing objects in memory that correspond to legacy data and placing them in the in-memory database.

When we are building applications that require a new database, an in-memory database can be populated with objects created by the Initializer. When the database schema is finally defined, the Initializer can be run with persistence enabled. The objects created by the Initializer are automatically written to the database to create the initial database content.

Temporary Object Creation

At any one time, several developers may be running the same tests. We need to ensure that the tests don't interact either with themselves or with other tests. How can we ensure that newly created objects have unique keys and contain all data required to make them valid? How can we ensure that several instances of the same test being run from several workstations aren't using the same values for keys, thus causing transient test failures?

Solution: Anonymous Object Creation

We created a TestScenarioManager, which is the hub of all test object creation. Whenever a new kind of object is needed for a test, a createAnonymousXxxx() method is added (with any arguments required for customization). These methods create a fully formed object that may be used in tests without worrying about details like unique-key constraints and unintended test interactions generated identifiers that are guaranteed to be unique.

Temporary Object Cleanup

Tests may create many new objects. Depending on where a test failed, the objects to be cleaned up could vary significantly. How can you ensure that all the temporary objects are cleaned up properly without having to write complex teardown logic?

Solution: Automatic Fixture Cleanup

To simplify teardown, each createAnonymousXxxx() method registers the newly created object as a transient object that needs to be deleted. Each test inherits a teardown method from our TestCase base class that automatically deletes all the registered test objects. This eliminates the need to write any test-specific teardown code. The test need only ensure that any objects it creates as part of the testing logic (as opposed to fixture setup) are also registered for automatic removal.