In-memory Testing Issues

The challenges for running tests faster by running them in memory are these.

• How do you eliminate the database access from the business logic? This requires separation of persistence from business logic, including object queries.

• How do you know which tests can run in memory? And how do you know which tests can only be run against a real database? We use standard JUnit test packaging conventions, and aggregate tests capable of being run in memory in a separate test suite from those that require a database.

• How do you specify whether they should be run in memory on a particular test run? This requires dynamic test adaptation.

• How do you deal with configuration-specific behavior? This requires environment plug-ins.

Separation of Persistence from Business Logic

Switching back and forth between testing in memory and against a database is possible only if application code and tests are unaware of the source of objects.

We have been building business systems using a "Business Object Framework" for about five years in both Smalltalk and Java. The objective of this framework is to move all the "computer science" out of the business logic and into the infrastructure. We have moved most of the "plumbing" into service provider objects and abstract classes from which business objects can inherit all the technical behavior. The technical infrastructure incorporates the Toplink Object/Relational (O/R) mapping framework. Toplink is primarily responsible for converting database data into objects and vice versa. It eliminates the code needed to implement these data/object conversions (which means no SQL in application code), and it does automatic "faulting" into memory of objects reached by traversing relationships between objects. This eliminates the need for having explicit "reads" in the business logic.

In essence, Toplink makes a JDBC-compliant data source (such as a relational database) look like an object database. Our infrastructure makes persistence automatic, which leaves developers to focus on the business objects in an application and not on the persistent storage of those objects. By removing all knowledge of persistence from application code, it also makes in-memory testing possible.

This approach is described in more detail in [Meszaros+1998].

Querying (Finding Objects by Their Attributes)

If applications access relational databases, querying by using table and column names, then replacing a relational database with an in-memory object database becomes problematic because an in-memory database contains objects, not tables. How do you hide the different sources of objects from the application, especially when you need to search for specific objects based on the values of their attributes?

Solution: Object Queries

All queries are performed against the objects and their attributes, not against the underlying database tables and columns. Toplink's query facility constructs the corresponding SQL table/column query for queries specified using objects and attributes. Application code is unaware of where and how objects are stored which provides for swapping the "where and how" between an in-memory and a relational database.

Our initial approach to querying was to move all database queries into "finder" methods on Home (class or factory) objects. But to support both in-memory and database querying, we had to provide two implementations of these finder methods: one that used Toplink's query facility against a relational database and another that used the API of the in-memory database to perform the query.

We quickly tired of having to implement queries twice and have now implemented support for the evaluation of Toplink's object/attribute queries against our in-memory database. With this technology, the same query can be used to find objects in our in-memory object database or can be translated into SQL to be sent to a relational database.^[1]

^[1] Since this technology was developed, Toplink has incorporated limited support for queries that run against its in-memory cache. This is a step in the right direction but does not yet address all our testing needs.

Configuration-Specific Behavior

When testing in memory, how do you handle functional features of databases, such as stored procedures, triggers, and sequences?

Solution: Environment Plug-ins

The functional features of databases can be implemented in memory by environment plug-ins (Strategy [Gamma+1995]). Each function provided by the database has a pair of plug-ins. When you test with a database, the database version of a plug-in simply passes the request to the database for fulfillment. The in-memory version of the plug-in emulates the behavior (side effects) of the database plug-in.

In a relational database, a sequence table may be used to generate unique primary key values. The in-memory version of the plug-in keeps a counter that is incremented each time another unique key is requested.

Stored procedures can be particularly problematic when you build an interface to an existing (legacy) database. On one project, we had to create an account whose state was initialized by a stored procedure. During in-memory testing, the state was not initialized, so business rules based on the account's state failed. We could have added code to set the state during the account object initialization, but we did not want to have any code specific to testing in the production system. We were able to avoid this using an InMemoryAccountInitializationStrategy that performed the required initialization during in-memory testing and a Null Object [Woolf1997] that did nothing when the database's stored procedure initialized the state.

Because an in-memory plug-in may behave differently than the database functionality it replaces, it is still necessary to run the tests against the database at some point. In practice, we require a full database test before changes are permitted to be integrated.

Environment Configuration

How and when do you set up the test environment with the appropriate configuration-specific behavior? How do you decide which environment to use for this test run?

Solution: Dynamic Test Adaptation

We use test decorators to specify whether the test environment should be in-memory or database. Using this technique, we can choose to run a test in memory for maximum speed, or we can run the test against the database for full accuracy. One hitch is the fact that you can choose to run all the tests, just the tests for a package, just the tests for a class, or just a single test method. To ensure that the tests run in the right environment regardless of how they are invoked, we have added methods to our TestCase base class that push the decorators down to the individual test instance level.

On a recent project, we created two Java packages: one containing tests that could only be run in memory (because the O/R mappings were not yet complete) and one for multimodal tests (tests that could be run either in memory or against a database.) As the project progressed, tests were moved from the in-memory test package to the multimodal test package. Combining this organizational scheme with dynamic test adaptation, we were able to run the multimodal tests against either the in-memory or the relational database.