Tests That Leave the System State Unchanged

Unfortunately, in some situations, this simple approach doesn't work. One such situation is where the reset step becomes too costly. This could be because it takes a long time or because it has to be performed too frequently, or a combination of the two. If you find that running acceptance tests becomes a bottleneck and that most of your time is spent waiting on system resets, it's time to do something about it.

If you haven't already automated the reset, this would be a reasonable time to do it. If you've already done it, take a look at refactoring the tests that modify the state of the system, so they always leave it in the same state as when they started. For instance, a test of deleting records could create the records it deletes, and a test that creates records could delete what it created at the end of the test.

Suppose we've written the tests shown in Listing 18.1 for creating and deleting user ids (they have to come from somewhere, after all).

Listing 18.1 Version 1.0

 public class UserIdStoryTest {    public void testCreate() {      login("super","superpassword");      assertTrue( createUserId( "new",                                "newpassword",                                "[e-mail]") );      assertFalse( createUserId( "fred",                                 "",                                 "") );    }    public void testDelete() {      login("super","superpassword");      assertTrue( deleteUserId( "bob" ) );      assertFalse( deleteUserId( "doug" ));    } } 

The create tests assume a user already exists with id of super and that none exists with the id of new. The delete tests also assume the existence of the id super as well as a user with id bob and no user with id doug. This is certainly easy to arrange in the initial state of the system, but once these tests run, the assumptions are no longer true, and the system must be reset.

The first attempt to createUserId has the expected result of true, because we're including all the fields needed for successful user id creation. That's why we use assertTrue. The second attempt, to add fred, fails, because we left off two required fields. The expected result is false, so we use assertFalse. Chapter 22 will go into more detail about how we know what to code in these tests.

We can refactor the test so that, while it still has some assumptions about the initial state, its execution doesn't affect the truth of those assumptions and can be run over and over without resetting the system. Listing 18.2 shows how we could do it (we've marked the changes with bold text).

Listing 18.2 Version 1.1

 public class UserIdStoryTest {    public void testCreate() {      login("super","superpassword");      assertTrue( createUserId( "new",                                "newpassword",                                "[e-mail]") );      assertFalse( createUserId( "fred",                                 "",                                 "") );      deleteUserId("new");    }    public void testDelete() {      login("super","superpassword");      createUserId( "john",                    "johnspassword",                    "[e-mail]");      assertTrue( deleteUserId( "john" ) );      assertFalse( deleteUserId( "doug" ));    } } 

As you can see, we've added actions to the test that undo the changes to the system state. In fact, we already had something like this at the beginning of the test: login. The login is necessary to set up the test, because only an already authorized user can create user ids. The createUserId we added to the testDelete method is just an additional setup step, creating the record we're about to delete. The deleteUserId we added to the createUserId method is a teardown step, done after the test to clean up and release resources. Some setup steps are mandatory, but the teardown steps aren't they just provide the ability to redo the test without having to reset the system.

Don't worry that time spent automating the system reset is wasted if you then have to refactor the tests to include setup and teardown steps. You'll still need the reset because, even when every test is supposed to leave the system in the same state, not every test will. For instance, what if the delete function fails? The delete test will create the user id john but not delete it, and the assumptions will then be violated. In this case, you'll need to resort to the system reset.