10.2 Testing Asynchronous Services

Asynchronous service objects come in many different flavors. Some start a single "worker thread," which handles all jobs consecutively when they are created. Others create a separate thread for each job. Still others have a fixed pool of threads, using them alternately to serve incoming requests. An important distinguishing feature is whether or not the triggering thread might be interested in the result of an asynchronous service job at one point in time.

Service without Result

Let's use an example to see the test-first development of a service object. In the beginning, there is usually a normal synchronous invocation, as requested by the following test:

 public class MyServiceTest extends TestCase {    public void testServiceInvocation() {       MyService service = new MyService();       assertFalse(service.hasFinished());       service.execute();       assertTrue(service.hasFinished());    } }

Suppose that during the further course of the implementation, the runtime of the execute() method becomes suddenly longer than many callers would want it to be. This observation and the knowledge that we don't have to rely on a result of the service execution lets the implementation and thus our tests march towards asynchronous invocation. Here is the first attempt:

 public void testServiceInvocation() {   MyService service = new MyService();   assertFalse("Not yet started", service.hasStarted());   service.invokeAsynchronously();   assertTrue("Started", service.hasStarted()); }

This test does not really force the service to run asynchronously (i.e., in another thread). For example, it would be conceivable to test for fast return of the invokeAsynchronously() method. But how fast would be fast enough? Due to a lack of stronger sanctions, we are pragmatic and look at the method's name as an obligation. Under this assumption, the implementation of the invokeAsynchronously() method looks like this in the simplest case:

 public class MyService {    private volatile boolean started = false;    ...    public void execute() {       //service execution    }    public void invokeAsynchronously() {       new Thread() {          public void run() {             started = true;             execute();          }       }.start();    }    public boolean hasStarted() {       return started;    } }

At first sight, everything appears to be correct and the test runs faultlessly. But if we run it often, the test runner's bar will sporadically turn red. ^[1] This is due to the wrong assumption that our service thread will happily start running without delay after the invocation of invokeAsynchronously(). In reality, we cannot predict when the new thread created in invokeAsynchronously() will actually start running, and whether the line,

 started = true;

will be reached before the main thread, namely,

 assertTrue("Started", service.hasStarted());

starts. To reduce the probability that a single test functions accidentally, we manipulate the test suite so that it doesn't run once, but always 10 times—a concession to nondeterminism. Mind that this doesn't give us any guarantees either, but it is a pragmatic way to improve our chances:

 public class MyServiceTest extends TestCase {    ...    public static Test suite() {       TestSuite suite = new TestSuite(MyServiceTest.class);       return new junit.extensions.RepeatedTest(suite, 10);    } }

The class junit.extensions.RepeatedTest is a test decorator that can be built around any arbitrary suite or single test.

A first attempt to make the sequence of events more predictable is to insert a short sleeping phase into the test thread, which the service thread can use for its work:

 public void testServiceInvocation() throws Exception {    MyService service = new MyService();    assertFalse("Not yet started", service.hasStarted());    service.invokeAsynchronously();    Thread.sleep(100);    assertTrue("Started", service.hasStarted()); }

This is sufficient for the present case to create a kind of "pseudodeter-minism." The chance that the 100ms wait time will expire before the other thread gets going is small.

Now we should test whether or not a second attempt to restart the service will fail; after all, MyService instances are designated for one-time use:

 public void testDoubleInvocation() {    MyService service = new MyService();    service.invokeAsynchronously();    try {       service.invokeAsynchronously();       fail("RuntimeException expected");    } catch (RuntimeException expected) {} }

And here is a first implementation attempt that doesn't work:

 public void invokeAsynchronously() {    if (started) {       throw new RuntimeException("MyService already started");    }    new Thread() {       public void run() {          started = true;          execute();       }    }.start(); }

Once again, the unpredictable timing of the threads plays a role. The test shows that the service thread has not yet had a chance to set "started" to "true" upon the second invocation of invokeAsynchronously(). Consequently, we had to introduce an additional variable:

 public class MyService {    ...    private boolean invoked = false;    public synchronized void invokeAsynchronously() {       if (invoked) {          throw new RuntimeException(             "MyService already started");       }       invoked = true;       new Thread() {          public void run() {             started = true;             execute();          }       }.start();    } }

In the method invokeAsychronously() before actually starting a new thread, we test that no service invocation has taken place yet and throw an exception in that case. Note that this method was now declared synchronized to prevent two threads from entering it at the same time. This "synchronized" is not being enforced by our tests though; we really begin to have doubts if all aspects of concurrent programming can be tackled by driving our development through tests only.

Service with Result

As long as we are happy with the fact that the service was started, our main task is met. However, the service is often started asynchronously only to be able to use the result of a lengthy calculation or information retrieved over a slow Internet connection at a later point in time.

There are many variations on how to allow the service thread to inform the requesting thread about the result. Doug Lea dedicates an entire chapter of his book to this issue. ^[2] The interesting thing from our perspective is that the test has to pause for some time before it can expect a result and check it for correctness.

Consider an asynchronous summation service (SumUpService); its result can be requested as String over the method getStringResult(). As long as there is no result yet, this method returns null. And here is the test:

 public void testInvocationWithResult() throws Exception {    int[] numbers = new int[] {1, 2, 3};    SumUpService service = new SumUpService(numbers);    service.invoke();    Thread.sleep(1000);    assertEquals("6", service.getStringResult()); }

Once more, we have used sleep(...) to give the worker thread a chance to complete the summation. In doing this, however, we encounter the problem that we don't know how long the calculation will really take. We can (normally) define an upper limit and expect termination of the service within this limit. This maximum value is normally higher than the average value and strongly dependent on the platform on which we are executing the tests. So our test may take much longer than it actually needs to.

The helper class RetriedAssert can help us out of this dilemma. ^[3] It allows repeated verification of an assert condition up to a maximum wait time. This means that our test will change as follows:

 public void testInvocationWithResult() throws Exception {    int[] numbers = new int[] { 1, 2, 3 };    final SumUpService service = new SumUpService(numbers);    service.invoke();    new utmj.threaded.RetriedAssert(2000, 100) {       public void run() throws Exception {          assertEquals("6", service.getStringResult());       }    }.start(); }

The maximum wait time (2000ms in this example) and the request interval (100ms) are determined when the anonymous inner instance is created. Consequently, we know that our test will wait up to a maximum of two seconds for the result, but that it will continue at the latest after 100ms from the end of the calculation. This way to invoke assertions also works for most of the other procedures that an asynchronous service can use to notify its result. The RetriedAssert class is superfluous when the service itself offers a way to wait for the result, including notification within a maximum timeout interval.

Expected Exceptions in Split-Off Threads

Another test should verify the occurrence of an exception while the service is running. We know that the implementation of SumUpService cannot handle numbers greater than 1000, which means that it throws an IllegalArgumentException during the service execution in these cases. We wrote the following test to verify this situation:

 public void testInvocationWithIllegalNumber() throws Exception {    int[] numbers = new int[] { 1, 1001, 3 };    final SumUpService service = new SumUpService(numbers);    try {       service.invoke();       fail("IllegalArgumentException expected");    } catch (IllegalArgumentException expected) {} }

The test fails although the implementation of SumUpService suggests something else, ^[4] at least at first sight:

 public class SumUpService {    ...    private void sumUp() throws InterruptedException {       int sum = 0;       for (int i = 0; i < numbers.length; i++) {          int each = numbers[i];          if (each > 1000) {             throw new IllegalArgumentException(                each + " too big");          }          sum = sum + each;       }       result = Integer.toString(sum);    }    public void invoke() {       new Thread() {          public void run() {             try {                sumUp();             } catch (InterruptedException ignore) {}          }       }.start();    } }

A closer look shows the problem clearly: the invoke() method starts only the service thread. Exceptions occurring in this thread are not passed to the original thread.

Luckily, there is a helper class to solve this problem, too: utmj.threaded. ExceptionAssert. This helper class allows us to check for uncaught runtime exceptions in split-off threads; all others have to be caught eventually. The use of this class is similar to RetriedAssert:

 public void testInvocationWithIllegalNumber() throws Exception {    int[] numbers = new int[] { 1, 1001, 3 };    final SumUpService service = new SumUpService(numbers);    new utmj.threaded.ExceptionAssert(       IllegalArgumentException.class, 2000) {       public void run() {          service.invoke();       }    }.start(); }

We pass the expected exception type as constructor parameter, including the maximum wait time. The test fails when this wait time has elapsed before this exception has occurred somewhere.

Unexpected Exceptions

We learned in the previous section how expected exceptions in split-off threads can be verified. But what about unexpected runtime exceptions?

In "normal" operation, each uncaught exception leads to a test error. Normally, we would also like to have this behavior for multi-threaded programs. And again, the utmj.threaded package offers us an appropriate test decorator, MultiThreadedTest, we can use to decorate the corresponding test suite:

 public class SumUpServiceTest extends TestCase {    ...    public static Test suite() {       TestSuite suite = new TestSuite(SumUpServiceTest.class);       return new utmj.threaded.MultiThreadedTest(suite);    } }

Behind the scenes, both ExceptionAssert and MultiThreadedTest use the same trick: they create a subclass of java.util.ThreadGroup, which handles uncaught runtime exceptions by overriding the method uncaughtException(Thread t, Throwable e).

^[1]The exact behavior depends on the operating system, the JVM you use, and sometimes the compiler's mood.

^[2][Lea00, chap. 4, p. 281].

^[3]This and all other helper classes in this chapter are included in the package utmj.threaded, available from the Web site to this book.

^[4]Of course, we built the if condition into sumUp() for demonstration purposes only.