Chapter 34: JUnitPerf API Reference

JUnitPerf is a small set of test decorators and related classes that extend the JUnit API. In order to use JUnitPerf, you will need to be familiar with the functionality and use of basic JUnit tests as well as junit.extensions.TestDecorator.

Package com.clarkware.junitperf

This package contains all of the classes that make up JUnitPerf. The key classes in this package are LoadTest and TimedTest (as well as TimedTest's associated Timer interface). ConstantTimer and RandomTimer provide implementations of the Timer interface. The remaining classes are used mainly by the framework.

Class ConstantTimer

 public class ConstantTimer 

Inheritance Information

Extends: Object

Implements: Timer

Description

ConstantTimer is a Timer that always returns a constant delay. (See the sections on Timer and LoadTest for more information.)

Constructor

  ConstantTimer(long delay)  public ConstantTimer(long delay) 

This constructor constructs a Timer with the specified delay in milliseconds .

Method

 getDelay() public long getDelay() 

This method returns the delay specified by the constructor.

Class LoadTest

 public class LoadTest 

Inheritance Information

Extends: Object

Implements: junit.framework.Test

Description

LoadTest runs a Test with a specified number of concurrent users and/or a specified number of iterations for each user . Each Test is run in a separate thread, and LoadTest waits until all these threads have completed before ending the test. To create a Test with 20 simulated users, you would use the following code:

 Test targetTest = new ServerTest("testProcessPage"); LoadTest loadTest = new LoadTest(targetTest, 20); 

To create a LoadTest that simulates users who run a test multiple times, first wrap the test in a junit.framework.RepeatedTest and decorate it, or use the convenience constructor provided by LoadTest to do the same thing (see the listing below and the Constructors section for examples).

You can combine LoadTests with Timers to provide ramping behavior. Adding a Timer instance provides a delay between the addition of each new user to the pool of active users. The timer's getDelay() method specifies the amount of time between additions.

The Timer specifies the delay between the addition of users, not the delay between successive runs of the same decorated Test, which is controlled by a Timer-unaware junit.extensions.RepeatedTest. However, it would be trivial to extend RepeatedTest to add Timer functionality to it. See the listing below for an example.

You can combine LoadTest with a TimedTest to provide performance criteria for the load test. Either a TimedTest can be used to decorate the LoadTest (to ensure that execution of the whole test does not exceed a certain period of time) or the LoadTest can decorate a TimedTest (so that no individual Test exceeds the specified time, even when the system is under load).

LoadTest lets users specify the atomicity of the test under decoration. There are two modes: atomic and non-atomic. A LoadTest that does not enforce atomicity waits only for the threads simulating users to terminate before terminating itself--the Test is assumed to police its own threads. If atomicity is enforced, the LoadTest waits on all threads spawned by the decorated Test. In addition, an uncaught exception thrown during atomic LoadTest execution will interrupt all the threads spawned by the test.

This code demonstrates the use of LoadTest with other test decorators to create complex performance expectations:

 import junit.framework.*; import junit.extensions.RepeatedTest; import com.clarkware.junitperf.*; import test.com.company.ServerTest; public class ServerLoadTest{   public static Test suite(){          /*create a basic JUnit test*/     Test basicTest = new ServerTest("testProcessPage");          /*       Wrap basicTest in a TimedTest--a page load should       never exceed 5 seconds.           */     Test timedTest = new TimedTest(basicTest, 5000L);          /*       Wrap timedTest in a RepeatedTimerTest (defined below) to simulate        multiple page hits with time between repeat page views.      */      /*average 3.5 s. between views, with a variation of 2.5 s.*/      Timer timer = new RandomTimer(3500L, 2.50);      /*25 page views per user*/      Test repeatedTest = new RepeatedTimerTest(timedTest, 25, timer);            /*        Now make the repeated page viewing part of a load test: 10        concurrent users, added at constant 5s intervals.       */       timer = new ConstantTimer(5000);       Test loadTest = new LoadTest(repeatedTest, 10, timer);              /*         Finally set a maximum time out for the load test with another         TimedTest: time should not exceed 350 s.         */       Test loadTestWithTimeOut = new TimedTest(loadTest, 350000);              return loadTestWithTimeOut;   }//end suite()        public static void main(String[] args){         junit.textui.TestRunner.run(suite());     } } /**  * Adds a delay specified by a Timer between iterations of  * the test.  */ class RepeatedTimerTest extends RepeatedTest{   Timer timer;   private int repeats;   public RepeatedTimerTest(Test test, int repeats, Timer timer){     super(test, repeats);     this.repeats = repeats;     this.timer = timer;     }   public void run(TestResult result) {     for (int i= 0; i < repeats; i++) {           /*verifies that the test should continue*/        if (result.shouldStop()){          break;        }        /*wait for delay given by timer*/        try {          Thread.sleep(timer.getDelay());        }         catch(InterruptedException ignored) {        }        /* run the Test*/        basicRun(result);         }    } }//class 

Constructors

  LoadTest(Test test, int users)  public LoadTest(Test test, int users) 

This constructor decorates the specified Test as a LoadTest with the specified number of concurrent users and a user-addition delay of zero milliseconds (all threads start at once).

  LoadTest(Test test, int users, int iterations)  public LoadTest(Test test, int users, int iterations) 

This constructor decorates the specified Test as a LoadTest with the specified number of concurrent users, the number of iterations per user (that is, the number of times the Test is repeated), and a user-addition delay of zero milliseconds (all threads start at once).

  LoadTest(Test test, int users, int iterations, Timer timer)  public LoadTest(Test test, int users, int iterations, Timer timer) 

This constructor decorates the specified Test as a LoadTest with the specified number of concurrent users, the number of iterations per user, and a user-addition delay drawn from the Timer argument.

  LoadTest(Test test, int users, Timer timer)  public LoadTest(Test test, int users, Timer timer) 

This constructor decorates the specified Test as a LoadTest with the specified number of concurrent users and a user-addition delay drawn from the Timer argument.

Methods

  countTestCases()  public int countTestCases() 

This method returns the total number of TestCases run by this LoadTest ( essentially decoratedTest.countTestCases() multiplied by the number of users).

  run()  public int countTestCases()  setEnforceTestAtomicity(boolean isAtomic)  public void setEnforceTestAtomicity(boolean isAtomic) 

This method sets the enforcement policy of the LoadTest with respect to test atomicity. The policy should be set to true when the completion of threads spawned by the decorated Test is essential to the completion of the Test but the decorated Test does not wait on its own spawned threads.

Class RandomTimer

 public class RandomTimer 

Inheritance Information

Extends: Object

Implements: Timer

Description

RandomTimer is a Timer (see the sections on Timer and LoadTest) that returns random delays based on the average delay and variation specified in the constructor.

Constructor

  RandomTimer(long delay, double variation)  public RandomTimer(long delay, double variation) 

Both the base delay and the potential variation are in milliseconds (despite the difference in types). getDelay() will return a number between delay and delay + variation. (In other words, the variation will never be negative.)

Method

  getDelay()  public long getDelay() 

Class ThreadBarrier

 public class ThreadBarrier 

Inheritance Information

Extends: Object

Description

ThreadBarrier waits for a constructor-defined number of threads to complete execution and reports on whether the number has been reached. LoadTest uses it in non-atomic mode to keep track of the status of the threads used for test-running.

Constructor

  ThreadBarrier(int numDispatched)  public ThreadBarrier(int numDispatched) 

This constructor constructs a ThreadBarrier with the given number of threads to wait for.

Methods

  onCompletion(Thread t)  public synchronized void onCompletion(Thread t) 

This method is called when the Thread argument has finished execution.

  isReached()  public boolean isReached() 

This method returns true if the thread barrier has been reached--that is, if the number of Threads specified in the constructor equals the number of times onCompletion() has been called.

If onCompletion() is called more times than the number of threads waited for, the isReached() method will return false. Because the ThreadBarrier class is intended for internal use, this result should not present a problem for test developers.

Class ThreadedTest

 public class ThreadedTest 

Inheritance Information

Extends: Object

Implements: junit.framework.Test

Description

ThreadedTest is a decorator that runs a Test in a separate thread. LoadTest uses this class internally to manage the creation of multiple users running a Test simultaneously .

Constructors

  ThreadedTest(Test test)  public ThreadedTest(Test test) 

This constructor creates a Test that will run in its own thread. The new thread belongs to the current thread group .

  ThreadedTest(Test test, ThreadGroup group, ThreadBarrier barrier)  public ThreadedTest(Test test) 

This constructor creates a Test that will run in its own thread. The new thread belongs to the specified thread group and will register its completion with the ThreadBarrier.

Methods

  countTestCases()  public int countTestCases()  run(TestResult result)  public void run(TestResult result)  toString()  public String toString() 

Class ThreadedTestGroup

 public class ThreadedTestGroup 

Inheritance Information

Extends: ThreadGroup

Description

ThreadedTestGroup is a subclass of ThreadGroup used by the framework for exception handling while using ThreadedTests. Uncaught exceptions in a ThreadedTestGroup are added as failures or errors to the TestResult passed into setTestResult() . This class is used by LoadTest to keep track of and manage threads that simulate users.

Constructor

  ThreadedTestGroup(Test test)  public ThreadedTestGroup(Test test) 

This constructor constructs a new ThreadGroup associated with the given Test (the association is maintained so that accurate failures will result from testing).

Method

  setTestResult(TestResult result)  public void setTestResult(TestResult result) 

This method sets the TestResult object that failures should be registered with.

  uncaughtException(Thread t, Throwable e)  public void uncaughtException(Thread t, Throwable e) 

This method is overridden from ThreadGroup to catch uncaught, non-ThreadDeath throwables and record them as test failures. It interrupts all the threads in the group if such a throwable is caught.

Class TimedTest

 public class TimedTest 

Inheritance Information

Extends: junit.framework.extensions.TestDecorator

Implements: junit.framework.Test

Description

TimedTest is a TestDecorator that fails the decorated Test if the specified time limit is exceeded. You could use this class on individual TestCases for fine-grained results, or on TestSuites/TestDecorators (such as RepeatedTests or LoadTests) for aggregate results. (See the section on LoadTest for information about how to combine the two decorators.)

The time a JUnit test takes to run includes the time consumed by the setUp() and tearDown() methods as well as the running of the actual test. Users of TimedTest should factor the effect of setUp() and tearDown() into their specified performance criteria.

You can construct two types of TimedTests. The first waits until the Test has completed and then checks the elapsed time against the specified maximum. The second fails immediately if the elapsed time exceeds the maximum.

This second kind of TimedTest carries a few caveats; you should weigh these drawbacks against the advantages of faster test runs, should the maximum time be exceeded. First, TimedTest spawns a thread to run the decorated test that is not externally terminated , even if the time runs out. Second, because of implementation details, a slim chance exists that a TimedTest using this method will not fail even if the maximum time has been exceeded. The likelihood of this occurrence increases as the number of threads running as peers of the thread that called run() on the TimedTest increases . Because of this possibility, it is bad idea to wrap a TimedTest using the second method in a LoadTest (where the number of threads running as peers of the TimedTest can be high).

Constructors

  TimedTest(Test test, long maxElapsedTime)  public TimedTest(Test test, long maxElapsedTime) 

This constructor decorates the specified Test with the given time in milliseconds. The TimedTest will wait for the completion of the test and then measure the elapsed time against the specified maximum. If the elapsed time exceeds the maximum, the test will fail. TimedTests that fail in this manner include the information on the expected versus actual time in their failure messages. The following code creates a new ServerTest instance that executes the testProcessPage method, and then decorates it so it will fail (after the fact) if the test takes longer than half a second to complete:

 long maximumTime = 500; Test targetTest = new ServerTest("testProcessPage"); TimedTest test = new TimedTest(targetTest, maximumTime);  TimedTest(Test test, long maxElapsedTime, boolean waitForCompletion)  public TimedTest(Test test, long maxElapsedTime, boolean                                            waitForCompletion) 

This constructor decorates the specified Test with the given time in milliseconds. Depending on the value of waitForCompletion, the TimedTest will either wait for the completion of the test and then measure the elapsed time against the specified maximum (true, see the previous constructor) or fail the test immediately if the specified time is exceeded (false).

This TimedTest will fail after 2.5 seconds if the ServerTest has not finished its execution by that point:

 long maximumTime = 2500; Test targetTest = new ServerTest("testPotentially30SecondMethod"); TimedTest test = new TimedTest(targetTest, 2500, false); 

Methods

  countTestCases()  public int countTestCases() 

See junit.framework.TestDecorator.

  outOfTime()  public boolean outOfTime() 

This method returns whether the TimedTest exceeded the specified maximum time. This method is intended to be called after run() as a reporting tool.

  run(junit.framework.TestResult result)  public void run(junit.framework.TestResult result) 

This method runs the test in one of the two ways specified by the constructor.

  runUntilTestCompletion(junit.framework.TestResult result)  protected void runUntilTestCompletion(TestResult result)  runUntilTimeExpires(junit.framework.TestResult result)  protected void runUntilTimeExpires(final junit.framework.TestResult result) 

These methods provide the two types of timed execution. They re used internally.

  toString()  public String toString() 

This method informs you whether the test is waiting or non-waiting, but it does not give information about the maximum time specified for this TimedTest instance.

Interface Timer

 public interface Timer 

Description

This interface defines a timer that can be used with LoadTest to regulate the addition of new users to the LoadTest This interface can be implemented by developers who wish to customize the simulation of user loads.

Method

  getDelay()  public long getDelay() 

This method returns a number of milliseconds representing the delay between two events regulated by this Timer.