7.1 Unit Tests


Unit tests are small pieces of software that test specific conditions in your source code. Typically, it is the developer's responsibility to write and maintain unit tests. Unit tests are often used to test such conditions as boundaries, unusual data types, interfaces between components , and any complex operation that needs to be continually verified as the software changes. Unit tests should be run regularly as the software is being built. Each development team uses their own methodology for building software. We recommend that the following simple rules be applied for working with unit tests:

  • Any piece of code checked into the main code base must have a corresponding unit test.

  • Code cannot be checked into the main code base until it passes its unit tests.

  • Unit tests should be kept current, being updated as the code is updated.

  • Unit tests should be part of a framework that runs regularly (every night, every week, and so on) and reports meaningful results.

A unit test can be something as simple as:

 int main() {    ... test some stuff   return 0; } 

A simple framework can lend organization to your unit test strategy. We include a unit test framework with the following features:

  • Each piece of functionality can be placed in a separate test function so it can be tested in isolation. New tests can be added by simply creating a new function.

  • A few simple macros are included that make it easy to verify certain conditions and report any that are not correct.

  • All exceptions are caught and reported as errors.

7.1.1 Using the Unit Test Framework

This section provides an overview of how you use the unit test framework that we provide.

  1. Write one or more unit test functions using the functionality provided:

    UTFUNC()

    Creates a unit test function of the specified name and adds the function name to the list of functions to run. UTFUNC() actually creates a small object, but this detail is hidden.

    setDescription()

    Specifies what the unit test does. This string is displayed when the test is run.

    VERIFY()

    Verifies that a specified condition is true. If the condition is true, nothing is output. If the condition is false, an error is generated and the specified condition is displayed as the message when this function is run.

  2. Run the unit test functions. The execution time is computed and any exceptions are caught. If a unit test has no VERIFY() statements, the result is set to eUnknown . If an exception is thrown, the unit test result is set to eFailure . Possible results for unit tests are as follows :

    eNotRun

    The default state if the unit test has not been run.

    eRunning

    The unit test is currently running.

    eUnknown

    A unit test that has no VERIFY() statements is labeled as unknown.

    eSuccess

    The unit test had no failures.

    eFailure

    The unit test had one or more failures.

  3. Use main() to call apUnitTest::gOnly().run() , which runs all of the unit tests in the framework object. The results are then written to the console (or other stream). main() also returns the cumulative state of the unit test framework object as if there are any failures or unknowns, or as 1 if there are only successes.

graphics/triangle.gif EXAMPLE

We have written unit tests for almost every component we present in this book. All unit tests are included on the CD-ROM. Here, we use one of the apBString unit tests as an example:

 UTFUNC(Pel8) {   setDescription ("Pel8 tests");   Pel8 b;   Pel16 w;   Pel32 l;   Pel32s ls;   float f;   double d;   std::string s;   apBString bstr;   bstr << (Pel8) 123;   bstr >> b;  bstr.rewind ();   bstr >> w;  bstr.rewind ();   bstr >> l;  bstr.rewind ();   bstr >> ls; bstr.rewind ();   bstr >> f;  bstr.rewind ();   bstr >> d;  bstr.rewind ();   bstr >> s;  bstr.rewind ();   VERIFY (b == 123);   VERIFY (w == 123);   VERIFY (l == 123);   VERIFY (ls== 123);   VERIFY (f == 123);   VERIFY (d == 123);   VERIFY (s.compare ("123") == 0);   bstr >> b;   VERIFY (bstr.eof()); } 

This function tests that a byte can be written to the binary string and then read back in multiple formats, to verify that the conversions were made correctly.

You can use the provided framework to write your own unit tests. Our framework encourages you to write a number of small, isolated tests instead of a large complex test. We strongly recommend that you test as much as possible in your unit tests, as we demonstrate in the following portion of the apRect unit test:

 UTFUNC(rect) {   setDescription ("Rect");   apRect rect (0, 1, 2, 3);   VERIFY (rect.x0() == 0);   VERIFY (rect.y0() == 1);   VERIFY (rect.width() == 2);   VERIFY (rect.height() == 3);   ... } 

This function is testing trivial inline functions and the apRect constructor. Do not assume that this simply works, or that other test functions will indirectly test these member functions. You should test these functions directly. Here is the unit test for the apRect default constructor:

 UTFUNC(defaultctor) {   setDescription ("default ctor");   apRect rect;   VERIFY (rect.x0() == 0);   VERIFY (rect.y0() == 0);   VERIFY (rect.width() == 0);   VERIFY (rect.height() == 0);   ... } 

Your unit test file contains one or more UTFUNC() functions as well as a main() function. If you want to include any custom pre- or post-processing, you can do so, as follows:

 int main() {   // Add any pre-processing here   bool state = apUnitTest::gOnly().run ();   // Add any post-processing here   apUnitTest::gOnly().dumpResults (std::cout);   return state; } 

apUnitTest is a Singleton object that contains a list of all unit tests to run. The results for each unit test are stored internally and can be displayed when dumpResults() is called. Unit test functions should not generate any output on their own, unless that is the point of the test. Any extra input/output can skew the execution time measurements.

7.1.2 Design of the Unit Test Framework

Figure 7.1 illustrates the overall design of the unit test framework.

Figure 7.1. Unit Test Framework Design

graphics/07fig01.gif

There is a base class, apUnitTestFunction , from which unit tests are derived using the UTFUNC() macro. There is a unit test framework object, apUnitTest , that maintains a list of all the unit tests, runs them, and displays the results. Each of these components is described in this section.

apUnitTestFunction Base Class

Each unit test is derived from the apUnitTestFunction base class using the UTFUNC() macro. The complete apUnitTestFunction base class is shown below.

 class apUnitTestFunction { public:   apUnitTestFunction (const std::string& name);   enum eResult {eNotRun, eRunning, eUnknown, eSuccess, eFailure};   const std::string& name         () const { return name_;}   eResult            result       () const { return result_;}   double             elapsed      () const { return elapsed_;}   const std::string& message      () const { return message_;}   const std::string& description  () const   { return description_;}   std::string        resultString () const;   void setDescription (const std::string& s) { description_ = s;}   void run (bool verbose = false);   // Run this unit test. Called by the unit test framework protected:   virtual void test() = 0;   // All unit tests define this function to perform a single test   bool verify (bool state, const std::string& message="");   // Fails test if state is false. Used by VERIFY() macro   void addMessage (const std::string& message);   // Adds the message string to our messages   bool         verbose_;     // true for verbose output   eResult      result_;      // Result of this unit test   std::string  name_;        // Unit test name (must be unique)   std::string  description_; // Description of function   std::string  message_;     // Message, usual a failure message   double       elapsed_;     // Execution time, in seconds }; 

The run() method runs a single unit test, measures its execution time, creates a catch handler to deal with any unexpected exceptions, and determines the result. Note that the actual unit test is defined within the test() method.

The implementation of the run() method is shown here.

 void apUnitTestFunction::run () {   std::string error;   apElapsedTime time;   try {     test ();   }   catch (const std::exception& ex) {     // We caught an STL exception     error = std::string("Exception '") + ex.what() + "' caught";     addMessage (error);     result_ = eFailure;   }   catch (...) {     // We caught an unknown exception     error = "Unknown exception caught";     addMessage (error);     result_ = eFailure;   }   elapsed_ = time.sec ();   // Make sure the test() function set a result or set eUnknown   if (result_ != eSuccess && result_ != eFailure)     result_ = eUnknown; } 

Note that the source code also includes a verbose mode to display immediate results, which we have removed for the sake of brevity.

apUnitTest Object

Our unit test framework object, apUnitTestObject , maintains a list of unit tests, runs all of the unit tests in order, and displays the results of those tests. Its definition is shown here.

 class apUnitTest { public:   static apUnitTest& gOnly ();   bool run (bool verbose = false);   // Run all the unit tests. Returns true if all tests are ok   void dumpResults (std::ostream& out);   // Dump results to specified stream   int size () const { return static_cast<int>(tests_.size());}   const apUnitTestFunction* retrieve (int index) const;   // Retrieves the specific test, or NULL if invalid index   void addTest (const std::string& name,                 apUnitTestFunction* test);   // Used by our macro to add another unit test private:   apUnitTest ();   static apUnitTest* sOnly_;  // Points to our only instance   std::vector<apUnitTestFunction*> tests_; // Array of tests }; 

A std::vector maintains our list of unit tests. The run() method steps through the list, in order, and executes all the unit tests, as shown.

 bool apUnitTest::run () {   bool state = true;   for (unsigned int i=0; i<tests_.size(); i++) {     apUnitTestFunction* test = tests_[i];     test->run ();     if (test->result() != apUnitTestFunction::eSuccess)       state = false;   }   return state; } 

graphics/triangle.gif EXAMPLE

In this example, we look at the output for the apBString unit test. (Note that we include the complete source code for the apBString unit test on the CD-ROM.) Running the unit test produces the following output:

[View full width]
 
[View full width]
Unit Test started at Thu Jul 04 23:39:46 2002 Unit Test finished at Thu Jul 04 23:39:46 2002 Test 1: Success : ctor : Constructor and simple accessor tests: graphics/ccc.gif 0 sec Test 2: Success : Pel8 : Pel8 tests : 0 sec Test 3: Success : Pel16 : Pel16 tests : 0 sec Test 4: Success : Pel32 : Pel32 tests : 0 sec Test 5: Success : Pel32s : Pel32s tests : 0 sec Test 6: Success : float : float tests : 0 sec Test 7: Success : double : double tests : 0 sec Test 8: Success : string : string tests : 0 sec Test 9: Success : eof : eof tests : 0 sec Test 10: Success : data : data tests : 0 sec Test 11: Success : bstr : bstr tests : 0 sec Test 12: Success : dump : dump tests : 0 sec Test 13: Success : point : point tests : 0 sec Test 14: Success : rect : rect tests : 0 sec Passed: 14, Failed: 0, Other: 0

The execution times are all reported as because each test is very simple. This unit test framework is portable across many platforms and the results are similar on each platform.

We can simulate a failure by adding a simple unit test function to our framework, as shown:

 UTFUNC (failing) {   setDescription ("Always will fail");   VERIFY (1 == 2); } 

The output would include these additional lines:

 Test 15: ***** Failure ***** : failing : Always will fail : 0 sec   Messages: 1 == 2 Passed: 14, Failed: 1, Other: 0 

Notice that the conditional is included as part of the failure message.

Macros in the Unit Test Framework

The unit test framework uses two macros: UTFUNC() and VERIFY() . In general, we tend to avoid macros; however, they are very useful in our unit test framework. Figure 7.2 provides a quick overview of the syntax used in macros.

Figure 7.2. Overview of Macro Syntax

graphics/07fig02.gif

Note that parameters used in macros are not checked for syntax; rather, they are treated as plain text. Parameters can contain anything, even unbalanced braces. This can result in very obscure error messages that are difficult to resolve.

The UTFUNC() macro creates a unit test function of the specified name by deriving an object from the apUnitTestFunction base class. UTFUNC() is defined as follows:

 #define UTFUNC(utx)                            \ class UT##utx : public apUnitTestFunction      \ {                                              \ UT##utx ();                                    \ static UT##utx sInstance;                      \ void test ();                                  \ };                                             \ UT##utx UT##utx::sInstance;                    \ UT##utx::UT##utx () : apUnitTestFunction(#utx) \ {                                              \   apUnitTest::gOnly().addTest(#utx,this);      \ }                                              \ void UT##utx::test () 

For example, the preprocessor expands the UTFUNC(rect) macro into the following code:

 class UTrect : public apUnitTestFunction { UTrect (); static UTrect sInstance; void test (); }; UTrect UTrect::sInstance; UTrect::UTrect () : apUnitTestFunction("rect") {   apUnitTest::gOnly().addTest("rect",this); } void UTrect::test () 

Every unit test function creates a new object with one static instance. These objects are constructed during static initialization, and automatically call addTest() to add themselves to the list of unit test functions to be run. Note that the last line of the expanded macro is the test() method, and your unit function becomes its definition.

The VERIFY() macro is much simpler than UTFUNC() . It verifies that a specified condition is true. Its definition is as follows:

 #define VERIFY(condition) verify (condition, #condition) 

Let's look at the following example:

 VERIFY (rect.x0() == 0); 

The preprocessor expands this macro into the following code:

 verify(rect.x0() == 0, "rect.x0() == 0"); 

The VERIFY() macro calls a verify() method that is defined in the apUnitTestFunction base class, as shown.

 bool apUnitTestFunction::verify (bool state,                                  const std::string& message) {   if (!state) {     result_ = eFailure;     addMessage (message);     if (verbose_)       std::cout << " FAILURE " << name_.c_str() << " : "                 << message.c_str() << std::endl;   }   else if (result_ != eFailure)     // Make sure we mark the unit test success, if possible.     result_ = eSuccess;   return state; } 

state is the result of the conditional expression. If the result is false , a failure message, including the string of the conditional, is written to an internal log. This failure message is displayed after all of the unit tests have been run.

setDescription() is a method that lets you include more descriptive information about the test. It is very useful if you have a number of tests and wish to clarify what they do.

7.1.3 Extending the Unit Test Framework

The unit test framework that we have included is only a beginning of a complete solution. We recommend using this framework as a basis to construct a fully automated unit test framework that runs unit tests at regular intervals, such as each night. An automated framework would do the following:

  • Obtain the most recent sources for your application. This involves obtaining a copy of the sources from whatever configuration management package you use.

  • Compile and build any intermediate libraries.

  • Compile each unit test.

  • Execute all unit tests, capture the results, and record which unit tests fail.

  • Generate a report, including a summary or detailed information about the tests.

  • Email the report to the appropriate software developers and engineers .

Our experience is that at least half of all unit test failures are not actually failures in the objects being tested. Failures tend to occur when an object has been modified, but the unit test lags behind.

graphics/star.gif

When updating your code, update the unit test at the same time.




Applied C++
Applied C++: Practical Techniques for Building Better Software
ISBN: 0321108949
EAN: 2147483647
Year: 2003
Pages: 59

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net