Section 7.4. Unit Testing
7.4. Unit TestingBefore a build is delivered, the person or program building the software should execute unit tests to verify that each unit functions properly. All code is made up of a set of objects, functions, modules, or other non-trivial units. Each unit is built to perform a certain function. The purpose of unit testing is to create a set of tests for each unit to verify that it performs its function correctly. Each unit test should be automated: it should perform a test without any input or intervention, and should result in a pass (meaning that the test generated the expected results), failure (the results of the test differed from what were expected), or error (meaning the code reached an error condition before the test could pass or fail). Many people require that unit tests have no dependencies on external systems (networks, databases, shared folders, etc.). Automated unit testing is a stepping stone to test-driven development . Test-driven development is a programming technique in which a programmer writes the unit tests before he writes the unit that they verify. By writing the tests first, the programmer ensures that he fully understands the requirements. It also guarantees that the tests will be in place, so that they aren't left until after all of the other programming activities are completed (and then possibly dropped, due to schedule pressure). The main activity in unit testing is creating test cases that verify the software. A test case is a piece of code that verifies one particular behavior of the software. Each test should be able to run without any user input; its only output is whether it passed, failed, or halted due to an error. The test cases for a software project are generally grouped together into suites, where there may be a number of suites that verify the entire software. It's often useful to design the suites so that each one verifies specific units or features; this makes the test cases easier to maintain. The most common (and effective) way for programmers to do unit testing is to use a framework, a piece of software that automatically runs the tests and reports the results. A framework typically allows a programmer to write a set of test cases for each unit. Most frameworks provide an automated system for executing a suite of unit tests and reporting the results. This allows a full battery of unit tests to be executed automatically at any time, with little or no effort. Unit testing frameworks are available for most modern programming languages. The framework usually provides some sort of object model, API, or other language interface that provides test cases with functionality for reporting whether the test passed or failed. Most frameworks allow the programmer to indicate which tests are associated with various units, and to group the test cases into suites. Table 7-9 shows some of the test frameworks available for various languages. (This list is by no means exhaustivethere are many other frameworks available for these and other languages.)
7.4.1. Test All of the Code, Test All of the PossibilitiesThe name "unit test" comes from the fact that each individual unit of code is tested separately. In object-oriented languages like Java, C#, and SmallTalk, the units are objects. In imperative languages like C, the units will correspond to functions or modules; in functional languages like Lisp and SML, the units will generally be functions. (Some languages, like Visual Basic and Perl, can be either imperative or object-oriented.) It takes multiple tests to verify a single unit. The framework will have a way to build suites of test cases and indicate that they correspond to a specific unit. A good test verifies many aspects of the software, including (but not limited to) these attributes:
Unit tests must be able to run within a developer's test environment. Real-time or production resources like databases, data feeds, input files, and user input are not necessarily available to the test. To get around this limitation, a programmer can use a mock object an object that simulates a resource that is unavailable at the time of the test. (It is beyond the scope of this book to describe how to implement mock objects.) 7.4.2. JUnitJUnit is the unit testing framework for Java. It was created by Erich Gamma and Kent Beck, based on Beck's work with SmallTalk. JUnit has been very influential in the world of unit testing; many unit test frameworks are ported from, or based on, JUnit. The test case examples below are JUnit test cases, which are part of an automated suite of unit tests for the FeeCalculation( ) function above. The unit tests allowed the programmers doing a code review of this function to successfully refactor it without injecting defects. After each refactoring, they executed the unit tests. If any of them failed, the programmers tracked down the problem and fixed it. The tests use a few additional commands defined by JUnit to tell the framework whether the unit test passes or fails:
Each assertion can optionally be given a message. In that case, if the test fails, the message is displayed in the test report generated by the framework. In JUnit, a test that completes without failing is considered to have passed. Every test in JUnit must be able to be run independently of every other test, and the tests should be able to be run in any order. The individual tests are grouped together into a test case. Each test is a method in a test case object, which inherits from junit.framework.TestCase. (The above assert commands are inherited from this class.) Each test case can optionally have a setUp( ) function, which sets up any objects or values required for the tests, and a tearDown( ) function, which restores the environment to its condition before the test was run. 7.4.3. Unit Testing ExampleThe examples in this section are the individual test methods from a test case object called testFeeCalculation. There are many tests that would exercise the fee calculation function shown in the Refactoring section above. This example shows six of them. All of them require an instance of the FeeCalculation class, which is set up using this setUp( ) function: public FeeCalculation feeCalculation; public void setUp( ) { feeCalculation = new FeeCalculation( ); }The first test simply verifies that the function has performed its calculation and has generated the right result by comparing the output to a known value, which was calculated by hand using a calculator: public void testTypicalResults( ) { Account accounts[] = new Account[3]; accounts[0] = new Account( ); accounts[0].principal = 35; accounts[0].rate = (float) .04; accounts[0].daysActive = 365; accounts[0].accountType = Account.PREMIUM; accounts[1] = new Account( ); accounts[1].principal = 100; accounts[1].rate = (float) .035; accounts[1].daysActive = 100; accounts[1].accountType = Account.BUDGET; accounts[2] = new Account( ); accounts[2].principal = 50; accounts[2].rate = (float) .04; accounts[2].daysActive = 600; accounts[2].accountType = Account.PREMIUM_PLUS; float result = feeCalculation.calculateFee(accounts); assertEquals(result, (float) 0.060289, (float) 0.00001); }This test passes. The call to feeCalculation( ) with those three accounts returns a value of 0.060289383, which matches the value passed to assertEquals( ) within the specified tolerance of .000001. The assertion does not cause a failure, and the test case completes. It's important to test unexpected input. The programmer may not have expected feeCalculation( ) to receive a set of accounts that contained no premium accounts. So the second test checks for a set of non-premium accounts: public void testNonPremiumAccounts( ) { Account accounts[] = new Account[2]; accounts[0] = new Account( ); accounts[0].principal = 12; accounts[0].rate = (float) .025; accounts[0].daysActive = 100; accounts[0].accountType = Account.BUDGET; accounts[1] = new Account( ); accounts[1].principal = 50; accounts[1].rate = (float) .0265; accounts[1].daysActive = 150; accounts[1].accountType = Account.STANDARD; float result = feeCalculation.calculateFee(accounts); assertEquals(result, 0, 0.0001); }The expected result for this test is 0, and it passes. It's not enough to just test for expected results. A good unit test suite will include tests for boundary conditions , or inputs at the edge of the range of acceptable values. There are many kinds of boundary conditions, including:
A few tests will verify that these boundary conditions are handled as expected. This unit test verifies that calculateFee( ) can handle an account with a zero interest rate: public void testZeroRate( ) { Account accounts[] = new Account[1]; accounts[0] = new Account( ); accounts[0].principal = 1000; accounts[0].rate = (float) 0; accounts[0].daysActive = 100; accounts[0].accountType = Account.PREMIUM; float result = feeCalculation.calculateFee(accounts); assertEquals(result, 0, 0.00001); }This test passes in an account with a negative principal (a calculator was used to come up with the expected result by hand): public void testNegativePrincipal( ) { Account accounts[] = new Account[1]; accounts[0] = new Account( ); accounts[0].principal = -10000; accounts[0].rate = (float) 0.263; accounts[0].daysActive = 100; accounts[0].accountType = Account.PREMIUM; float result = feeCalculation.calculateFee(accounts); assertEquals(result, -9.33265, 0.0001); }In this case, the programmer expects the correct mathematical result to be returned, even though it may not make business sense in this context. Another programmer maintaining the code can see this expectation simply by reading through this unit test. The next test verifies that the software can handle a duplicate reference. feeCalculation( ) takes an array of objects. Even if one of those objects is a duplicate reference of another one in the array, the result should still match the one calculated by hand: public void testDuplicateReference( ) { Account accounts[] = new Account[3]; accounts[0] = new Account( ); accounts[0].principal = 35; accounts[0].rate = (float) .04; accounts[0].daysActive = 365; accounts[0].accountType = Account.PREMIUM; accounts[1] = accounts[0]; accounts[2] = new Account( ); accounts[2].principal = 50; accounts[2].rate = (float) .04; accounts[2].daysActive = 600; accounts[2].accountType = Account.PREMIUM_PLUS; float result = feeCalculation.calculateFee(accounts); assertEquals(result, 0.0781316, 0.000001); }It's also possible to create tests that are expected to fail. The programmer expects calculateFee( ) to choke on one particular boundary conditionbeing passed null instead of an array: public void testNullInput( ) { Account accounts[] = null; float result = feeCalculation.calculateFee(accounts); assertTrue(true); }The assertion assertTrue(true) will never fail. It's included for the benefit of any programmer reading this unit test. It shows that the test is expected to get to this line. Unfortunately, calculateFee throws a NullPointerException error. In this case, that's exactly the behavior that the programmer expects. The unit test can be altered to show that it expects the call to calculateFee( ) to fail: public void testNullInput( ) { Account accounts[] = null; try { float result = feeCalculation.calculateFee(accounts); fail( ); } catch (NullPointerException e) { assertTrue(true); } }The fail( ) assertion is placed after calculateFee( ) to verify that it throws an exception and never executes the next statement. The assertTrue(true) assertion is then used to show that the call is expected to throw a specific error, and the test expects to catch it. These test methods by no means represent an exhaustive test case for the FeeCalculation class. But even this limited set of tests is enough, for instance, to ensure that a refactoring has not broken the behavior of the class. It would not take a programmer much longer to come up with a more exhaustive test case for this example. 7.4.4. Test-Driven DevelopmentTest-driven development means that the unit tests are created before the code is built. Before a programmer begins to build a new object, she must first create the test case that verifies that object. By the time the test case is finished, she has defined all of the expected inputs, outputs, boundary cases, and error conditions for the object, and she has a test case that verifies that it works. As she builds each part of the object, she can run its unit tests to verify the code that has just been built. Many defects found by testers or users have to do with unexpected inputs or error conditions; since the programmer has already planned out all of the ways that the object might fail, she will catch many more of these the first time she builds the software. Test-driven development also helps programmers understand the requirements better. It's possibleand often temptingto begin coding with only a partial understanding of what it is that the code is supposed to do. It is not uncommon for a programmer to "go off half-cocked" and begin coding before really taking the time to understand the behavior of the code. This is understandable; it's more fun to write code than it is to sit and pore through requirements documents. It's even more tempting when there are no requirements documents. If the programmer has to build the software simply based on notes from some conversations and a vague understanding of the scope, it's much more fun to just jump in and start coding! Taking the time to write the unit tests really firms up the requirements in the programmer's mind, and often helps her to see exactly where she is missing information. There are several important benefits of test-driven development . The most obvious one is that it guarantees that unit tests are always written. When a team is under pressure to release, it's very tempting to release the code that seems to work. And when the unit tests are the last thing that the team has to do, a senior manager facing a deadline will often decide to release the build as is. Without the unit tests, the code will have more defects. These defects will have to be fixed later, often after the programmers have moved on to other projects and this code is no longer fresh in their minds. Another important benefit of test-driven development is that the unit tests have a very positive influence on the design of the code and the object model. Many design problems stem from the fact that when an object is built, the programmer makes a decision about the interface that later turns out to make that interface difficult to use. By the time she writes code that uses that interface, the object is already written, and it may be difficult to rewrite it to accommodate the way it really needs to be used. The unit test forces her to start by writing code that uses the object; many poor interface decisions immediately become apparent, before the code for the object is written. A complete suite of unit tests also makes it much easier to refactor the software. The unit test suite can be run after each refactoring, helping the programmers to immediately identify any defects that they might have accidentally injected. Running unit tests after each refactoring removes most of the risk and ensures that the refactoring really does not alter the behavior of the software. Unit testing is an efficient way to build better software. Test-driven development often yields code that has fewer defects than standard development, and many programmers who do test-driven development find that they are able to produce that code more quickly than they had in the past. 7.4.5. Everyone Is Responsible for QualityIn some organizations, there seems to be a growing tension between the programmers and testers. The testers will find an increasing number of defects, which they feel should have been caught before the build was delivered to them. The programmers, on the other hand, start to feel that they aren't responsible for testing of any kindnot even unit testsbecause they feel the testers should catch every possible problem. Project managers who start to sense this tension often feel powerless to do anything about it. In situations like this, automated unit tests can help. Many programmers are confused about exactly what it is that software testers do. All they know is that they deliver a build to the QA team. The QA people run the program and find bugs, which the programmers fix. It is often hard for them to figure out where unit testing ends and functional testing begins. A good project manager will keep an eye out for this confusion, and help to clarify it by making sure that the programmers understand what kinds of testing are expected of them. There are different kinds of testing that serve different purposes. The purpose of unit tests is to verify that the software works exactly as the programmer intended. Software testers, on the other hand, are responsible for verifying that the software meets its requirements (in the SRS) and the needs of the users and stakeholders (in the Vision and Scope Document). Many defects arise when a programmer delivers software that worked as he intended, but did not meet the needs of the users. It's the software tester's job to catch these problems (see Chapter 8). This is why both the programmers and testers must test the softwarethey are looking for different problems. If this distinction is clear to the programmers, they should understand why unit testing is their responsibility. By adopting unit tests and test-driven development, the programmers can develop a very clear picture of exactly what their testing responsibilities are. Before the programmers deliver their code to QA, the code should pass every unit test. This does not necessarily mean that the software does what it's supposed to do, but it does mean that it works well enough that a tester can determine whether it does its intended job. It should make intuitive sense that, since the QA team runs the software as if they were users, they do not have access to the individual units (objects, functions, classes, database queries, modules, etc.) that the programmers create. If these units are broken, it is often possible for those defects to be masked, either in the user interface or elsewhere in the software. Even if those units seem to function properly, there may be defects that will only be found when those units are used together in complex ways. This is what QA engineers dothey simulate complex actions that could be performed by the users in order to find these defects. From this perspective, it is not hard for a programmer to see that only the programmers can test those units, and that those units must be working before the testers can do their jobs. Implementing automated unit tests can ensure that those units work, letting the QA team concentrate on more complex behavior. By helping the programmers understand the line between unit testing and functional testing, and take on the responsibility for unit testing, the project manager can help reduce the tension on the project. 7.4.6. Unit Testing Saves Programming Time and EffortSome project managers find that programmers are resistant to unit tests. Sometimes the programmers resent the assumption that their code isn't perfect, even when there is a history of defects that had to be fixed on previous projects. Other times, they assume that the QA team's job is simply to find the programmers' bugs. But mostly, they don't like spending time writing and running unit tests that won't be delivered in the build. It feels like a waste of time, without adding much value. But in fact, the opposite is true. Most programmers who adopt unit tests find that it actually reduces the total time it takes to build software. It may take time to write the tests up front, but it costs more time to go back later and fix all of the bugs that the unit tests would have caught. One way that unit tests help the programmers deliver better code is by improving the object interfaces or function definitions that the programmers use. Many experienced programmers will recognize the feeling of regret that comes when they start using an object that they built to encapsulate some functionality, only to realize later that they should have designed the interface differently. Sometimes there is functionality that the object needs to provide that the programmer didn't think of; at other times, it may be that the methods are awkward to work with, and could have been laid out better. But the code for the object is already built, and it's too late to fix it. She will just have to work around the awkwardness, even though building the object right from the beginning would have saved her some time and effort. Or she may be frustrated enough to go back and rebuild the object entirely, which costs even more time. This is a common trap that plagues object-oriented designthere's no way to know how easy it is to use an object until code is built that uses it, but there's no way to build that code until the object is done. In practice, test-driven development is an effective way to avoid that trap. The programmer can build a series of simple unit tests before the object is built. If the object is needed, she can build a mock object to simulate it. By the time she is ready to define the interface, she has worked out many of the details and has discovered and avoided the potential problems with the interface before the object is built. Programmers also find that effort is unnecessarily wasted in situations when one person has to use an object that was designed by someone else. Often a programmer finds that the object's interface is not clear. There could be ambiguous function names or variables, or it could be unclear how to use objects that are returned by certain functions. This is another case when unit tests can be very useful. When a programmer consults documentation for an object or an API manual, the first thing she usually looks for is an example of the functionality that she is trying to implement. The unit tests serve the same purpose. When an object comes bundled with a series of unit tests, the programmer can consult them to see how the object was intended to be used. Not only do they provide her with a substantial amount of example code, but they also show her all of the behavior that the object is meant to exhibit, including the errors that it is expected to handle. Note: Additional information, tutorials, and examples of unit testing can be found in Pragmatic Unit Testing by Andrew Hunt and David Thomas (The Pragmatic Bookshelf, 2003). |
EAN: 2147483647
Pages: 122