Additional Resources

cc2e.com/2203

Federal truth-in-advising statutes compel me to disclose that several other books cover testing in more depth than this chapter does. Books that are devoted to testing discuss system and black-box testing, which haven't been discussed in this chapter. They also go into more depth on developer topics. They discuss formal approaches such as causeeffect graphing and the ins and outs of establishing an independent test organization.

Testing

Kaner, Cem, Jack Falk, and Hung Q. Nguyen. Testing Computer Software, 2d ed. New York, NY: John Wiley & Sons, 1999. This is probably the best current book on software testing. It is most applicable to testing applications that will be distributed to a widespread customer base, such as high-volume websites and shrink-wrap applications, but it is also generally useful.

Kaner, Cem, James Bach, and Bret Pettichord. Lessons Learned in Software Testing. New York, NY: John Wiley & Sons, 2002. This book is a good supplement to Testing Computer Software, 2d ed. It's organized into 11 chapters that enumerate 250 lessons learned by the authors.

Tamre, Louise. Introducing Software Testing. Boston, MA: Addison-Wesley, 2002. This is an accessible testing book targeted at developers who need to understand testing. Belying the title, the book goes into some depth on testing details that are useful even to experienced testers.

Whittaker, James A. How to Break Software: A Practical Guide to Testing. Boston, MA: Addison-Wesley, 2002. This book lists 23 attacks testers can use to make software fail and presents examples for each attack using popular software packages. You can use this book as a primary source of information about testing or, because its approach is distinctive, you can use it to supplement other testing books.

Whittaker, James A. "What Is Software Testing? And Why Is It So Hard?" IEEE Software, January 2000, pp. 70 79. This article is a good introduction to software testing issues and explains some of the challenges associated with effectively testing software.

Myers, Glenford J. The Art of Software Testing. New York, NY: John Wiley, 1979. This is the classic book on software testing and is still in print (though quite expensive). The contents of the book are straightforward: A Self-Assessment Test; The Psychology and Economics of Program Testing; Program Inspections, Walkthroughs, and Reviews; Test-Case Design; Class Testing; Higher-Order Testing; Debugging; Test Tools and Other Techniques. It's short (177 pages) and readable. The quiz at the beginning gets you started thinking like a tester and demonstrates how many ways a piece of code can be broken.

Test Scaffolding

Bentley, Jon. "A Small Matter of Programming" in Programming Pearls, 2d ed. Boston, MA: Addison-Wesley, 2000. This essay includes several good examples of test scaffolding.

Mackinnon, Tim, Steve Freeman, and Philip Craig. "Endo-Testing: Unit Testing with Mock Objects," eXtreme Programming and Flexible Processes Software Engineering - XP2000" Conference, 2000. This is the original paper to discuss the use of mock objects to support developer testing.

Thomas, Dave and Andy Hunt. "Mock Objects," IEEE Software, May/June 2002. This is a highly readable introduction to using mock objects to support developer testing.

cc2e.com/2217

http://www.junit.org. This site provides support for developers using JUnit. Similar resources are provided at cppunit.sourceforge.net and nunit.sourceforge.net.

Test First Development

Beck, Kent. Test-Driven Development: By Example. Boston, MA: Addison-Wesley, 2003. Beck describes the ins and outs of "test-driven development," a development approach that's characterized by writing test cases first and then writing the code to satisfy the test cases. Despite Beck's sometimes-evangelical tone, the advice is sound, and the book is short and to the point. The book has an extensive running example with real code.

Relevant Standards

IEEE Std 1008-1987 (R1993), Standard for Software Unit Testing

IEEE Std 829-1998, Standard for Software Test Documentation

IEEE Std 730-2002, Standard for Software Quality Assurance Plans

cc2e.com/2210

Checklist: Test Cases

• Does each requirement that applies to the class or routine have its own test case?

• Does each element from the design that applies to the class or routine have its own test case?

• Has each line of code been tested with at least one test case? Has this been verified by computing the minimum number of tests necessary to exercise each line of code?

• Have all defined-used data-flow paths been tested with at least one test case?

• Has the code been checked for data-flow patterns that are unlikely to be correct, such as defined-defined, defined-exited, and defined-killed?

• Has a list of common errors been used to write test cases to detect errors that have occurred frequently in the past?

• Have all simple boundaries been tested: maximum, minimum, and off-by-one boundaries?

• Have compound boundaries been tested that is, combinations of input data that might result in a computed variable that's too small or too large?

• Do test cases check for the wrong kind of data for example, a negative number of employees in a payroll program?

• Are representative, middle-of-the-road values tested?

• Is the minimum normal configuration tested?

• Is the maximum normal configuration tested?

• Is compatibility with old data tested? And are old hardware, old versions of the operating system, and interfaces with old versions of other software tested?

• Do the test cases make hand-checks easy?