Test-to-Pass and Test-to-Fail


There are two fundamental approaches to testing software: test-to-pass and test-to-fail. When you test-to-pass, you really assure only that the software minimally works. You don't push its capabilities. You don't see what you can do to break it. You treat it with kid gloves, applying the simplest and most straightforward test cases.

You may be thinking that if your goal is to find bugs, why would you test-to-pass? Wouldn't you want to find bugs by any means possible? The answer is no, not initially.

Think about an analogy with a newly designed car (see Figure 5.2). You're assigned to test the very first prototype that has just rolled off the assembly line and has never been driven. You probably wouldn't get in, start it up, head for the test track, and run it wide open at full speed as hard as you could. You'd probably crash and die. With a new car, there'd be all kinds of bugs that would reveal themselves at low speed under normal driving conditions. Maybe the tires aren't the right size, or the brakes are inadequate, or the engine is too loud. You could discover these problems and have them fixed before getting on the track and pushing the limits.

Figure 5.2. Use test-to-pass to reveal bugs before you test-to-fail.


NOTE

When designing and running your test cases, always run the test-to-pass cases first. It's important to see if the software fundamentally works before you throw the kitchen sink at it. You might be surprised how many bugs you find just using the software normally.


After you assure yourself that the software does what it's specified to do in ordinary circumstances, it's time to put on your sneaky, conniving, devious hat and attempt to find bugs by trying things that should force them out. Designing and running test cases with the sole purpose of breaking the software is called testing-to-fail or error-forcing. You'll learn later in this chapter that test-to-fail cases often don't appear intimidating. They often look like test-to-pass cases, but they're strategically chosen to probe for common weaknesses in the software.

ERROR MESSAGES: TEST-TO-PASS OR TEST-TO-FAIL

A common class of test cases is one that attempts to force error messages. You know the oneslike saving a file to a floppy disk but not having one inserted in the drive. These cases actually straddle the line between test-to-pass and test-to-fail. The specification probably states that certain input conditions should result in an error message. That seems pretty clear as a test-to-pass case. But, you're also forcing an error, so it could be viewed as test-to-fail. In the end, it's probably both.

Don't worry about the distinction. What's important is to try to force the error messages that are specified and to invent test cases to force errors that were never considered. You'll likely end up finding both test-to-pass and test-to-fail bugs.




    Software Testing
    Lessons Learned in Software Testing
    ISBN: 0471081124
    EAN: 2147483647
    Year: 2005
    Pages: 233

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net