|
A Practitioner's Guide to Software Test Design Authors: Copeland L. Published year: 2003 Pages: 14-15/161 |
 |
Current Challenges
When I ask my students about the challenges they face in testing they typically reply:
-
Not enough time to test properly
-
Too many combinations of inputs to test
-
Not enough time to test well
-
Difficulty in determining the expected results of each test
-
Nonexistent or rapidly changing requirements
-
Not enough time to test thoroughly
-
No training in testing processes
-
No tool support
-
Management that either doesn't understand testing or (apparently) doesn't care about quality
-
Not enough time
This book does not contain "magic pixie dust" that you can use to create additional time, better requirements, or more enlightened management. It does, however, contain techniques that will make you more efficient and effective in your testing by helping you choose and construct test cases that will find substantially more defects than you have in the past while using fewer resources.
Test Cases
To be most effective and efficient, test cases must be designed, not just slapped together. The word "design" has a number of definitions:
-
To conceive or fashion in the mind; invent: design a good reason to attend the STAR testing conference. To formulate a plan for; devise : design a marketing strategy for the new product.
-
To plan out in systematic, usually documented form: design a building; design a test case.
-
To create or contrive for a particular purpose or effect: a game designed to appeal to all ages.
-
To have as a goal or purpose; intend.
-
To create or execute in an artistic or highly skilled manner.
| Key Point |
To be most effective and efficient, test cases must be designed, not just slapped together. |
Each of these definitions applies to good test case design. Regarding test case design, Roger Pressman wrote:
"The design of tests for software and other engineering products can be as challenging as the initial design of the product itself. Yet ... software engineers often treat testing as an afterthought, developing test cases that 'feel right' but have little assurance of being complete. Recalling the objectives of testing, we must design tests that have the highest likelihood of finding the most errors with a minimum amount of time and effort."
Well designed test cases are composed of three parts :
-
Inputs
-
Outputs
-
Order of execution
| Key Point |
Test cases consist of inputs, outputs, and order of execution. |
Inputs
Inputs are commonly thought of as data entered at a keyboard. While that is a significant source of system input, data can come from other sources—data from interfacing systems, data from interfacing devices, data read from files or databases, the state the system is in when the data arrives, and the environment within which the system executes.
Outputs
Outputs have this same variety. Often outputs are thought of as just the data displayed on a computer screen. In addition, data can be sent to interfacing systems and to external devices. Data can be written to files or databases. The state or the environment may be modified by the system's execution.
All of these relevant inputs and outputs are important components of a test case. In test case design, determining the expected outputs is the function of an "oracle."
An oracle is any program, process, or data that provides the test designer with the expected result of a test. Beizer lists five types of oracles:
-
Kiddie Oracles - Just run the program and see what comes out. If it looks about right, it must be right.
-
Regression Test Suites - Run the program and compare the output to the results of the same tests run against a previous version of the program.
-
Validated Data - Run the program and compare the results against a standard such as a table, formula, or other accepted definition of valid output.
-
Purchased Test Suites - Run the program against a standardized test suite that has been previously created and validated. Programs like compilers, Web browsers, and SQL (Structured Query Language) processors are often tested against such suites.
-
Existing Program - Run the program and compare the output to another version of the program.
Order of Execution
There are two styles of test case design regarding order of test execution.
-
Cascading test cases - Test cases may build on each other. For example, the first test case exercises a particular feature of the software and then leaves the system in a state such that the second test case can be executed. In testing a database consider these test cases:
-
Create a record
-
Read the record
-
Update the record
-
Read the record
-
Delete the record
-
Read the deleted record
Each of these tests could be built on the previous tests. The advantage is that each test case is typically smaller and simpler. The disadvantage is that if one test fails, the subsequent tests may be invalid.
-
-
Independent test cases - Each test case is entirely self contained. Tests do not build on each other or require that other tests have been successfully executed. The advantage is that any number of tests can be executed in any order. The disadvantage is that each test tends to be larger and more complex and thus more difficult to design, create, and maintain.
|
A Practitioner's Guide to Software Test Design Authors: Copeland L. Published year: 2003 Pages: 14-15/161 |
|
