Lesson 7: Testing Your Application

Application testing ensures that your application actually performs as specified. When your application is viewed as nearly complete and reasonably stable, debugging and testing should begin in earnest—testing to detect errors, and debugging to fix them. This lesson examines some of the techniques of testing.

As much as possible, testing should be conducted under widely different scenarios that mimic real-life conditions under which an application might have to function. For example, a program designed to run under Microsoft Windows 95 and Microsoft Windows NT should be tested under both operating systems. Whereas developers try to create code that can't be broken, testers concentrate on breaking the code—that is, finding the weak points. Developers who must wear both hats should bring equal zeal to both tasks.

After this lesson, you will be able to:

• Understand terminology pertaining to testing.
• Design an effective test plan.
• Employ common techniques for testing your application.

Estimated lesson time: 15 minutes

Glossary of Testing Terms

Like any rigorous field, testing has a unique lexicon. This section defines some of the terms that pertain to testing, making subsequent discussions in the lesson more meaningful. The glossary is arranged by level, starting with the simplest type of testing and proceeding to more involved testing practices.

• Unit testing verifies a discrete piece of code, such as a loop, a block, a subroutine, or an event. In formal terms, a unit test applies to the smallest piece of code for which a practical test can be conducted.
• Integration testing is the next highest level of testing, and is concerned with confirming that no problems arise from combining units of code into more complex processes. For example, two functions might test successfully as units, but the effects of providing one function with the output of the other should also be considered. Threads must be subjected to stringent integration testing to ensure that they perform as expected when running simultaneously.
• System testing is concerned with the full build of an application. At this level, the emphasis is less on bug hunting and more on checking that the application and its environment interact with each other correctly. The level of testing that would be conducted at this phase would be more systemwide—features such as correct initialization from the registry, performance, unexpected termination of resources, logon failures, and error recovery.
• Stress testing verifies how an application behaves under adverse conditions such as low memory, high network traffic, or insufficient disk space.
• Regression testing is the repetition of previous tests after changes have been made to the source code. The purpose of regression testing is to verify that expected bug fixes were successful, that the application still works as expected, and that new bugs have not been introduced by the code revisions.
• Beta testing is when a developer distributes a pre-release version of an application to a selected group of users. The users are generally chosen for their familiarity with past versions of a product, their willingness to use the new version under a wide variety of conditions, and a demonstrated ability to communicate what they like and dislike about a product.
• User acceptance testing is when a tested version of the specified deliverable is made available to a selected number of users who have already received training in the use of the system. The users chosen to perform the tests are expected to give the system the kind of usage that it will receive in real life.

Developing a Test Plan

A test plan is a written version of the entire test suite for an application. A properly created test plan completely describes all of the testing that needs to be done, and identifies what constitutes the success of any particular test. Test plans should be written to provide directions to someone other than the author on how to test an application. The primary criterion for a successful test plan is whether a different tester can pick up the plan halfway through the project and continue testing.

Custom applications can be effectively tested through three methods:

• In-house testing by the developers and the testing/quality assurance (QA) group
• Beta testing by selected users
• User acceptance testing by selected users

Elements of a Test Plan

A test plan provides a formal basis from which to develop repeatable regression tests. As applications evolve, or as new builds are created during the debug cycle, it is essential that the existing stability of the application as a whole has not been damaged. A test plan also provides a basis from which the test strategy can be inspected and discussed by all interested parties.

A good test plan starts with a description of the application and the functionality to be tested, and is followed by a brief discussion of the test objectives. The plan should include the following elements:

• The objectives of the test exercise.
• A description of how the tests should be performed, explaining the various degrees of reliance that should be made on key testing components such as test scripts, manual checklists, and user involvement.
• A description of the environment in which the tests should occur, including the operating system and, if relevant, its version number. For example, the original release of Windows 95 has slightly different characteristics than the same operating system with Service Pack 1 installed. A test plan might need to consider these differences.
• A listing of the test data that must be made available for the tests to be valid.
• A discussion of any restrictions placed on the test team that might affect the reliability of the test results. For example, if you are testing a system designed for hundreds of users accessing a large central database, it can be impossible for a small organization to simulate this volume of usage.
• A declaration of the relative orders of importance placed on different criteria—for example, your concern for robustness compared to that of performance.
• A list of features that should not be tested, with a commentary explaining your reasons.
• An intended test schedule that identifies milestones. This should tie into the overall project plan.

Test Scenarios

After designing a test plan, the next step is to list each test scenario, using the same breakdown of functionality as presented in the design specification. Each scenario should include:

• A reference to the item to be tested.
• The expected results.
• Commentary that describes how the test results confirm that the item being tested is functioning as expected.

Lesson Summary

This lesson defined some of the common terminology pertaining to software testing, and described how to write an effective test plan. A test plan incorporates detailed information about an intended program of testing procedures including:

• Unit testing for small sections of code.
• Integration testing to verify how code sections operate together.
• System testing to verify the overall stability of the entire program.
• Stress testing to determine how the program reacts to conditions of limited system resources.
• Regression testing to ensure that a revision to the code does not introduce further errors.
• Beta testing to gather user input.
• User acceptance testing to reflect real-life usage.

Small development teams of less than four or five people usually have to serve as both programmers and testers. In larger organizations with more personnel, developers and testers often form two distinct teams. Even in this case, pro-grammers can benefit from the information in this lesson, because the entire development process runs more smoothly when both teams understand each other's work.