Validation and Testing

Many people do not understand why software is so prone to errors and in fact simply accept software bugs as a fact of life. The same people who accept their PC rebooting twice a day would, however, be intolerant of picking up the phone and not receiving dialtone because of a software bug in the telephone company phone switch. One can easily think of other examples of mission critical software, such as control software for nuclear power plant operations or the software in your car's anti-lock brake computer. Even state-of-the art development techniques cannot eliminate 100 percent of software bugs but proper software validation and testing can go a long way toward detecting most bugs before software is released to end users. This section discusses some of the common types of testing performed during the software development life cycle.

Unit Testing

Unit testing is the testing of a single software module, usually developed by a single developer. In most organizations, unit testing is the responsibility of the software developer.

Subsystem and System Testing

An application typically is made up of one or more software modules. System testing will test all software modules in an application. On larger applications, this may be preceded by subsystem testing. One of the focuses of subsystem and system level testing is to test all the interactions between modules.

Black-Box and White-Box Testing

Two different approaches to developing software tests are black-box and white-box test design methods . Black-box test design treats the software system as a "black-box," and doesn't explicitly use knowledge of the internal software structure. Black-box test design is usually described as focusing on testing functional requirements. Synonyms for black-box include: behavioral, functional, opaque -box, and closed-box. White-box test design allows one to peek inside the "box," or software component, and focus specifically on using internal knowledge of the software to guide the selection of test data. Synonyms for white-box include: structural, glass-box, and clear-box.

While black-box and white-box are terms that are still in popular use, many people prefer the terms "behavioral" and "structural." Behavioral test design is slightly different from black-box test design because the use of internal knowledge isn't strictly forbidden, but rather simply discouraged. In practice, it hasn't proven useful to use a single test design method. Many organizations use a mixture of different methods so that they aren't hindered by the limitations of a particular approach.

It is important to understand that these methods are used during the test design phase. The influence of the test design method used is hard to see once the tests are implemented. Note that any level of testing (unit testing, system testing, etc.) can use any test design methods. Unit testing is usually associated with structural test design, simply because the developer designing a unit test is typically the same person who wrote the code. Subsystem and system level tests are more likely to use behavioral test design methods.

Alpha and Beta Testing

Alpha testing refers to internal company testing of an application prior to its external release. Beta testing is typically done by external users prior to official release of the software. In both cases, users exercise the application software for its intended purpose and report back any bugs they may encounter. Many companies have found both alpha and beta testing extremely useful because it allows much more extensive testing than could have ever been accomplished solely by the in-house development and quality assurance teams .

Several years ago, Sun Microsystems began an extensive alpha test program of its Solaris operating system. Literally hundreds of engineers throughout the company who were unrelated to the actual operating system development installed early builds of Solaris, often six months or more before customer release. This testing was so successful that the Solaris group went the next step and began installing weekly alpha release updates on their main file server machine, providing production service to over 400 engineers. It certainly doesn't take long to get bugs fixed in that environment. The Solaris alpha test program was so successful that many other software product groups within Sun are now alpha testing their software on internal engineering desktops and servers.

Beta testing may also have the advantage of providing users early access to new features and thus building or maintaining a customer base in rapidly changing markets such as the Internet. Netscape has certainly been one of the most widespread sponsors of beta test programs. Some of Netscape's web browser products have undergone half a dozen or more beta releases with millions of users. Before widespread use of the Internet such feedback was impossible simply because of distribution issues. Netscape can beta test six releases, one per week, of its software with a million or more Internet downloads for each release. Just to produce and distribute a million CDROMs would take six weeks for most software vendors .

Stress Testing

The purpose of stress testing is to ascertain that application software will meet its design requirements even under full performance loads and under all possible input conditions. Because software performance is so closely tied to system hardware, stress testing most often is accomplished on the actual production hardware, or a replica thereof.

Production Acceptance

After all other testing has been completed, the final step is for software to undergo production acceptance. This is where operations personnel will integrate the software into a production baseline and perform final regression testing. The main purpose of these tests is to document the correct operation of the software in its production environment and address all issues related to its production rollout. A production acceptance process tailored to client-server systems is presented in the book titled Managing The New Enterprise. Chapter 13 presents our own version of production acceptance, the Web-Centric Production Acceptance process, which is specifically tailored to today's web-centric applications.