Who does the testing?

4.3 Who does the testing?

Until recently, the common preference concerning who actually performed the testing favored the independent tester. While this is still valid in some very critical software situations, the definition of independent has been changing for most applications.

On the basis of the concept that everyone is responsible for their own work and that this responsibility also applies to groups, the task of testing is being returned to the developers. This is not to say that programmers should test all their own work, but that the development group is responsible for the quality of the software that they deliver.

A programmer should test only that software for which he or she has sole responsibility. Once the work of more than one person is to be tested, an independent tester, that is someone other than the persons involved, should carry out the testing. Even at this level, though, the testers should come from within the development group responsible for the full set of software. Outside testers are only necessary at the full software system test level when all the developers have an investment in the software.

Unit, module, and most integration testing are the proper tasks of the development organization. This is consistent with total quality concepts and the idea that every person (or in this case organization) is responsible for the quality of his or her own work. The very early testing is in the form of debugging, and as the unit tests cover more of the software, they flow into module tests. Module tests, too, are primarily debugging in nature. Even the initial integration tests can be thought of as advanced debugging, although this is more of an organizational decision than an industrywide convention.

The characteristic of debugging that separates it from rigorous testing is that defects are generally fixed on the spot without much formal change control. At whatever time the organization institutes some level of change control, the testing is usually considered out of the debugging process and into rigorous testing. This is not to say that there is no configuration control up to this point. Configuration control is already in effect on the documentation. Any change that affects the requirements or design must be processed formally to maintain the integrity of the documentation and the system as a whole. Changes that merely fix mistakes in the code can be made with minimum control at this stage, since the only elements involved are individual units or modules, or small groups of two or three closely interfacing modules prior to actual integration.

There should, however, be at least an audit trail of the changes maintained in the UDF. This will be used for error and defect history analysis as the development proceeds. Software quality practitioners should monitor the testing at the unit and module level to be sure that such an audit trail is provided. Software quality practitioners are also an appropriate resource for the error and defect history analysis. Conclusions reached as a result of the analysis should be fed back, as improvements, into the development process.

As the time for full-scale integration and system testing arrives, a test team that is organizationally independent from the producers should take over the testing. Because the goal of the test program is to find defects, the objectivity of an independent test team greatly enhances the quality of the testing. The independent testers will perform the testing tasks all the way to user or acceptance testing. This team is probably the group that produced the formal test program documents. User or acceptance testing should be performed by the users themselves, preferably in the user's environment, to help ensure that the software meets the user's expectations, as well as the officially approved requirements. Table 4.1 suggests appropriate testers for each type of testing described above. As each organization's test program matures, the identification of the testers for each type of test will be based on the organization's experience and testing approach.

Table 4.1: Who Tests What

Type of Testing	Tester
Debugging	Programmer
Unit testing	Programmer
Module (or object) testing	Programmer
Module (or object) integration testing	Third party
Subsystem and system integration testing	Third party
System testing	Developer test team
User/acceptance testing	User test team

Regression tests are conducted by many different persons involved in the SLC. The developers will regressively test changes to modules and subsystems as they make changes in response to trouble reports generated during formal testing or maintenance. The test team will also have occasion to use regression testing as they verify that new modules or subsystems do not adversely affect the system as a whole. Software quality practitioners can even use regressive testing techniques as they perform some of their audit and review tasks.

The software quality practitioner's primary role in the testing process, aside from reviewing and approving the test documents, is to monitor the testing as it progresses. The software quality practitioner will audit the tests against their plans and procedures and report the status of the test program to management. There are added benefits if software quality practitioners have been doing more than just cursory reviews of the documentation as it has been produced. The cross-fertilization of the technical knowledge of the system and the test planning for the system can produce better results in both areas.