Testing Perspective

The testing perspective is a way of looking at any development product and questioning its validity. The person examining work products from this perspective utilizes a thorough investigation of the software and all its representations to identify faults. The search for faults is guided by both systematic thinking and intuitive insights. It is a perspective that makes reviews and inspections just as powerful a tool as execution-based testing. A review will almost never find something that is missing that is, a review typically only seeks to validate what exists and does not systematically search to determine if all things that should be in the software actually are in it. The testing perspective requires that a piece of software demonstrate that it not only performs according to its specification, but performs only to that specification. Thus, a product is tested to determine that it will do what it is supposed to do, and it is also tested to ensure that it does not do what it is not supposed to do.

Inspections, Reviews, and Test Executions

Software testing is typically accomplished by a combination of inspections, reviews, and test executions. The purpose of these activities is to observe failures.

An inspection is an examination of software based on a checklist of typical problems. Most items on a checklist are based on programming language semantics and/or coding conventions for example, ensuring that each program variable is initialized before its first use and that pointers or references have been set to reasonable values before they are used. Modern compilers for object-oriented programming languages can detect many of the problems called out on traditional inspection checklists.

A review is an examination of software with the purpose of finding errors and faults even before the software is executed. Reviews are made in the context of the system being developed and have a deeper interest in the software than do inspections. A review delves into the meaning of each part of a program and whether it is appropriate for meeting some or all of the application's requirements. A review is intended to uncover errors such as missed or misunderstood requirements or faults in a program's logic. Some reviews examine programming details such as whether variable names are well chosen and whether algorithms are as efficient as they could be.

Test execution is testing software in the context of a running program. Through executing the software, a tester tries to determine whether it has the required behavior by giving the program some input and verifying that the resulting output is correct. Among the challenges to testers are identifying suitable inputs, determining correct outputs, and determining how to observe the outputs.

Testing using program execution (versus inspection and review) is the primary focus of this book, although we extend the idea of execution to include not only execution of the software under testing, but a special kind of review that uses the symbolic execution of nonexecutable representations of the system. Recall how we defined software as code and all its representations.

The testing perspective may be adopted by the same person who developed a product undergoing testing or by another person who brings an independent view of the specification and the product. Anyone assigned to test specific work products and every person assigned to a project at one time or another should adopt the testing perspective. We will refer to anyone who adopts this perspective by the title tester. A developer testing his or her own work is a tester, and so is the person who applies the testing perspective full time.

The testing perspective is as follows:

Skeptical: Wants proof of quality.

Objective: Makes no assumptions.

Thorough: Doesn't miss important areas.

Systematic: Searches are reproducible.

In this chapter we discuss aspects of object-oriented technologies using this testing perspective. First, we will review central concepts of object-oriented programming. What features of these concepts affect the testing of software that was developed using them? We will also delineate some assumptions we make in regard to using object-oriented technologies properly. Then we will look at various products of the development process and discuss the potential causes of failures in the software they represent.