What Software Testing Isand Isn t

What Software Testing Is and Isn't

Informally, software testing (or just "testing" in the context of this book) is the process of uncovering evidence of defects in software systems. A defect can be introduced during any phase of development or maintenance and results from one or more "bugs" mistakes, misunderstandings, omissions, or even misguided intent on the part of the developers. Testing comprises the efforts to find defects. Testing does not include efforts associated with tracking down bugs and fixing them. In other words, testing does not include the debugging or repair of bugs.^[1]

^[1] We recognize that some people who test software are also responsible for debugging that software. This is particularly true during unit testing and integration testing. However, we distinguish between the two activities. Testing is the process of finding failures. Debugging is the process of tracking down the source of failures bugs and making repairs. There can be overlap in the sense that testing can sometimes be structured to help locate bugs. However, testing and debugging are two separate activities.

Testing is important because it substantially contributes to ensuring that a software application does everything it is supposed to do. Some testing efforts extend the focus to ensure an application does nothing more than it is supposed to do.^[2] In any case, testing makes a significant contribution to guarding users against software failures that can result in a loss of time, property, customers, or life.

^[2] Certainly this is important in systems in which "enhancements" threaten life or property. However, testing for additional functionality is hard to do without reading code, which few testers ever do. Without reading code, a tester has to anticipate mistakes and enhancements that a developer might make and then develop tests to detect them. Consider, for example, the challenge of detecting Easter eggs hidden in software.

What is software? We define software as the instruction codes and data necessary to accomplish some task on a computer. We also include all representations of those instructions and data. In particular, representations include not only program source code and data files, but models created during analysis and design activities. Software can and should be tested in all its representations. Just as architects and builders can examine blueprints for a new building to spot problems even before ground is broken, so we can examine analysis and design models for software before the first line of program source code is written. We will show how you can test these models using a form of "execution."

Testing helps ensure that a product meets requirements, but testing is not quality assurance. Some people mistakenly equate testing and quality assurance. In many organizations, QA is typically responsible for developing test plans and performing system testing. QA might monitor testing during development and keep statistics. Testing is a necessary but insufficient part of any quality assurance process. Quality assurance addresses activities designed to prevent defects as well as to remove those defects that do creep into the product. A project's quality assurance group sets standards that project members should follow in order to build better software. This includes defining the types of documents that should be created to capture design decisions, the processes that guide project activities, and the measures that quantify the results of decisions.

No amount of testing will improve the quality of a computer program. Testing helps in identifying failures so that developers can find bugs and remove them. The more testing we do of a system, the more convinced we might be of its correctness. Yet testing cannot in general prove a system works 100% correctly. Thus, testing's primary contribution to quality is to identify problems that we wish we could have prevented in the first place. The mission of QA is to prevent problems in the first place. That requires processes beyond testing.

Testing can contribute to improved quality by helping to identify problems early in the development process. Fortunately, you can do some testing quite early in the development process even before code is written. We describe useful techniques in this book, but these techniques require that testers work more closely with developers and that developers work more closely with testers.