Definitions

Let's briefly define some of the concepts we'll discuss in this chapter:

• Unit tests
  Unit tests test a single unit of functionality. In Java, this is often a single class. Unit tests are the finest level of granularity in testing, and should test that each method in a class satisfies its documented contract.

• Test coverage
  This refers to the proportion of application code that is tested (usually, by unit tests). For example, we might aim to check that every line of code is executed by at least one test, or that every logical branch in the code is tested.

• Black-box testing
  This considers only the public interfaces of classes under test. It is not based on knowledge of implementation details.

• White-box testing
  Testing that is aware of the internals of classes under test. In a Java context, white-box testing considers private and protected data and methods. It doesn't merely test whether the class does what is required of it; it also tests how it does it. I don't advocate white-box testing (more of this later). White-box testing is sometimes called "glass-box testing".

• Regression tests
  These establish that, following changes or additions, code still does what it did before. Given adequate coverage, unit tests can serve as regression tests.

• Boundary-value tests
  These test unusual or extreme situations that code under test should be able to handle (for example, unexpected null arguments to a method).

• Acceptance tests (sometimes called Functional tests)

  These are tests from a customer's viewpoint. An acceptance test is concerned with how the application meets business requirements. While unit tests test how each part of an application does its job, acceptance tests ignore the implementation details and test the ultimate functionality, using concepts that make sense to a user (or customer, in XP terminology).

• Load tests
  These test an application's behavior as load increases (for example, to simulate a greater population of users). The aim of load testing is to prove that the application can cope with the load it is expected to encounter in production and to establish the maximum load it can support. Load tests will often be run over long periods of time, to test stability. Load testing may uncover concurrency issues. Throughput targets are an important part of an application's non-functional requirements and should be defined as part of business requirements.

• Stress tests
  These go beyond load testing to increase load on the application beyond the projected limits. The aim is not to simulate expected load, but to cause the application to fail or exhibit unacceptable response times, thus demonstrating its weak links from the point of view of throughput and stability. This can suggest improvements in design or code and establish whedier overloading the application can lead to erroneous behavior such as loss of data or crashing.