Load and Stress Testing

Load and stress tests are part of the process of validating that a system meets its nonfunctional requirements, which typically define such operational attributes for the system as performance, reliability, scalability, and robustness.

Load testing validates the system's performance in terms of handling a specified number of users while maintaining a defined transaction rate. Stress testing assesses a system's reliability and robustness when its design load is exceeded. Although the system may decline some requests when overloaded, it should still have the resilience to keep functioning without suffering a potentially embarrassing outage.

Load tests also test for system scalability. This testing takes the form of proving that through an increase in processing power, whether by adding processors or additional machines to the production environment, the system can scale to keep pace with the anticipated increase in load.

The expected system performance criteria can be detailed precisely in a number of ways but are typically stated as a required transaction rate with the system under a stated load. For example, the requirements could state that a request must be handled within two seconds when 10,000 users are accessing the system. For contractual reasons, the specification of load criteria must be highly detailed, taking into account such operational elements as environment, database size, network speed, and hardware configuration, to name but a few.

Warning

Be wary of vague performance criteria, such as "Requests must be handled in a timely manner." Such statements are impossible to quantify and hence impossible to accurately test. Remove any ambiguity and make sure you have hard figures for defining the performance measures your system must meet.

Performance Concerns

A key failing on many projects is that performance testing is not carried out until near the end of the project. If performance problems are not detected until later iterations in the project, drastic changes to the system's underlying architecture may be necessary. Last minute efforts to correct performance-related problems put the quality and stability of the finished application at great risk. Moreover, such late changes could result in unacceptable delays in the delivery of the system.

A good test strategy combined with a software architecture that considers performance upfront in the development process guards against this danger.

You should use early project iterations for building exploratory prototypes that seek to validate the ability of the software architecture to meet all nonfunctional requirements. Tests constructed against such prototypes validate the architecture. Furthermore, these same tests remain available throughout the development of the system to ensure compliance with performance objectives as the application evolves.

Testing of this nature requires a suitable test automation tool. You can develop your own load-testing tool by building a framework to execute your automated functional tests multiple times from independently executing threads. Fortunately, the Apache Software Foundation has already developed such a tool in the form of JMeter.