Defining the Test Scope

After determining the peak loading and throughput, we start designing the performance test. A good performance test, as we mentioned earlier, tells us how to build the web site to support peak loading. It also tells us how to prepare the web site for future growth. To make these preparations , we need detailed information on how key components of the site, such as the servers, databases, and networks, perform under increasing load.

The test requires a starting point. Even for the most complex web sites, the best tests begin with a single server. Determining the capacity of just one machine tells us the following:

• The accuracy of capacity estimates on a per server basis

• The database and other back-end resources required by one server

• The network capacity consumed by this server

• Front-end resources such as HTTP server and router capacity consumed to support this server

For example, if we estimate we need ten servers to support the 100,000 user day from the previous section, let's validate this estimate early in the performance test. Begin the validation process by determining how much load each server can support. (If you want to learn how to actually estimate the equipment required by your web site, see Chapter 15.)

The estimates calculated previously from this loading estimate show a peak load of 5,000 users over a ten-minute period. These users request 42 pages/second at peak, which translates to 252 requests /second to the web site. The web site requires 50% headroom (double the capacity it needs for the projected peak). This gives the site a generous margin for error in case the initial capacity estimates are wrong, and it also gives the site capacity in case of a pathological surge in traffic caused by some unforeseen event.

Given this, we estimate each server handles 8.4 pages/second or 50.4 HTTP requests/second:

 (42 pages/second * 2 headroom cushion) / 10 servers = 8.4 pages/second (252 requests/second * 2 headroom cushion) / 10 servers = 50.4 requests/second 

Likewise, this transaction rate represents the workload generated by a fraction of the peak users. Using a similar calculation, we get an estimate for the number of users supported by a single server. In this case, the test for a single machine simulates the workload generated by 1,000 users:

 (5,000 peak users * 2 headroom cushion) / 10 servers = 1,000 concurrent users/server 

This number represents the maximum number of concurrent users expected per server during peak loading.

Building the Test

After some analysis to develop peak loading estimates, we build the actual test to simulate peak conditions. We want to begin the test with a single server and determine how many requests per second and users the server can support. Based on preliminary pen-and-paper estimates, we expect to get 1,000 users and 50.4 requests/second with this server. Testing is the only way to determine if these estimates are valid.

Setup

In order to simplify this example, let's map each concurrent user to one virtual user. This means we'll need 1,000 virtual users to perform our testing. (Later in the book, we'll discuss some techniques for reducing the number of virtual users required for a test.) Next , let's establish the scenarios these virtual user will execute, as well as the proportion of users assigned each scenario. For our example, let's assume we're testing a financial web site, and we want the following test scenarios and weighting :

• Check portfolio ”50% of users

• Perform trade ”30% of users

• Perform market research ”10% of users

• Setup new account ”10% of users

The next step, of course, is to turn these scenarios into test scripts. At this point, we also need to answer some key questions about the scripts, such as whether they should include think time, cookies, or browser caching. After completing scenario preparation and validating that the performance test scripts execute correctly and without generating errors on the server , the test actually begins against our single server.

Simple-to-Complex Strategy

If at all possible, start with a very simple test configuration, and work outward to more complex scenarios. Consider the test configuration shown in Figure 6.4.

Figure 6.4. An example test environment

Begin with a simple configuration of one application server, one HTTP server, and a database. If possible, start the tests without firewalls and SSL. After resolving bottlenecks in this environment and doing initial tuning on key components, begin increasing complexity by adding components to the test environment incrementally. After adding each new component, retest to determine its impact on performance, tuning as necessary, before adding another component.

By working from the inside of the web site outward, and reducing the complexity of initial tests, we learn

• How the fundamental components of the system perform. If key components such as the web application suffer from severe performance problems or require substantial tuning, we need to find and address these problems as early as possible in the testing. Web applications tend to have the greatest impact on site performance, and they merit careful performance analysis.

• The impact of each component on performance. By knowing the capacity of the base systems, we can measure the impact of additional components, such as a firewall, on the test environment's performance. This also proves very useful in planning for site growth, and for determining problems in production.

The performance test provides validation for capacity planning estimates. It also helps us identify underperforming components in test environment.