Key Performance Metrics Criteria
As a performance analyst, tester, or developer, you must produce a blueprint on how to performance test the Web application to ensure the high-level performance goals are met. If you don t create a performance test plan, you may find out about requirements too late in the SDLC to properly test for them. Using the performance requirements criteria in the section above, you now need to identify key metrics that will be monitored and analyzed during the actual performance test.
TIP
The breakpoint of your Web application can be defined in various ways. Examples include a server that exceeds predefined resource utilization targets, too many server errors, or unacceptable response times due to processing delays.
Key metrics for the performance test include the following:
This may seem like a moot point, and no server errors are acceptable because they result in a bad user experience. However, during stress testing you will probably come across these errors, so you should be prepared to understand why they are occurring and decide whether they will happen in your live production environment, with real users hitting your Web application. For example, often when a stress test first begins and then again when it shuts down, errors are caused by too much load occurring too quickly, or by uncompleted page requests. These errors are caused by your stress test, so you can ignore them because they are unlikely to reoccur in the production environment.
This is an important aspect of performance testing. By identifying this up front, you will be able to determine the maximum allowable level your servers should endure. When performance testing, this will be a key element in determining the maximum load to apply to your Web application. This metric can differ for each Web application and should be documented for support, development, test, and management teams to agree on. For example, you might ramp the Web tier up until we reach 75 percent CPU utilization. At this level we are serving approximately 2,000 users per server, which meets the concurrent user targets identified by our performance requirements. With documentation of these metrics, the support team can monitor the production Web servers looking for spikes that meet or exceed the performance requirement. The support team can begin to scale the Web application up or out to support the increased traffic.
These issues often arise when running extended performance tests. For example, if you execute a stress test for a short period of time you may not find the memory leak or other stability issues that only occur after an extended period of heavy activity. Many times multiple tests can be executed to accomplish different goals. You may want to run a quick one-hour test to determine your Web application s maximum throughput, and then run a weekend-long extended stress test to determine if your Web application can sustain this maximum load.
These will occur in almost every Web application where complex business logic requires coding. The key is to minimize process delays to an acceptable amount of time. It s a good idea to know what s acceptable before performance testing, so you don t waste time escalating an issue to your development team that does not require fixing because it meets performance goals. Examples of processing delay acceptability are shown in Table 2-5 and include stored procedures taking more than 500 milliseconds and any Web page duration (measured by the time taken field in your Web tier logs) taking more than one second to process. Table 2-5 shows an example of a performance metric acceptability table. Your requirements may be different; the key point is to come up with a set of requirements that make sense for your Web application.
Metric | Location | Acceptable Level |
CPU utilization | Performance Monitor | < 75% |
Memory available MB | Performance Monitor | > 128 MB |
Memory pages/second | Performance Monitor | < 2 |
ASP execution time | Performance Monitor | < 1 second |
DB processing delays | SQL Profiler | < 500 milliseconds |
Web tier processing delays | Time Taken field of Web logs | < 1 second |