Building the Performance Team

When we consider building a performance test, we often overlook the people who make the testing successful. Testing requires skills from every component used by your web site throughout all phases of the testing cycle (planning, execution, measuring, analysis, and tuning).

For new web applications with links to existing data or function, it's fairly common to focus exclusively on the newly created function and to ignore what's happening inside the existing components . This myopic approach to testing frequently proves disastrous. Web applications work differently than existing thick client applications: They exert more simultaneous requests , and often use the existing applications differently. In short, legacy applications often require tuning for the web environment.

For example, consider a legacy application used by in-house stock brokers. Customers call the brokers , who then use a thick client application to create orders, retrieve pricing, or check the customer's account. The company decides to build a web application giving customers direct access to these functions over the Internet. However, your new web application uses the same business logic at the mainframe as the old thick client application.

Not surprisingly, this web application generates many more transactions per day than the thick client application. Along with more transactions comes the need for more connections to your back-end database ”which implies potential licensing and performance impacts not only on the new application, but also on those who continue to use the old system.

Obviously, you need to get the team supporting the mainframe business logic involved with your testing. Involving them early in the test planning phase gives them time to understand your performance goals and to do some tuning in preparation for your testing. As you continue the testing, they assist by capturing data at the mainframe and performing additional tuning.

Beyond this one example, consider the peripheral systems involved in your test, and pull together the right people to make the test successful. Below is a list of folks we often find useful as part of the larger test team. Review this list and consider the relationships you need for a thorough and accurate test.

Quality Assurance People

We consider quality assurance people first because they're often overlooked. Some enterprises relegate performance testing to something outside their standard QA processes and only assign the application programmers to the task. This is a mistake.

Performance testing really operates as an extension of QA, assuming that your QA team is in the business of assuring that applications behave satisfactorily when put into production. Many QA groups actually own the performance test. This includes creating the test scripts based on either the design or current customer usage patterns.

Even if QA does not own the test, they often make a good resource for test scenarios, as mentioned above. They also often provide many of the performance targets (such as maximum response time) used in the testing.

Web Application Programmers

Use the application programmers to troubleshoot the application during testing. Problem analysis usually becomes one of the biggest headaches in deploying a distributed application. The test team needs someone intimately familiar with the application's internals to fix problems as they arise.

Web Application Architect

You need the application architect's input to build a good test. For large applications, use the architect's guidance to identify the subset of function and infrastructure required to accurately model the entire application's behavior. The architect also understands how users should interact with the web site. Use this information when designing your test scripts. However, always try to back up this information with observation of actual users interacting with the system.

Finally, the architect knows all the systems, such as databases and content servers, that are interacting with the web application. Use this information both in designing your test system and in resolving any performance problems you encounter during the testing.

System Administrators

System administrators set up and configure the test machines. During the test, they monitor the resource consumption on all the machines in order to identify machines and processes that are acting as bottlenecks.

Network Administrators

Just as your system administrators install, configure, and monitor the servers and clients , the network administrators do the same for the network. They validate your capacity estimates, set up an isolated network (hopefully), and monitor the network during testing.

Systems Analysts

Systems analysts monitor the legacy systems used by your web site. As we discussed earlier, new web applications often drive legacy systems much harder than existing thick client applications. You need a systems analyst to monitor these legacy components during your performance testing. They also provide important information about planned outages and backups of the systems, which you need for scheduling performance tests. Also, they manage the legacy resources and help you schedule your testing so that you do not interfere with existing production applications.

Database Administrators

Almost all web applications connect to databases. Large web sites may connect to a variety of databases on multiple systems, including mid- tier and host systems. Each database used by your web site often has a different DBA or support team. Find these folks, and include them on your team before the testing begins.

Work with the DBAs to configure the database for the traffic you're planning to send its way. In particular, coordinate available database connections to match the size of your web site's database connection pools. During the testing, use the DBAs to monitor the databases for potential performance problems.

The DBAs are very important players on the web site performance team. These folks, along with the systems analyst, have skills you cannot reproduce on your team in the short term . Call on them early to help you understand the issues regarding their systems before a problem occurs.

Legacy Application Programmers

Often web applications use legacy applications in ways unforeseen in the legacy applications' original designs. Having a legacy application programmer assigned as a point of contact and assistance is important. It's important to build teamwork and a sense of ownership for the performance test within this broad group of support personnel. Obtain commitments from this varied group of folks and their management to make your testing successful.