Test Scripts Basics

Before going into the complexity of how to make individual test scripts represent multiple users, let's discuss the basic test script design best practices.

Model the Real Users

As we discussed early in this chapter, the closer your test scripts mimic actual user behavior, the more accurate your performance test. Again, use logs and usability data whenever possible to build accurate test scenarios. Failing the availability of this data, consult your marketing team or conduct your own research to understand how your user visits typically flow.

Develop Multiple, Short Scripts

Some script writers try to cover all possible user activities for a site visit with one script. Generally, the best scripts cover about three to five page requests . For example, for an e-Commerce site like Pet Store, your scripts might include a Search script that searches for an item and views it, a Browse script that browses for an item and views it, and a Purchase script that selects an item and adds it to a shopping cart.

Developing short scripts results in more flexibility later in the process when you actually define the test scenarios. Here are a few specific reasons for writing smaller scripts:

• Smaller scripts allow you to assign different weights to different activities on the web site. For example, maybe 70% of the customers use a Browse scenario, while only 30% use the Search scenario. Recording these activities in different scripts allows for greater flexibility in assigning user activity at test execution time.

• Smaller scripts allow you to build more elaborate scenarios for the virtual users. For example, if the marketing team discovers a large number of users use both Search and Browse prior to making a purchase, you easily build this new scenario by combining three smaller ones (Search, Browse, and Purchase).

• Maintenance for larger scripts often proves difficult. For example, if a portion of the web site changes, the entire script often requires regeneration. With smaller scripts, only the scripts using the new feature require regeneration.

Write Atomic Scripts

Along the same lines as developing multiple, short scripts, we also recommend writing atomic scripts. These also allow you more flexibility when defining the test scenarios. For example, don't log in at the beginning of each script and log out at the end. Instead, write separate login and logout functions, and remove these activities from the individual scripts.

Atomic scripts allow you to more readily build new scenarios from existing scripts. For example, using the Pet Store application, you might define a scenario by stringing together the following scripts:

 Sign-in + Browse + Purchase + Search + Purchase + Sign-out 

However, Pet Store only requires a user to sign in just before they purchase something. In this case, atomic scripts allow you to rearrange the scenario as follows :

 Browse + Sign-in + Purchase + Search + Purchase + Sign-out 

If the individual scripts include Sign-in and Sign-out, we cannot rearrange the scenario. When pulling together scripts in this manner, be extra careful about how your load driver handles cookies. Another advantage of keeping the logout script separate is that many customers do not log out of web sites. Creating a separate logout function allows you to build different larger scripts accurately reflecting the actual logout rate of your users.

Develop Primitive Scripts

Use primitive test scripts to exercise different primitive functions of your web site. Primitives start with the simplest path through your web site and become increasingly complex. They allow you to test the paths through each layer of your web site's architecture. For example, start with a simple primitive "pingHTML" script to test your HTTP server performance and a simple "pingServlet" script to test the most basic web application path . If you choose to write primitives, build them to exercise round-trip times to each of your back-end systems, such as servlet to database, servlet to LDAP directory, servlet to Session Bean to Entity Bean, and the like. Figure 7.3 shows an example subset of primitives.

Figure 7.3. Example of primitive scripts. From an IBM internal presentation entitled, "Characterizing WebSphere Performance," by Gennaro Cuomo and Ruth Willenborg. IBM Corp. 1999. Reprinted with permission from IBM Corp.

In developing the primitive scripts, you use subsets of your web application, with perhaps a few special functions added to support primitive testing. For example, the WebSphere Performance Benchmark Sample (Trade2) ^[5] includes a set of web primitives in addition to the application.

^[5] WebSphere Performance Benchmark Sample is available at <http://www14.software.ibm.com/webapp/download/search.jsp?go=y&rs=benchmark>. The package includes application code, primitives, design documentation, and source code.

Primitives prove extremely useful in debugging problems in your web site and isolating performance problems to a particular layer. For example, if pingHTML fails to perform up to expectations, you quickly isolate your performance issues to the HTTP server or preceding web site components (router, network, and so on). Also use primitives when the site is in production to test the health of different components of the web site.

Test primitives prove useful, but they are not mandatory for good testing. You probably don't want to invest in writing these scripts unless you find a need for them. Also, not every web application supports the development of primitives. For example, some web applications use their back-end EJB or database functions to generate each dynamic page. In this case, you cannot use a test case to isolate problems at the servlet level or the EJB/database level without adding primitive function support to your web application.