Pet Store OverviewImportant! We do not recommend Pet Store as a performance benchmark application for running under load conditions. See Sun's Java web site for suitable performance benchmarks, such as ECperf. [2] However, Pet Store contains classic e-Commerce functions such as Browse, Search, Sign-in, Add-to-Cart, Checkout, and Update Account, and it provides a good example for discussing test script concepts. The home page for Pet Store, shown in Figure 7.1, contains the following choices:
Figure 7.1. Java Pet Store demo home page. Sun Microsystems 2002. Reprinted by permission from Sun Microsystems.
If the user wants to browse and chooses one of the categories, a list of pets in this category returns. Similarly, if the user selects Search, a list of pets containing the search attribute returns. Figure 7.2 shows the high-level browse and search hierarchy for Pet Store. As shown in this diagram, the browse for Fish displays four fish, Angelfish, Goldfish, Koi, and Tiger Shark, whereas the browse for Reptiles only returns two reptiles, Iguana and Rattlesnake. Search returns a variable number of pets, depending on the search criteria entered. Figure 7.2. Java Pet Store hierarchy
Pet Store functions resemble those of many e-Commerce applications. In addition to the Browse and Search capabilities, the application provides the capability to make purchases, which requires a log-on (called Sign-in by Pet Store). A purchase requires standard information such as address, credit card, and number of items. With this background on how Pet Store works, let's start developing test scripts for a web site. Determining User BehaviorAs mentioned previously, the best starting point in developing test scripts is analysis of an existing web site's traffic patterns. Your HTTP server typically generates logs of all requests made through it. For example, Listing 7.1 shows a subset of an access log from IBM HTTP Server taken from a run of the Pet Store web application. The HTTP server logs URL requests , including any parameters. [3]
The first log entry references a query on the Fish category. A bit further down, the log contains an entry for a purchase of an item labeled EST-3. By examining the logs, we begin to see the activities the users typically perform against the web site, as well as the ratio of one type of activity compared to another. Listing 7.1 also shows some of the difficulty in examining logs. This log represents a very small snippet of activity, yet it is hard to identify particular users or to correlate static requests with dynamic requests. Analysis tools exist to automatically scan logs and organize their information into usage patterns. These tools often prove useful when trying to extract this information for large web sites. Listing 7.1 Sample HTTP server log127.0.0.1 - - [28/Dec/2001:17:26:30 -0500] "GET /estore/control/category?cate- gory_id=FISH HTTP/1.0" 200 6467 127.0.0.1 - - [28/Dec/2001:17:26:34 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:26:34 -0500] "GET /estore/control/prod- uct?product_id=FI-SW-02 HTTP/1.0" 200 6388 127.0.0.1 - - [28/Dec/2001:17:26:34 -0500] "GET /estore/images/button_cart-add.gif HTTP/1.0" 200 481 127.0.0.1 - - [28/Dec/2001:17:26:38 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:27:00 -0500] "GET /estore/control/productde- tails?item_id=EST-3 HTTP/1.0" 200 6038 127.0.0.1 - - [28/Dec/2001:17:27:00 -0500] "GET /estore/images/fish4.gif HTTP/1.0" 200 7501 127.0.0.1 - - [28/Dec/2001:17:27:04 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:27:04 -0500] "GET /estore/control/cart? action=purchaseItem&itemId=EST-3 HTTP/1.0" 200 7760 127.0.0.1 - - [28/Dec/2001:17:27:04 -0500] "GET /estore/images/button_checkout.gif HTTP/1.0" 200 534 127.0.0.1 - - [28/Dec/2001:17:27:04 -0500] "GET /estore/images/button_remove.gif HTTP/1.0" 200 1189 127.0.0.1 - - [28/Dec/2001:17:27:04 -0500] "GET /estore/images/cart- update.gif HTTP/1.0" 200 489 127.0.0.1 - - [28/Dec/2001:17:27:16 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:27:16 -0500] "GET /estore/control/checkout HTTP/1.0" 200 6981 127.0.0.1 - - [28/Dec/2001:17:27:16 -0500] "GET /estore/images/button_cont.gif HTTP/1.0" 200 1275 127.0.0.1 - - [28/Dec/2001:17:27:18 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:27:18 -0500] "GET /estore/control/placeor- der HTTP/1.0" 200 6617 127.0.0.1 - - [28/Dec/2001:17:27:19 -0500] "GET /estore/images/button_submit.gif HTTP/1.0" 200 455 127.0.0.1 - - [28/Dec/2001:17:27:24 -0500] "GET /estore/control/white HTTP/1.0" 200 0 127.0.0.1 - - [28/Dec/2001:17:27:24 -0500] "POST /estore/control/verify- signin HTTP/1.0" 200 10038 127.0.0.1 - - [28/Dec/2001:17:27:29 -0500] "GET /estore/control/white HTTP/1.0" 200 0 The more realistic the scenario, the better the test scripts. Pulling data from logs generally gives you the best understanding of how users interact with an existing web site. (Of course, don't forget to account for new features or functions in your web site test scenarios.) Log analysis works well for existing web sites; it proves less useful for new or significantly modified web sites. In these cases, the results of usability testing often provide better scenario data. Also keep in mind that the smallest details matter in building a scenario: Does the user enter the web site through the home page or bookmark another entry point? Does the user log out of the web site or just leave for another site? Accurately representing your users in your test scripts influences the throughput rate and user loading that your test achieves. This in turn corresponds to how accurately you portray your web site's production performance. A Typical Test ScriptUsing the data on how you expect users to walk through your site, you create test scripts to simulate these paths. A test script represents an execution path through a set of web pages. During the test, this same script is executed over and over again, simulating multiple users. In order to simulate many users executing many different paths through a web site, a collection of scripts is typically used. In this chapter, we call this collection of scripts the test scenario . Using the data on how you expect users to walk through your site, you actually create test scripts to simulate these paths. Typically, we create test scripts by "recording" a user's activities while performing a series of tasks at the web site. Many test tools provide script recording capability. In this chapter, we use two common products, LoadRunner, from Mercury Interactive Corporation, and SilkPerformer V, from Segue Software, Inc., to provide our examples. Chapter 8, Selecting the Right Test Tools, covers test tools and script generation capabilities in more detail, and Appendix C also contains a broader list of vendor tools. A typical test script consists of a variety of things, including URLs, think times, and cookies.
Let's look at an example of a recorded test script. Listing 7.2 shows a subset of a LoadRunner test script for Pet Store. [4] The test script specifies a sequence of two URL requests and also a think time of two seconds between submitting the URL requests. The first URL retrieves the Pet Store home page, and the second URL requests a selection of Fish.
Listing 7.2 Example script. 2002 Mercury Interactive Corporation.//subset of Pet Store browse script //first URL request to Pet Store home page web_url("language", "URL=http://ruthless/estore/control/language?language=English", "TargetFrame=", "Resource=0", "RecContentType=text/html", "Referer=", "Snapshot=t1.inf", "Mode=URL", LAST); //think time lr_think_time( 2 ); //browse for fish web_url("nav-fish.gif", "URL=http://ruthless/estore/control/category?category_id=FISH", "TargetFrame=", "Resource=0", "RecContentType=text/html", "Referer=http://ruthless:80/estore/control/language?language=English", "Snapshot=t2.inf", "Mode=URL", LAST); return 0; } |