Building Transaction Load-Test Scripts and Profiles


Testing transactions is the ultimate goal. You want to understand which loading levels and transaction mixes degrade performance. Identifying possible compliance problems and mitigating them before the service is introduced prevents unpleasant service launches that melt down.

The first step is determining the transactions you will use for testing. They must be representative, reflecting the way customers actually use these services. For example, not every customer who connects to a web site will buy a product, and the transaction mix should reflect that reality rather than have everyone order a product with each transaction. You still might want to have a 100 percent order rate for ultimate stress testing, but having realistic results that assist in managing performance is equally important.

If the service you are testing will actually use an external provider for credit authorization, clearing payments, or Customer Relationship Management (CRM), those behaviors must be included. One issue is that there are fees for these services, and testing large transaction volumes is expensive. There is also the problem of transactions creating data that has to be backed out later after the testing is completedno one wants thousands of products ordered in the test to be actually shipped and paid for.

What is needed instead is easy access to the initiation of the external service. A simple application can simulate receiving the request, waiting a defined time, responding, and then replicating the external activities in a controlled environment. Such simulators of external activities can also introduce errors or timeouts to see how such conditions affect performance under load.

For a new service, you may need some initial guessing because there is no real operational data. Good instrumentation makes better operational data available after deployment. The feedback from actual operations is used to modify the loading assumptions as needed. Future tests will be more accurate because they use more accurate loading characteristics.

After the appropriate transaction set has been identified, the individual transactions must be recorded and prepared for the testing phase.

Testers typically use a transaction recorder to capture real transactions and structure them into scripts that can be replicated to different load generators and executed repeatedly. The key is that the capture should not require any staff effort to effect the conversion to a test script, as most of the early products of this type did.

The script of the recorded transaction is staticit captures one user accessing one set of information, ordering one product, and using one credit card; therefore, it is of limited value. Running the same transaction repeatedly will not offer much insight into the actual operational behavior under a wider variety of loads and transactions. In fact, I know of a team that tested against the same transaction for so long that they unconsciously adapted their code to produce stunning performance for that transaction only.

Variability in the scripts is needed, and good load testing tools must provide this. The variability should be structured in several ways. For example, a file with a list of user names, products, catalog numbers, or other important information could be used as input.

File-driven variability is usually used first because repeating the tests accurately is helpful in the early testing phases. Adding randomly generated transactions is also helpful for testing a wider range of behaviors.

The actual testing process uses the generated scripts to simulate the activity of real customers. The test procedure includes other parameters that govern its operations, including the following:

  • The length of the test

  • The transaction mix

  • The number of virtual users being simulated

  • The iterations of each transaction

  • The ramping characteristicsadding load smoothly or in large discontinuous steps, as an example

  • The inter-step pause (think) times for each virtual user

A complex environment has dozens to hundreds of possible configuration options that can be changed from test to test, and those permutations can quickly get out of hand. However, statistical methods exist, in a discipline known as Design of Experiments (DOE), that make it possible to achieve robust results without resorting to the full range of all permutations. Such methods have been used with great success in a wide variety of scientific, engineering, and quality control disciplines. References and resources about DOE are available on the Web.

Remember that an important goal of load testing is to break the service by forcing it into operational areas in which performance rapidly degrades. The breakdown provides a rich source of data if the test bed is instrumented properly. What you also want to know is where the performance broke. Was it network congestion, server overload, poor application design, the database, or a combination of factors that caused the problem? This information offers one key to alleviating performance problems or at least to pushing the inflection point toward the right, meaning higher loads are sustainable before nonlinear behavior occurs.

The system elements are already instrumented to provide information on loads and exceptions; sometimes the elements can provide all the information needed, by monitoring their internal states during web transaction load testing. At other times, you'll need to drive specialized transactions into a portion of the test bed to determine server, application, network, or database delays.




Practical Service Level Management. Delivering High-Quality Web-Based Services
Practical Service Level Management: Delivering High-Quality Web-Based Services
ISBN: 158705079X
EAN: 2147483647
Year: 2003
Pages: 128

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net