Using the Application Center Test Tool | Performance Tuning and Optimizing ASP.NET Applications

Microsoft's stress testing tool is Application Center Test (ACT). ACT is included in the Visual Studio .NET Enterprise Developer and Architect Editions. ACT stress tests Web servers and analyzes performance and scalability problems with Web applications. ACT is a successor to the original Web Application Stress Tool (WAS), informally referred to as Homer . ACT supports many of the features left out of WAS by providing greater control in the test scripting interface and a more robust feature list.

Understanding ACT Features

ACT provides many features that help to perform capacity planning and stress testing on ASP.NET applications. ACT provides the following distinct features:

Simulates a large group of users by opening multiple connections to the server and rapidly sending Hypertext Transfer Protocol (HTTP) requests .
Supports programmable dynamic testing, which allows for robust functional testing.
Supports several different authentication schemes and the Secure Sockets Layer (SSL) protocol, making it ideal for testing personalized and secure sites.
Tests can be scripted, allowing for reproducible performance testing and tuning.
Compatible with all Web servers and Web applications that adhere to the HTTP protocol.

ACT collects and summarizes all of the response times for each test request. Metrics such as TTFB, TTLB, and RPS are useful in analyzing the throughput and latency of an application. Also, ACT integrates with the system's performance counters to simplify correlation analysis between utilization and the other metrics.

Understanding ACT Test Architecture

To accurately measure performance, you should run the tests in an environment that matches as closely as possible with the production environment. Because many production environments are comprised of server farms, it is unlikely you can duplicate the server numbers in the test environment. The ideal test environment would be to run the tests in production if it does not interfere with business usage. However, it is feasible to gather useful data and optimize an application's performance in a scaled-back environment. Gains in performance on a single server will translate to similar performance gains in a server farm environment.

At a minimum, the Web server and Web client should be on separate machines so that you can isolate the resource utilization to specific tasks . Use ACT on the client machine to simulate HTTP requests and to measure client metrics such as TTLB and RPS. At the same time, use either ACT on the client machine or the Windows Performance Monitor on the Web server to measure the resource utilization.

For all the examples used in this chapter, the hardware is as follows :

The Web server is an IBM PL300 with a Pentium III 733MHz processor, 512MB RAM, and 10GB available hard drive space.
The Web client is an IBM ThinkPad A20M with a Pentium III 700MHz processor, 512MB RAM, and 6GB available hard drive space.

Setting Up a Test

You can create and run ACT tests from two user interfaces:

Visual Studio .NET (VS.NET) shell integration: Enables you to create and run tests and view performance data while developing the application from within VS.NET
Stand-alone user interface: Enables you to create and run tests and provides more in-depth reporting information and graphs

The ACT stand-alone user interface (UI) provides much more detailed reports than the VS.NET integrated version. The test scripts live in project folders that you can open from either interface. To add a test project through the VS.NET integrated development environment (IDE), follow these steps:

From the menu, select File Add Project New Project.
In the Add New Project window, expand the Other Projects node.
Select the ACT Projects node. Available templates will be displayed in the right panel.
Highlight the ACT Project template, set the Name and Location for the project, and click the OK button.

To add a test project with the ACT stand-alone UI, follow these steps:

Start Act.exe from the menu. The default menu path is Programs Microsoft Visual Studio .NET Visual Studio .NET Enterprise Features Microsoft Application Center Test.
Within the UI, select File New Project. A dialog box will display.
Within the dialog box, set the Name and Location for the project and click the OK button.

Recording a Browser Session

The easiest way to set up a test is by recording a browser session. The browser session will be automatically scripted to a VBScript ( .vbs )file.

To record a browser session from the VS IDE, follow these steps:

Right-click the ACT project and select Add New Item.
Select the Browser Recorded Test ( .vbs ) Template from the Add New Item dialog box.
When the browser opens, direct the browser to the Web application and record the sequence of operations to execute.
When finished, click Stop on the recorder dialog box.

To record a browser session from the ACT UI, follow these steps:

From the Actions menu, choose New Test to start the New Test Wizard. Click Next to open the Test Source dialog box.
Select the Record a New Test option, click Next twice to accept the VBScript language default, open the Browser Record dialog box, and click Start Recording to open an instance of Internet Explorer (IE).
Replace "about: blank" with your starting URL in the Address textbox and record the sequence of operations to execute.
Return to the wizard, click Stop Recording, close IE, and click Next.
In the Test Properties dialog box, type the name of the test, click Next and then Finish to dismiss the wizard.

For simplicity, our ACT test script samples in this section will be from the ACT UI, which provides more detailed information than running ACT from the VS.NET IDE. We use the Duwamish 7.0 sample site included in the Visual Studio Samples package as the sample Web application.

To install the Duwamish site, run the self-installation package Duwamish.msi located in C:\Program Files\Microsoft Visual Studio .NET\Enterprise Samples\Duwamish 7.0 VB . A C# version is also available in the Duwamish 7.0 CS directory. This will extract the Web application files as well as the Duwamish SQL database.

In the AspNetChap7ACTTest project, we have created a test script by recording a browser session on the Duwamish site. This session captures a user logging in to the site, adding several books to their shopping cart, and completing the checkout process. Figure 7-6 illustrates how the ACT UI will appear after the session has been recorded.

Figure 7-6: ACT user interface

Note that we have named our test "Login_Select_Checkout_1," as shown below the Tests category. The right panel displays the test script.

Defining a Dynamic Test

Another type of test you can run in ACT is the dynamic test. In the dynamic test, you can script the content and logic for the test in VBScript. To create an empty dynamic test from the VS.NET IDE, follow these steps:

Right-click the ACT project and select Add New Item.
Select the generic Test (.vbs) Template from the Add New Item dialog box.
A new .vbs file will be added to the project under the test project with the initial script line Test.SendRequest("http://localhost") .

To create an empty dynamic test from the ACT UI, follow these steps:

In the New Test Wizard, choose the Create an Empty Test option and then click Next.
Select a scripting language from the list and then click Next.
Type a name in the Test Name box and then click Next.
Click Finish to add the empty test file to the project.

Similar to the layout shown in Figure 7-6, the script for a dynamic test is editable in the large script window on the right portion of the ACT UI.

Scripting the test requests and responses in a dynamic test requires using the Application object and Test object models, which are covered in the "Using the ACT Test Object Model" section.

Saving a Test

Whether the test is a browser-recorded session or a dynamic test, the information required to run the test resides in the test script file. Saving the test script file allows you to reuse the test later. You can copy a self-contained test script from one project to another.

Customizing a Test

After you have created the test script, you can customize it to meet the application's performance requirements. You can access the test properties by right-clicking a test and selecting Properties from the stand-alone user interface. If using the VS.NET integration user interface, the properties of a test are available in the Properties window when a test is selected in Solution Explorer. You can customize the following test properties:

Test load level
Test duration
User groups
Performance counters
Reporting level

Test Load Levels

The test load level controls the number of simultaneous browser connections the tool will employ during the test. This number is equal to the number of copies of the test script that will be running simultaneously during the test. You can equate this number to the number of users simultaneously using the site. You can configure the test load level from the test's Properties dialog box.

To set the test load level, follow these steps:

Open the test's Properties dialog box by right-clicking the test script and selecting Properties.
Select the General tab if it is not enabled.
In the Test Load Level section, set the value for the Simultaneous Browser Connections to the required level for the test.

Figure 7-7 shows the Properties dialog box for the Login_Select_Checkout_1 test script. In the sample, we have set the value to 10, such that ACT will generate 10 simultaneous connections during the test. If more than one Connection object is used in the script, then the actual number of simultaneous connections will be higher than the defined value.

Figure 7-7: Sample Properties dialog box

Test Duration

You can set the duration of a test as a specific length of time or when a specific number of requests have occurred. The test duration section of the test properties allows customization of either parameter to control the duration of a test. Figure 7-7 shows the Test Duration section in the test's Properties dialog box. Depending on the test requirements, select either the time duration or the number of iterations as the test duration setting.

To run the test for a specific length of time, select the Run Test for a Specific Duration option, and enter the length of time in days, hours, minutes, and seconds to run the test. A test duration of one to five minutes is more than adequate in capturing the basic performance features of an application. However, the size and complexity of the test script may require a longer duration to exercise the application completely. Characteristics such as memory or other resource leaks will require a longer duration to reveal.

To run the test for a number of iterations, select the Run Test a Specific Number of Times option, and enter the iteration value. Bear in mind that the number of iterations and the number of simultaneous connections are interrelated. Setting the number of iterations to 5 for this sample would in essence set the test to simulate 10 simultaneous connections five times, or 50 overall connections during the test duration.

Note the Warm Up Time (Seconds) field. This is the setting that instructs ACT to ignore the performance metrics for an initial x number of seconds to account for the warm-up time needed to start the application, initialize components , and set caching. Although this setting is within the Run Test for a Specific Duration section, the Warm Up Time (Seconds) field applies to either test duration setting. A 10-second warm-up time is sufficient for most applications.

Setting User Groups and Users

Most Web applications require distinct user information in their performance testing. You can generate users dynamically by ACT or pull them from a set of predefined users. ACT can automatically generate users during the test on an as-needed basis. Automatically creating users can fulfill the specific load level for a test run. If the predefined user population is small, then potential problems can be generated when the specific load level exceeds this user population. If user authentication is required, or you want to view or reuse the user-specific cookies, you can create users and user groups and then select them in the test's Properties dialog box.

You define a user group on a per-project basis under the Users node in ACT. Highlighting the Users section will display the user groups defined for the project, including the Default Users Group. To add a new user group, right-click Users and select Add. You can assign each user in the group a username, password, and domain. To define a domain, it is necessary to define it in the form of domain\username . Figure 7-8 shows the custom user group SampleUserGroup1 with five defined users.

Figure 7-8: Defining users in a user group

You can add users to a user group with three methods :

Adding a user individually
Generating a set number of users
Importing users from a data file

To add users individually, click each entry in the panel to the right and enter the appropriate data. To generate a set of users, select Actions Generate Users. In the Generate Users dialog box, enter the number of users to generate, the username prefix, and the password for the users. In this sample we want to generate five users with the name prefix of "user" and a password of "pass." Figure 7-9 shows the results of our entries.

Figure 7-9: Generate Users dialog box

To import the users from a data file, select the Import Users command from the Actions menu. Browse to the data file, select it, and then click Open. ACT will parse the file and add the users to the group, along with any password information. ACT ignores parsing errors, so you should verify the resulting list of users for accuracy and completeness.

The format for each line in the data file should be as follows:

 name, password

You must assign a user group to a specific test script to be used during the test run. To assign a test group to a test script, open the test's Properties dialog box and select the Users tab. Then select the appropriate user group and click OK.

Setting Up Performance Monitor Counters

One of ACT's features is the ability to integrate performance counters into its test report data. Select the Counters tab from the test's Properties dialog box to view the counter properties. You can specify how often the counter data is collected within the Counters window. Enter the interval value in the Counter Collection interval box. Keep in mind that a small interval value may reduce the ACT client performance because resources are required to collect the counters.

To add performance counters to a test script, click the Add button to open the Browse Performance Counters dialog box. Use this dialog box to add the appropriate counters to be monitored during the test run. A nice feature of ACT is the ability to monitor counters from remote servers. This allows you to monitor both the client and server performance counters in a single resultset. To add a counter, select the appropriate computer, performance object, counter, and counter instance. Click the Add button to add it to the test. Clicking the Explain button displays a detailed description of the selected counter in the Explain Text dialog box, as shown in Figure 7-10.

Figure 7-10: Browse Performance Counters dialog box

Setting the Reporting Level for a Test

You set the reporting level in the Advanced Settings dialog box, which you open by clicking the Advanced button in the test's Properties dialog box. The reporting level controls the amount of information generated and saved during a test run. Using the less-detailed reporting option will decrease the amount of time it takes to create a report, especially for tests that request a larger number of unique paths.

If detailed results are used, reports will include information for each page that was requested in the test. If detailed results are not used, then average values for the test run will be calculated and saved, but the page-level information will not be stored in the report.

Running a Test

Once a test script and test properties are ready, it is time to run the test. You can run the test manually or as a scheduled task. To run a test manually, select the test script and click Start Test on the toolbar. Alternatively, right-click the test and select the Start Test command.

Once the test starts, you can view the test status as it progresses. The test status window shows a summary of the test results, as illustrated in the upper section of Figure 7-11. Click Details to display more information about the test as it runs (as shown in the lower section of Figure 7-11). The Graph tab displays a colored graph of the RPS, total requests, socket errors, and HTTP errors for the test. The Status tab displays information about when the test has reached a certain stage ”in other words, when the test is initialized and when the test is gathering information at the end of the test.

Figure 7-11: Test Status window

This summary provides an initial analysis of the test to determine if the test is valid or needs to be modified. If the initial metrics show an issue with the test, such as HTTP errors or socket errors, then the test may be invalid or not performing properly. You can stop the test prematurely by clicking the Stop Test button. ACT will then compile the metrics from the test up to its stopping point.

To run a test as a scheduled task, you must use a Windows Script Host file. Because the ACT executable Act.exe does not accept command-line arguments, the script file needs to call the Application object model to programmatically initialize and execute a test script. The script file ScheduleTaskRunScript1.vbs provides an example of how to execute a test script programmatically. You can set up the script as a scheduled task using the Scheduled Task Wizard.

Using the Report Interface

Once the test completes, you can view results of the test in the Results node within ACT. Once you select the Results node, the Test Runs panel displays the test runs related to each test script. In our sample project, we recorded a script to test the performance of the category pages in the Duwamish sample site. Figure 7-12 shows the Overview Report for four test runs of varying simultaneous connections (15, 25, 50, and 75). Expanding the test in the Test Runs panel allows you to view each test run report. The Report panel contains a drop-down list with Overview, Graphs, and Requests. The Overview summary shows many basic performance counters that you can use to get a general idea of the test's performance.

Figure 7-12: Summary results from the sample test

Checking Basic Performance Counters

You need to check some basic performance counters after a test run to verify the test was valid. They include the following:

Web server processor usage: To obtain an upper limit on the application's performance, the Web server's processor use should reach at least 80 percent. If it does not, try increasing the load by increasing the number of simultaneous browser connections.
ACT client processor usage: If it is at maximum usage (near 100 percent) and the Web server is not, the client machine might not be able to create enough load to determine the server's maximum capacity.
HTTP errors: Verify that there are no excess HTTP 403, 404, or 500 errors. Encountering these errors would invalidate the test.

You can compare a test run report against another test run from either the same test or a different test. You can view multiple test runs simultaneously by selecting the check box preceding each run. In Figure 7-12, we have selected the four test runs for viewing.

You can do a basic test analysis from these counters. For example, the metrics for throughput (RPS) and latency (TTLB) are included in the set of basic counters. From Figure 7-12, you can deduce a trend of the TTLB increasing as the number of requests increases (60.37, 60.14, 64.80, and 72.81). You can also see the beginnings of a bottleneck between the third and fourth tests, which has a TTLB increase much greater than that between the second and third tests.

Saving Test Reports

Once you have generated test reports, the test data lives in an Extensible Markup Language (XML) file in the ACT project directory. The name of the file is comprised of the name of the test script and a date timestamp of when the test was run. The XML file format makes it flexible to transform the data into any other custom format. Saving the test project will save the test scripts as well as the test run data.

One of the features of ACT is the ability to create custom graphs from the test data. To generate a custom graph, select Graphs from the Reports drop-down menu. Select a source for the Y-axis from the Source list and a source for the X-axis from the Measured Against drop-down list. This will generate and display the graph using the two selected values.

Two commonly used graphs available in ACT are the latency vs. user load and throughput vs. user load. In ACT you can measure latency as TTLB, throughput as RPS, and user load as the number of simultaneous connections. For the Category_Fiction_History test runs, Figure 7-13 shows the RPS vs. browser connections graph, and Figure 7-14 show the TTLB vs. browser connections graph. Note that the relative curvature of the two graphs is comparable to the theoretical graphs (Figures 7-1 and 7-2) discussed in the "Understanding Performance Testing Concepts" section.

Figure 7-13: Custom graph of RPS vs. browser connections

Figure 7-14: Custom graph of TTLB vs. browser connections

A limitation of the ACT version from VS.NET 1.0 is that any custom graphs will not be saved in the project. You need to set up graphs each time you reopen an ACT project.

Analyzing the Test Results from ACT

ACT provides an abundance of metrics within the test run reports. The real science is analyzing the results to identify key performance issues in the application. Table 7-2 describes the key metrics to consider in the test run reports.

Table 7-2: ACT Test Run Key Metrics
METRIC	MEASURED ON	DESCRIPTION
Browser Connection	Client	Measures user load
Requests per Second	Server	Measures throughput
TTLB	Client	Measures latency
% Processor Time /_Total instance	Server	Measures utilization
Available Bytes (Memory)	Server	Measures utilization

These metrics allow you to quantify your application's performance and determine if there are any performance bottlenecks.

Throughput vs. User Load

Graphing the throughput against the user load provides a useful measurement of the throughput limitations of an application. Take, for example, Figure 7-13, which depicts the graph for the tests run against the category pages in the Duwamish sample site. Although the throughput has not yet degraded, the increase in the level of RPS becomes non-linear as the user load increases. This is a common behavior and is usually associated with beginnings of a performance bottleneck in the application. In this example, analyzing this graph allows you to identify the performance of the category pages and provide an upper limit to its throughput. Graphs such as these provide a good indication of areas to investigate for performance gains.

Latency vs. User Load

Graphing latency against the user load provides a useful measurement of the critical effects of overutilization. In our example, shown in Figure 7-14, the latency increases steadily as the user load increases. At our upper user load limit of 75 simultaneous connections, TTLB still had not reached a critical increase, so you can conclude that the performance limitation on the application will not be latency.

Response Code Results

The test results from our sample test script were from user loads of five up to 75 simultaneous connections. We could not go higher than 75 because we began to encounter a large number of unsuccessful page requests. When this occurs, the metrics can get skewed. The HTTP response code of 200 denotes a successful request. Take, for example, a test where 20 percent of the page requests returned a non-200 error ( unsuccessful request); these request errors could potentially reduce the performance of the Web server and make the test results less accurate. Common error responses include the following:

500 (Internal Server Error): This indicates that the server encountered errors in attempting to fulfill the request. This is usually caused by the application's inability to handle the user load.
404 (Not Found): This indicates Internet Information Server (IIS) is not responding to the request in a timely manner. This is likely caused by IIS not being able to handle the user request because of a tag that references a missing image or source file.
403 (Server Too Busy): This indicates that the server understood the request but is refusing to fulfill it.

These response codes commonly occur when the number of concurrent browsers is too high for the Web server to handle. A good practice is to invalidate any test run that generates page request errors greater than 2.5 percent of the total page requests. In other words, a meaningful test generates a response code of 200 (successful request) for greater than 97.5 percent of the total number of requests.

Utilize Percentiles

ACT provides the test report data in subdivided percentiles. These provide a better measurement of how the response times are distributed, whereas an average can sometimes be misleading. For example, if 99 responses have a TTLB of 50 milliseconds, but just one response in this group took 1 second, the average would be skewed to 59 milliseconds . Percentiles lessen this issue by providing a distribution to represent the metrics of the majority of the request. A percentile represents a value in which a percentage of the requests are greater and the remaining percentage is less. The values within the 25th and 75th percentiles are the best indication of response time. These values represent the response time of the majority of the page requests.

Performance Tuning

The user load tests ran against the Duwamish sample site provide a base performance metric. Reviewing the metrics showed an increasing number of socket errors after the 75 simultaneous connections mark. What you can ascertain from the test results is that the limitation of 75 simultaneous connections is most likely caused by the Web server's hardware capability as it started to overload and reject page requests.

To performance tune the application, you would attempt to increase the capacity of the server, either with more memory or a faster CPU, run the tests again, and take note of the changes in the RPS value, TTFB, and TTLB. You would check the Web server's processor use levels again, and, if necessary, increase the load for the next test run. If the processor use does not increase to 80 percent or more, you may need use performance counters to investigate bottlenecks other than processor speed.

Using the ACT Application Object Model

The ACT Application object model programmatically automates ACT to start and stop tests, analyze reports, and execute other tasks available from the ACT user interface. You can instantiate the objects within the ACT Application object model within scripts using either VBScript or JScript. Table 7-3 describes the major objects in the ACT Application object model.

Table 7-3: Major ACT Application Object Model Objects
OBJECT	DESCRIPTION
Project	Represents an ACT project. The Project object is the main object in the Application object model and provides references to all aspects of the ACT project, including the Tests and Properties collections.
Test	Represents an ACT test script. The Tests collection represents the collection of tests in the ACT project and is a property of the Project object. Each test object represents one of the test scripts in the project.
Controller	The Controller object represents the ACT server and includes methods that allow tests and applications to start and stop any test in an ACT project.

The code to create the Project object in VBScript is as follows:

 Dim oProject Set oProject = CreateObject("ACT.Project")

Table 7-4 summarizes the basic methods for the Project object.

Table 7-4: Project Object Basic Methods
METHOD	DESCRIPTION	CODE SAMPLE
Open()	This method opens the project.	oProject.Open(strPath, strProjectFileName, bPrompt)
Close()	This method closes the project.	oProject.Close

As mentioned earlier, you can set the properties of a project programmatically through the Project Properties collection. Each property is in the format " Component:Name ," where the component string is equal to "Project." The code to reference the Properties collection and to set a property value is as follows:

 Dim oProperties Set oProperties = oProject.Properties oProperties.Item("Project:LogEnable").value = False

Similarly, the same format of " Component:Name " applies for Test Properties, except the component string is equal to "TestProperties." You can find the complete list of project and test properties in the .NET documentation.

The code in VBScript to reference a Test object is as follows:

 Dim oTest Set oTest = oProject.Tests.Item(strTestName)

Table 7-5 summarizes the useful methods and properties for the Controller object.

Table 7-5: Controller Object Main Methods and Properties
METHOD/PROPERTY	DESCRIPTION	CODE SAMPLE
StartTest()	This method starts a test.	oController.StartTest(oProject, oTest, bWaitForCompletion)
StopTest()	This method stops a test.	oController.StopTest
TestIsRunning	This property checks if a test is running.	bIsRunning = Controller.TestIsRunning

The file ScheduleTaskRunScript1.vbs includes an example using the Application object model. Note the usage of the object methods and properties in the following code snippet from the file:

 ' Procedure to run the specified test from the ACT project. Sub RunTest(oProject, strTestName, oController)     Dim oTest     ' check if test is already running     If oController.TestIsRunning Then         WScript.Echo("ACT is already running a test.")     Else          ' get a reference to the specific test and start the test         Set oTest = oProject.Tests.Item(strTestName)         Call oController.StartTest(oProject, oTest, False)     End If End Sub

Using the ACT Test Object Model

The ACT Test object model provides an interface to the HTTP client that ACT uses to send requests. You use it in the test scripts to control the actions of the HTTP client, allowing for full control of the details of a test. Whereas the ACT Application object model focuses on running a defined test script programmatically, the ACT Test object model provides the application programming interface (API) to define a customized test script. Bear in mind this difference because certain objects with the same name in both models will represent different objects.

The Test Object

The Test object is the main object in the Test object model, and it controls the test session. This Test object should not be confused with the Test object from the Application object model. This Test object is the root object that allows access to the other objects including the Request object, Connection object, User object, and Response object. A built-in Test object named "Test" is available in all dynamic tests, so there is no need to explicitly create a new Test object. Table 7-6 summarizes the main methods of the Test object.

Table 7-6: Test Object Main Methods
METHOD	DESCRIPTION
CreateConnection()	Creates a new Connection object representing the connection between the client and the server
CreateRequest()	Creates a new Request object that can be used to send a request using the Connection object's Send method
SendRequest()	Sends a simple URL request to a Web server and returns the server response in a Response object
Trace()	Writes an entry to the ACTTrace.log file
GetCurrentUser()	Returns the User object for the current user
GetNextUser()	Returns the User object for the next user in the Users group and sets the next user as the current user

The Request Object

The Request object represents the HTTP request to be sent to the test server. The VBScript code to get a reference to the Request object is as follows:

 Dim oRequest Set oRequest = Test.CreateRequest()

Table 7-7 summarizes the main properties of the Request object, which can be set before being sent to the server.

Table 7-7: Request Object Properties
PROPERTY	DESCRIPTION	CODE SAMPLE
Body	The HTTP Body	oRequest.Body = "__VIEWSTATE=..."
HTTPVersion	The HTTP version. "HTTP/1.1" is the default value.	oRequest.HTTPVersion = "HTTP/1.0"
Path	The HTTP Path component of the URL.	oRequest.Path = "/Duwamish7vb/secure/checkout.aspx"
Verb	The HTTP Verb for the request. Default is "GET."	oRequest.Verb = "POST"

The Connection Object

The Connection object represents the connection from the client (test script) to the Web server. You create the connection from the Test object and can send multiple requests as long as it is not closed. The following VBScript code shows how to create a connection to Web server web01 using port 80 with no SSL:

 Dim oConnection ' call CreateConnection method with arguments (strServer, lPort, bUseSSL) Set oConnection = Test.CreateConnection("web01", 80, false)

It is important to note that the connection is implicitly opened when the Test object CreateConnection() method is called. There is no explicit Open() method on the Connection object.

Table 7-8 describes the main methods and properties of the Connection object.

Table 7-8: Connection Object Methods and Properties
METHODS/PROPERTIES	DESCRIPTION
Close()	This method closes the connection.
Send()	This method sends a HTTP request represented by the Request object to the Web server.
IsOpen	This property checks the state of the connection the last time it was tested by the client. Note that if the server subsequently disconnects and the connection is no longer open, the property will still be set to "True."

The following VBScript code shows how to use each method:

 Sub SendRequest(oRequest)     Dim oConnection, oResponse     ' call CreateConnection method with arguments (strServer, lPort, bUseSSL)     Set oConnection = Test.CreateConnection("web01", 80, false)     If (oConnection.IsOpen) Then        Set oResponse = oConnection.Send(oRequest)        If (oResponse Is Nothing) Then            Call Test.Trace("Error: host not found or request is invalid.")        Else            Call Test.Trace("Status code:" & oResponse.ResultCode)        End If     End If     oConnection.Close End Sub

The User Object

The User object represents the current user in the test. The object exposes three properties: Name , Password , and Cookies , allowing programmatic access to change the user's credentials and view the user's HTTP cookies. The Test object methods GetCurrentUser() and GetNextUser() return the User object.

Table 7-9 describes the main properties of the User object.

Table 7-9: User Object Main Properties
PROPERTIES	DESCRIPTION
Cookies	Gets the Cookies collection from the specified domain for a unique user
Name	Gets or sets the user name
Password	Gets or sets the user password

You must address an important issue when stress testing with a finite set of unique users. Invariably, collisions will occur if multiple test users access the application with the same user credentials. For example, both user 1 and user 2 log in with the same user ID and password. User 1 adds an item to the shopping cart (item added to the database). User 2 deletes the item from the database. If user 1 tries to delete the item, he will encounter an error (item does not exist). Take into account this issue during the design of the test scenarios.

The Response Object

The Response object represents the HTTP response from the Web server. The properties of the Response object are read-only. The object is useful in developing functional test scripts that require programming logic based on the response from the Web server. Table 7-10 describes some useful properties of the Response object.

Table 7-10: Response Object Properties
PROPERTY	DESCRIPTION
Body	Gets the body of the HTTP response from the response buffer. This is useful for checking values returned from the server.
BytesReceived	Gets the number of bytes received by the client in the response.
ResultCode	Gets the HTTP response code. You can use this to check the status of the request.
TTLB	Gets the TTLB of the response was received. The time is in milliseconds.

The Body and ResultCode properties are useful in checking the content and status of the response, and the BytesReceived and TTLB properties are good metrics for analyzing the performance of the response. You can find the complete list of Response object properties, as well as the methods and properties for the other objects in the Test object model, in the ACT documentation.

Scripting Dynamic Tests with the Test Object Model

You can handle several test scenarios by scripting a dynamic test within ACT. Recording a test enables you to repeat a certain test, but it does not allow you to configure disparate test scenarios within a single test script. A dynamic test script allows you to configure every aspect of the test, from the header information sent in the request to the target page to the response from the Web server, as the following sample scripts will demonstrate .

Simulating Browser Compatibility

One of the more important application tests is to test for browser compatibility. You can handle this by scripting a dynamic test to imitate the browser request headers sent by common browsers. The Request object exposes a Headers collection whose values can be set to match those of common browsers. You can find a code sample in the test script BrowserTest.vbs in the AspNetChap7ACTTest project. The function MakeIE60GETRequest(oRequest) shows how to set up the headers to simulate a request from an Internet Explorer 6.0 browser (see Listing 7-1).

Listing 7-1: The MakeIE60GETRequest() Function

 Function MakeIE60GETRequest(oRequest)     ' function returns a request object with the appropriate header setup     Dim oHeaders     If Not (oRequest Is Nothing) Then         ' set request line         oRequest.Verb = "GET"         oRequest.HTTPVersion = "HTTP/1.1"         Set oHeaders = oRequest.Headers         With oHeaders             Call .RemoveAll()             ' set header fields             Call .Add("Accept", "*/*")             Call .Add("Accept-Language", "en-us")             Call .Add("Connection", "Keep-Alive")             Call .Add("Host", "(Automatic)")             Call .Add("User-Agent", _                  "Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0)")             Call .Add("Accept-Encoding", "gzip, deflate")             Call .Add("Cookie", "(Automatic)")         End With     Else         Test.Trace("Invalid Request object")     End If     Set MakeIE60GETRequest = oRequest End Function

Listing 7-2 shows the code to send the request and check the result code.

Listing 7-2: The SendRequest() Function

 Const SERVER_NAME = "localhost" Const PATH = "/Default.asp" Dim oIE6Request Set oIE6Request = Test.CreateRequest Call MakeIE60GETRequest(oIE6Request) Call SendRequest(oIE6Request) Sub SendRequest(oRequest)     Dim oConnection     Set oConnection = Test.CreateConnection(SERVER_NAME, 80, False)     If (oConnection Is Nothing) Then         Test.Trace("Error: Unable to create connection.")     Else         If (oConnection.IsOpen) Then             oRequest.Path = PATH             Set oResponse = oConnection.Send(oRequest)             ' check for a bad connection or request             If (oResponse Is Nothing) Then                 Test.Trace("Error: invalid request or host not found ")             Else                 Test.Trace("Server response:" & oResponse.ResultCode)             End If             Call oConnection.Close()         Else             Test.Trace("Connection was closed")         End If     End If End Sub

You could simulate various browser requests in this manner by setting the correct header data for each browser type. Note this test script verifies that the server does not generate an error for the browser-specific request header, but it does not verify the server responded in a correct manner for the specific browser. For example, the response may still contain code that renders fine in IE6 but may encounter rendering issues in Netscape 6. Likewise, the script cannot verify if the server lowers the resolution to 240x320 when the client browser is a Pocket IE. To verify these issues, an option is to save each browser response during execution of the test script and then verify each response by loading it into the appropriate browser.

Note the use of the Test.Trace() method, which appends a string to the ACTTrace.log file. The log file is in %ProgramFiles%\Microsoft ACT\ACTTrace.log . The ACTTrace.log file is overwritten when a new test is started, so you must manually move or rename it if you want to save the information from each test run.

Configuring a Dynamic Test for Multiple Pages

At one time or another, you must test an application to verify all the pages and components are functional. This requires the tester to run through every page in the application. With ACT, you can automate this function with a dynamically configurable test script. The sample test script PageTest.vbs in project AspNetChap7ACTTest demonstrates how to use a text file ( RequestPaths.dat ) to configure the target pages for a test. The text file stores the request paths in the format:

 /Duwamish7vb/default.aspx /Duwamish7vb/secure/logon.aspx?ReturnUrl=%2fDuwamish7vb%2fsecure%2faccount.aspx /Duwamish7vb/categories.aspx?id=830 /Duwamish7vb/viewsource.aspx?path=categories.src /Duwamish7vb/shoppingcart.aspx

where each line defines a unique test request path. The script reads each line and assigns it to the Request.Path property before opening a connection, as shown in Listing 7-3.

Listing 7-3: Dynamic Script to Test Multiple Pages

 Dim strURLPath, arrURLPaths(100) Dim lCount, lNumberOfItemsInArray Dim oConnection, oUser, oResponse, oRequest strURLPath = "" lCount = 0 lNumberOfItemsInArray = 0 strURLPath = "" Set oConnection = Test.CreateConnection(SERVER_NAME, 80, False) Set oRequest = Test.CreateRequest lNumberOfItemsInArray = GetDataFromFile( (DATA_FILE_PATH & _     DATA_FILE_NAME) , arrURLPaths) For lCount = 0 To (lNumberOfItemsInArray - 1)      ' loop through each URL     strURLPath = arrURLPaths(lCount)       ' get an element from the array     oRequest.Path = strURLPath       ' assign Request.Path property     Set oResponse = oConnection.Send(oRequest)       ' send the request     If (oResponse Is Nothing) Then        ' check for a bad request or connection          Call Test.Trace("E: Invalid request or host not found." & VbCrLf)     Else          Call Test.Trace("I: Requested '" & strURLPath & "'"& vbCrLf)          Call Test.Trace("I: Server response: "& oResponse.ResultCode & VbCrLf)     End If Next

The result code is checked for each response, and the results are written to the Trace log. After the test is run, messages in the Trace log would show the requested page, and the result code from the response for each path is listed in the .dat file:

 [08/29 - 16:03:44!2880]   1>I: Requested '/Duwamish7vb/default.aspx' [08/29 - 16:03:44!2880]   1>I: Server response: 200 [08/29 - 16:03:45!2880]   1>I: Requested '/Duwamish7vb/categories.aspx?id=830' [08/29 - 16:03:45!2880]   1>I: Server response: 200 [08/29 - 16:03:46!2880]   1>I: Requested '/Duwamish7vb/categories.aspx?id=831' [08/29 - 16:03:46!2880]   1>I: Server response: 200

There are a multitude of test scenarios that you can implement using the Test object model. These two samples should have provided a starting point from which will spring many other imaginative test scripts. You can find other interesting test scripts in the ACT Samples directory, located in the ACT installation directory.

Understanding Common Runtime Issues with ACT

The following sections focus on common runtime issues encountered while using ACT. These issues all prevent the user from running the ACT tool with their application. Some issues relate to configuration, and others require workarounds to overcome some shortcomings within ACT:

Error encountered when using ACT with NTLM authentication
Provider load failure error message encountered in ACT
Low percentage of meaningful response codes
ACT user interface limits test script length

NTLM Authentication Error

During testing of a Web application with NTLM authentication enabled, you may encounter the following error:

 401 - The requested resource requires user authentication

This error occurred because HTTP 1.0 does not support keep-alive connections. This error has been known to occur in the ACT, Developer Edition. To resolve this error, set the HTTP version in the test script from this:

 oRequest.HTTPVersion = "HTTP/1.0"

to this:

 oRequest.HTTPVersion = "HTTP/1.1"

Also, using NTLM authentication requires the test users to be valid domain users. The format for the users in the Users node should be as follows:

 Domain_name\user_name

Provider Load Failure Error Message Encountered

The common error message encountered at the start of a test is this:

 Could not start the test. Provider load failure (0x80041013)

This error is commonly a result of the ACT account ACTUser not having the appropriate rights on the local machine. The ACTUser account requires batch log on rights to access the ACT objects through Distributed COM (DCOM).

To verify the rights are set correctly, follow these steps on the local machine:

Open Local Security Policy under Administrative Tools.
Select the Local Policies node and then double-click User Rights Assignment.
Double-click Log On as a Batch Job and then verify that ACTUser is selected in the Local Policy Setting and Effective Policy Setting columns . Add ACTUser if it does not appear on the list.
If your server is a domain member, in Local Security Settings of the Microsoft Management Console (MMC), right-click Security Settings and then click Reload.

Both the ACT Controller and ACT Broker programs run under the ACTUser account identity. If the password for the ACTUser account is modified from its original value, you must update the identity of both programs by using the DCOM configuration utility dcomcnfg .exe .

Low Percentage of Meaningful Response Codes

The ACT test results will only be meaningful if a majority of the response codes are successful. An HTTP response code of 200 indicates that the server successfully processed a request. In addition, an overloaded server commonly returns these result codes:

Response Code 403 indicates that the server understood the request but is refusing to fulfill it.
Response Code 404 indicates that the server could not find a linked resource, usually an image file or an include file.
Response Code 500 indicates that the server encountered errors in attempting to fulfill the request.

In our performance testing, we expect to see greater than 97.5 percent of the response codes returned as successful 200 codes. Otherwise, we do not treat the results as meaningful. This percentage is somewhat arbitrary, although it is loosely based on a statistical approach that uses standard deviations as a measure of certainty . You may choose to use an alternative percentage, but make sure that it is a high value and that every test within a group of tests is measured against a consistent value.

ACT Test Script Length Limitation

The ACT user interface for the test script limits the length of the test script to 1024KB. This limitation is strictly a limitation of the ACT script user interface and not a limitation of the ACT test script engine. ACT can execute files larger than 1024KB. A common practice is to edit the test script outside of ACT in your favorite script editor. The VS.NET IDE is more than adequate for this purpose. After the script is complete, simply copy and paste it into the ACT script panel.

Note

The Microsoft newsgroups provide helpful information for resolving issues with ACT. Microsoft developers regularly monitor these newsgroups and will post answers and suggestions to posted questions. The newsgroups for ACT and application performance are at microsoft.public.vsnet.enterprise.tools and microsoft.public.dotnet.framework.performance .