Lesson 2: Running Tests

Lesson 2: Running Tests

In the preceding lesson, you learned how to plan testing and how to create different types of tests using Visual Studio .NET. In this lesson, you ll learn how to run those tests and capture their results.

After this lesson, you will be able to

• Use ACT to run load tests on Web applications

• Record results and performance data from a load test

• Mimic different browser types with ACT and check browser responses for contained strings within test scripts

• Use the Windows Scripting Host to test .NET assemblies

• Create command files to build and test .NET assemblies

• Run command files at specified times using the Windows Task Manager to perform unattended nightly build and test runs

Estimated lesson time: 30 minutes

Running Tests with ACT

Lesson 1 briefly showed you how to create a load test and run it in ACT. This section describes how to configure and run ACT tests in more detail by describing how to perform the following tasks:

• Log test results.

• Configure the test to use multiple simultaneous connections.

• Run tests with different user name and password combinations.

• Add performance counters to a test.

• Mimic different browsers and different network connection types.

These tasks are explained in the following sections.

Logging Test Results

By default, ACT creates a log file for each test project in the /Perflogs folder on the drive where ACT is installed.

To change the location where ACT stores the log file, follow these steps:

1. Right-click the project item in the left pane of the ACT window, and choose Properties from the shortcut menu. ACT displays the test project s Properties dialog box, as shown in Figure 10-11.

   

   Figure 10-11. Setting log file location

2. Click the Debugging tab, and enter a location for storing test project logs in the Folder To Save Log In box. Click OK when you ve finished.

ACT log files record different amounts of information, depending on the test s TraceLevel settings. To change the amount of information logged, set the Test object s TraceLevel property within a test script. For example, the following line of script tells ACT to log all messages:

VBScript

Test.TraceLevel = -1

JScript

Test.TraceLevel = -1;

The TraceLevel property has the settings shown in Table 10-2.

Table 10-2. TraceLevel Settings

Setting	Logs
-1	All information, including requests sent to the application, responses from the application, and messages written by Trace methods in the test script.
0	No information turns off logging.
1	Requests sent to the application and responses from the application.
2	Messages written by Trace methods in the test script. This is the default setting.

Use the Test object s Trace method to write messages to the log from within the test script. For example, the following script writes a message to the log if ACT is not able to connect to the server:

VBScript

Dim g_oConnection ConnnectToServer Sub ConnectToServer() Set g_oConnection = Test.CreateConnection("http://www.contoso.com, 80, false) If (g_oConnection is Nothing) Then Test.Trace "Error: Unable to create connection to contoso" End If End Sub

JScript

var g_oConnection ConnectToServer(); function ConnectToServer() { g_oConnection = Test.CreateConnection ("http://www.contoso.com", 80, false); if (g_oConnection == null) Test.Trace("Error: Unable to create connection to contoso"); }

Using Multiple Connections

ACT can simulate different server load levels by adding simultaneous connections to use during the test. By default, ACT uses one connection to send requests to the Web application. Additional connections are the equivalent of additional simultaneous users.

To increase the number of connections used for a test, follow these steps:

1. Right-click the test item in the left pane of the ACT window, and choose Properties from the shortcut menu. ACT displays the test s Properties dialog box, as shown in Figure 10-12.

   

   Figure 10-12. Test properties

2. Change the number in the Test Load Level section to set the number of simultaneous connections. Click OK when you ve finished.

Adding Users and Passwords

ACT can supply user names and passwords to a Web application through the user s groups stored with the test project. This allows you to test how a Web application authenticates users and also permits you to test cookies.

To add user names and passwords to a user group, follow these steps:

1. Select Users in the left pane of the ACT window to expand the list of user groups.

2. Select a user group, and then enter user names and passwords in the table displayed in the right pane of the ACT window. Figure 10-13 shows changes to the Default Users Group.

   

   Figure 10-13. Adding users

3. Right-click the test item in the left pane of the ACT window, and select Properties. ACT displays the test s Properties dialog box, as shown in Figure 10-14.

   

   Figure 10-14. Setting test properties to activate users

4. Click the Users tab, and select Specify Users. Click OK when you ve finished.

ACT automatically changes users for each new iteration of a test and for each new connection it creates. You can also manually change users within a test using the Test object s GetNextUser method, as shown here:

VBScript

Sub ChangeUser() Test.GetNextUser Test.Trace("Next user name: " & Test.GetCurrentUser.Name) End Sub

JScript

function ChangeUser() { Test.GetNextUser(); Test.Trace("Next user name: " + Test.GetCurrentUser.Name); }

Adding Performance Counters

You can track a Web application s performance during a load test by adding performance counters to the test s properties. These performance counters are the same ones that you used to monitor server performance in Chapter 9, Building and Deploying Web Applications, using the Microsoft Management Console (MMC) Performance Logs and Alerts snap-in.

To add performance counters to a load test, follow these steps:

1. Right-click the test item in the left pane of the ACT window, and choose Properties on the shortcut menu. ACT displays the test s Properties dialog box.

2. Click the Counters tab, and then click Add. ACT displays the Browse Performance Counters dialog box, as shown in Figure 10-15.

   

   Figure 10-15. Adding counters

3. Select the counters you want to monitor, and click Add. Click Close when you have finished adding counters.

4. Click OK to close the test s Properties dialog box.

When a test runs, ACT saves performance statistics in an XML file that is listed in the test results pane of the ACT window, as shown in Figure 10-16.

Figure 10-16. Viewing performance results

Mimicking Browser Types

ACT can mimic different types of browsers by modifying the user agent information included in the header it sends with each request. The Browser Type test in the ACTSamples project that is installed with ACT demonstrates how to do this.

You can copy the Browser Type test to your own test projects to test browser compatibility and to test whether your application responds correctly to unsupported browser types. Because ACT does not actually display the responses, however, you can t rely on this test to visually check that your Web application is displayed correctly on different browsers.

Getting Content of Responses

ACT returns the server s response to a request as the Response object s Body property. You can use the VBScript Instr function to test whether a response contains an expected result, as shown here:

VBScript

Set oResponse = oConnection.Send(oRequest) If Instr(oResponse.Body, "Correct!") Then Test.Trace("Correct response.") Else Test.Trace("Incorrect response.") End If

JScript

oResponse = oConnection.Send(oRequest); if (oResponse.Body.search"Correct!")) Test.Trace("Correct response."); else Test.Trace("Incorrect response.");

The Body Search test in the ACTSamples project further demonstrates using Instr to locate items in a response.

More Info
The ACT online Help provides a complete reference to the testing object model and includes conceptual information about using ACT to load test Web applications.

Running Tests with .NET Assemblies

ACT was written primarily to interact with Web applications through requests and responses. This is great for testing the user interface aspects of a Web application, but it is limiting when you re trying to perform tests on the underlying classes used by the application.

For example, say that you want to use ACT to test the FlashCardClass, as shown in the section Unit Testing, in Lesson 1. How would you do it? Probably by attempting something like this:

VBScript

Test.TraceLevel = -1 Dim FlashCard Set FlashCard = CreateObject("vbFlashCards.FlashCardClass") If (FlashCard Is Nothing) Then Test.Trace("Error: FlashCard not created.") Test.StopTest End If

JScript

Test.TraceLevel = -1; var FlashCard; FlashCard = CreateObject("csFlashCards.FlashCardClass"); if (FlashCard == null) { Test.Trace("Error: FlashCard not created."); Test.StopTest(); }

The only problem is that this doesn t work! Because Web applications are based on the Microsoft .NET Framework, you can t use their namespaces as an argument to the CreateObject method. CreateObject expects a Component Object Model (COM) object s programmatic identifier (progID), not a namespace.

To use scripting to test a .NET assembly, follow these steps:

1. In order to register a .NET assembly for use from COM, the assembly must be compiled as a separate class library. Web application assemblies can t be used from COM. Register the .NET assembly for use with COM.

2. Make sure that the COM application can find the .NET assembly.

3. Create the .NET object using the progID registered for the .NET assembly.

The following sections describe these steps in more detail.

Registering .NET Assemblies

Use the RegAsm.exe utility to register a .NET assembly with the system registration database. The RegAsm.exe utility is installed with the .NET Framework.

To use the RegAsm.exe utility, follow these steps:

1. Open a Visual Studio .NET command prompt. From the Start menu, point to All Programs, Microsoft Visual Studio .NET, Visual Studio .NET Tools, and then choose Visual Studio .NET Command Prompt.

2. Within the command prompt, navigate to the folder containing the assembly to register, and run the following command:

RegAsm.exe assembly.dll

The variable assembly.dll is the name of the assembly to register.

RegAsm.exe registers the public classes from the .NET assembly using their namespace and class names as their progIDs. For example, registering the Shapes assembly created for COM interoperation in Chapter 7, Advanced Web Forms Programming, results in the registry entries shown in Figure 10-17.

Figure 10-17. Shapes system registration database entries

Locating the .NET Assembly

After you register a .NET assembly, COM components need a way to find the assembly. There are a number of ways to do this, but the three simplest ways are:

• Install the assembly in the global assembly cache (GAC).

• Copy the assembly to the same folder as the COM component that will use it.

• Copy the COM component to the folder where the assembly resides.

Assemblies installed in the GAC are available to all applications on the computer. Assemblies must be configured to use strong names before they can be installed in the GAC. For complete instructions on configuring and installing assemblies in the GAC, see the Visual Studio .NET online Help.

For assemblies that are not intended for sharing with other applications, copying the assembly or the COM component to the same folder might be the easiest solution. This is particularly true for testing, when you might not want an assembly to be generally available.

Creating the .NET Object

After you have registered a .NET assembly and made sure that the COM components can find it, you can create an instance of the .NET object using the COM object creation methods, such as the VBScript CreateObject function.

To see how this works with a testing tool such as the Windows Scripting Host, follow these steps:

1. Register a Web application s assembly as described in the section Registering .NET Assemblies, earlier in this chapter.

2. Copy CScript.exe from the \Windows\system32 or \WINNT\system32 folder to the Web application s \bin folder. This is the copy of CScript.exe that you will use to run your testing script; it must be in the same folder as the .NET assembly so that the COM and .NET objects can interoperate.

3. Create a VBScript or JScript file, and run it using the copy of CScript.exe stored in the \bin folder.

The following code demonstrates how to test FlashCardClass using the Windows Scripting Host:

VBScript

' TestFlash.vbs ' FlashCardClass unit test Option Explicit Dim FlashClass Set FlashClass = CreateObject("vbFlashCards.FlashCardClass") ' Check if class created. If (FlashClass is Nothing) Then WScript.Echo "Error: FlashCardClass not created." WScript.Quit End If ' Test Shuffle FlashClass.Shuffle 1, 1000 Dim X, Y x = FlashClass.FirstNumber y = FlashClass.SecondNumber ' Shuffle again FlashClass.Shuffle 1, 1000 If (x = FlashClass.FirstNumber) or (y = FlashClass.SecondNumber) Then WScript.Echo "Warning: Number not unique." End If ' Test Answer If (FlashClass.FirstNumber + FlashClass.SecondNumber = FlashClass.Answer) Then WScript.Echo "Success: Answer adds up!" Else WScript.Echo "Error: Answer incorrect." End If

JScript

//TestFlash.js //FlashCardClass unit test var FlashClass; FlashClass = new ActiveXObject("vbFlashCards.FlashCardClass"); //Check if class created. if (FlashClass == null) { WScript.Echo("Error: FlashCardClass not created."); WScript.Quit(); } //Test Shuffle FlashClass.Shuffle(1, 1000); var X, Y; x = FlashClass.FirstNumber; y = FlashClass.SecondNumber; //Shuffle again FlashClass.Shuffle(1, 1000); if ((x == FlashClass.FirstNumber) (y == FlashClass.SecondNumber)) WScript.Echo("Warning: Number not unique."); //Test Answer if (FlashClass.FirstNumber + FlashClass.SecondNumber == FlashClass.Answer()) WScript.Echo("Success: Answer adds up!"); else WScript.Echo("Error: Answer incorrect.");

To run the preceding script, use a command line similar to the following:

@rem VBScript bin\cscript TestFlash.vbs > logs\FlashErr.log @rem JScript bin\cscript TestFlash.js > logs\FlashErr.log

Running Unattended Builds and Tests

Running unit, integration, and load tests on a regular basis provides constant quality assurance and checks for regressions. Most development teams run builds and a set of tests nightly. Running the tests at night prevents conflicts with check-ins and doesn t disrupt workflow while a machine is tied up doing a build. Because they run late at night, these builds are usually unattended, meaning that no one has to be there to start them or monitor their progress.

To run unattended builds and tests, follow these steps:

1. Create a command file to set the environment variables, build, and test commands you want to run.

2. Use the Windows Task Manager to schedule the command file to be run.

3. Monitor the log files written by the commands to ensure that the builds and tests are succeeding.

The following sections describe these steps in more detail.

Creating a Command File

You can create a command file (.bat) to execute any set of commands you want to run. In general, the command file has three main sections that perform the following tasks:

• Set environment variables, such as PATH and LIB, that are required to find the components used by the build

• Run the Visual Studio .NET development environment to build the assembly

• Run the test tools used to test the assembly

The environment variables used by Visual Studio .NET can be found in the VsVars32.bat command file that is installed in Visual Studio .NET s Command\Tools folder. You can cut and paste the settings from this file to your command file.

To run Visual Studio .NET from the command line, use the DevEnv.exe command with the project s solution file (.sln). For example, the following command line builds the FlashCards application using the Debug configuration and records any errors in the Logs folder:

DevEnv vbFlashCards.sln /build Debug > Logs\builders.log

You can put all of the steps together in a single command file. For example, the following command file sets the required environment variables, builds a Web application, and runs test scripts on the assembly:

@rem Build.bat @rem ======================================================================== @rem Environment settings from vsvars32.bat @rem ======================================================================== @SET VSINSTALLDIR=E:\Program Files\Microsoft Visual Studio .NET\Common7\IDE @SET VCINSTALLDIR=E:\Program Files\Microsoft Visual Studio .NET @SET FrameworkDir=C:\WINDOWS\Microsoft.NET\Framework @SET FrameworkVersion=v1.0.3705 @SET FrameworkSDKDir=C:\Program Files\Microsoft Visual Studio .NET\FrameworkSDK @rem Root of Visual Studio common files. @rem @rem Root of Visual Studio ide installed files. @rem @set DevEnvDir=%VSINSTALLDIR% @rem @rem Root of Visual C++ installed files. @rem @set MSVCDir=%VCINSTALLDIR%\VC7 @set PATH=%DevEnvDir%;%MSVCDir%\BIN;%VCINSTALLDIR%\Common7\Tools; %VCINSTALLDIR%\Common7\Tools\bin\prerelease;%VCINSTALLDIR%\Common7\Tools\bin;%Framework SDKDir%\bin;%FrameworkDir%\%FrameworkVersion%;%PATH%; @set INCLUDE=%MSVCDir%\ATLMFC\INCLUDE;%MSVCDir%\INCLUDE; %MSVCDir%\PlatformSDK\include\prerelease;%MSVCDir%\PlatformSDK \include;%FrameworkSDKDir%\include;%INCLUDE% @set LIB=%MSVCDir%\ATLMFC\LIB;%MSVCDir%\LIB; %MSVCDir%\PlatformSDK\lib\prerelease; %MSVCDir%\PlatformSDK\lib;%FrameworkSDKDir%\lib;%LIB% @rem ======================================================================== @rem End environment settings @rem ======================================================================== @rem ======================================================================== @rem Development environment build commands @rem ======================================================================== devenv vbBuildSnippet.sln /build debug > logs\builderrs.log @rem ======================================================================== @rem End build commands @rem ======================================================================== @rem ======================================================================== @rem Testing commands @rem ======================================================================== bin\cscript RunTests.vbs > logs\testerrs.log @rem ======================================================================== @rem End testing commands @rem ========================================================================

Scheduling Tasks

To schedule a command file to run on a regular basis, use the Windows Task Manager.

To run the Task Manager, follow these steps:

1. From the Start Menu, point to All Programs, Accessories, System Tools, and then choose Scheduled Tasks. Windows displays the Scheduled Tasks folder.

2. Double-click Add Scheduled Task to run the Scheduled Task Wizard.

3. Click Next, and then click Browse and specify the command file to run. After you ve selected the command file, the wizard displays the schedule page, as shown in Figure 10-18.

   

   Figure 10-18. Choosing a schedule

4. Select an option indicating when to run the task, and click Next. The wizard displays the time page, as shown in Figure 10-19.

   

   Figure 10-19. Choosing a time to run

5. Enter a time to run the task, and click Next. The wizard displays the password page, as shown in Figure 10-20.

   

   Figure 10-20. Entering user name and password

6. Enter the user name and password under which to run the command, click Next, and then click Finish to complete the wizard.

Viewing Log Files

Because builds and tests scheduled with the Task Manager run unattended, it is important to monitor log files created by the commands in the build process. The build and test commands shown earlier in this section used the redirect operator (>) to write output from commands to log files stored in a Log folder.

Viewing log files in Notepad is easy, as shown in Figure 10-21.

Figure 10-21. Viewing a build log