Lesson 1: Creating Tests | MCAD(s)MCSD Self-Paced Training Kit(c) Developing Web Applications With Microsoft Visual Basic. Net and Microsoft V[. .. ]0-315

Lesson 1: Creating Tests

All software intended for public consumption should receive some level of testing. The more complex or widely distributed a piece of software is, the more essential testing is to its success. Without testing, you have no assurance that software will behave as expected. The results in a public environment can be truly embarrassing.

For software, testing almost always means automated testing. Automated tests use a programming language to replay recorded user actions or to simulate the internal use of a component. Automated tests are reproducible (the same test can be run again and again) and measurable (the test either succeeds or fails). These two advantages are key to ensuring that software meets product requirements.

In this lesson, you ll learn how to write a test plan, how to ensure that the product is thoroughly tested using different types of tests, and how to create different types of tests using Visual Studio .NET.

After this lesson, you will be able to

Understand a software test plan
Identify different types of software testing
Write unit tests for software components
Create integration tests for completed components
Explain how a test library can identify regressions
Use ACT to create Web application load tests
Test Web applications for different client configurations

Estimated lesson time: 30 minutes

Developing a Test Plan

The first step in testing is developing a test plan based on the product requirements. The test plan is usually a formal document that ensures that the product meets the following standards:

Is thoroughly tested
Untested code adds an unknown element to the product and increases the risk of product failure.
Meets product requirements
To meet customer needs, the product must provide the features and behavior described in the product specification. For this reason, product specifications should be clearly written and well understood.
Does not contain defects
Features must work within established quality standards, and those standards should be clearly stated within the test plan.

Having a test plan helps you avoid ad hoc testing the kind of testing that relies on the uncoordinated efforts of developers or testers to ensure that code works. The results of ad hoc testing are usually uneven and always unpredictable. A good test plan answers the following questions:

How are tests written?
Describe the languages and tools used for testing.
Who is responsible for the testing?
List the teams or individuals who write and perform the tests.
When are the tests performed?
The testing schedule closely follows the development schedule.
Where are the tests and how are test results shared?
Tests should be organized so that they can be rerun on a regular basis.
What is being tested?
Measurable goals with concrete targets let you know when you have achieved success.

Some of these questions might have more than one answer, depending on the type of test. For instance, individual developers are often responsible for writing the first level of tests for their own code, while a separate testing team might be responsible for ensuring that all code works together. The following sections describe the different types of tests and the techniques used with Visual Studio .NET to perform these tests.

Types of Tests

The test plan specifies the different types of tests that will be performed to ensure the that product meets customer requirements and does not contain defects. Table 10-1 describes the most common test types.

Table 10-1. Types of Tests
Test type	Ensures that
Unit test	Each independent piece of code works correctly.
Integration test	All units work together without errors.
Regression test	Newly added features do not introduce errors to other features that are already working.
Load test (also called stress test)	The product continues to work under extreme usage.
Platform test	The product works on all of the target hardware and software platforms.

These test types build on each other, and the tests are usually performed in the order shown in Table 10-1. The testing process follows the flow diagram shown in Figure 10-1.

figure 10-1 the testing cycle

Figure 10-1. The testing cycle

The process shown in Figure 10-1 is based on a modular product design in which the product is developed as a set of components that can be programmed, tested, integrated, and released individually. This is the modern approach, and you should be wary of any project that proposes a monolithic design in which integration and testing are performed only at the end of the development cycle.

Unit Testing

A product unit is the smallest piece of code that can be independently tested. From an object-oriented programming perspective, classes, properties, methods, and events are all individual units. A unit should pass its unit test before it is checked into the project for integration.

Unit tests are commonly written by the developer who programmed the unit and are either written in the same programming language as the product unit being tested or in a similar scripting language, such as VBScript. The unit test itself can be as simple as getting and setting a property value, or it can be more complicated. For instance, a unit test might take sample data and calculate a result and then compare that result against the expected result to check for accuracy.

Group the unit tests together logically. Tests that check each property and method in a class can be included in a single procedure, which is then called from the Sub Main of a separate testing application. For example, the following test procedure creates an instance of the FlashCardClass created in Chapter 2, Creating Web Forms Applications, and then tests each of its properties and methods:

Visual Basic .NET

Function TestFlashCard() As Boolean 'Create class. Dim FlashCard As New MCSDWebAppsVB.FlashCardClass() 'Test if class exists. If IsNothing(FlashCard) Then Console.WriteLine("FlashCardClass failed") 'Return False and end here. Return False End If 'Test shuffle method using maximum and minimum values. FlashCard.Shuffle(Integer.MinValue, Integer.MaxValue) 'Test properties. Dim bResult As Boolean = True Dim intFirst As Integer = FlashCard.FirstNumber Dim intSecond As Integer = FlashCard.SecondNumber Dim dblAnswer As Double = intFirst + intSecond If dblAnswer <> FlashCard.Answer Then Console.WriteLine("Error: Numbers don't add up!") bResult = False End If 'Shuffle again. FlashCard.Shuffle(Integer.MinValue, Integer.MaxValue) 'Make sure the new values are unique. If intFirst = FlashCard.FirstNumber Then Console.WriteLine("Warning: FirstNumber not unique after shuffle") End If If intSecond = FlashCard.SecondNumber Then Console.WriteLine("Warning: SecondNumber not unique after shuffle") End If 'Check different operators. FlashCard.Operation = "-" If FlashCard.FirstNumber -_ FlashCard.SecondNumber <> FlashCard.Answer Then Console.WriteLine("Error: - operator doesn't substract.") bResult = False End If FlashCard.Operation = "x" If FlashCard.FirstNumber * FlashCard.SecondNumber <> FlashCard.Answer Then Console.WriteLine("Error: x operator doesn't multiply.") bResult = False End If FlashCard.Operation = "+" If FlashCard.FirstNumber + FlashCard.SecondNumber <> FlashCard.Answer Then Console.WriteLine("Error: + operator doesn't add.") bResult = False End If 'Return success/failure. Return bResult End Function

Visual C#

static bool TestFlashCardClass() { //Create class. csFlashCards.FlashCardClass FlashCard = new MCSDWebAppsCS.FlashCardClass(); //Test is class exists. if (FlashCard == null) { Console.WriteLine("FlashCardClass failed"); //Return false and end here. return false; } //Test shuffle method using maximum and minimum values. FlashCard.Shuffle(int.MinValue, int.MaxValue); //Test properties. bool bResult = true; int intFirst = FlashCard.FirstNumber; int intSecond = FlashCard.SecondNumber; double dblAnswer = intFirst + intSecond; if (dblAnswer != (double)FlashCard.Answer()) { Console.WriteLine("Error: Numbers don't add up!"); bResult = false; } //Shuffle again. FlashCard.Shuffle(int.MinValue, int.MaxValue); //Make sure new values unique. if (intFirst == FlashCard.FirstNumber) Console.WriteLine("Warning: FirstNumber not unique after shuffle"); if (intSecond == FlashCard.SecondNumber) Console.WriteLine("Warning: SecondNumber not unique after shuffle"); //Check different operators. FlashCard.Operation = "-"; if (FlashCard.FirstNumber - FlashCard.SecondNumber != FlashCard.Answer()) { Console.WriteLine("Error: - operator doesn't substract."); bResult = false; } FlashCard.Operation = "x"; if (FlashCard.FirstNumber * FlashCard.SecondNumber != FlashCard.Answer()) { Console.WriteLine("Error: x operator doesn't multiply."); bResult = false; } FlashCard.Operation = "+"; if (FlashCard.FirstNumber + FlashCard.SecondNumber != FlashCard.Answer()) { Console.WriteLine("Error: + operator doesn't add."); bResult = false; } //Return success/failure. return bResult; }

The most important factor in a unit test is thoroughness. Unit tests should exercise each piece of code in the application and use a range of possible values. The preceding sample demonstrates using a wide range by passing the integer MinValue and MaxValue properties to the Shuffle method.

The tests should report errors at the unit level so that locating and fixing problems is straightforward. The preceding sample does this by displaying a message in the Output window using Console.WriteLine. Alternatively, you can write errors to a testing log file using Debug.WriteLine or raise alerts using Debug.Assert.

Integration Testing

The first integration test always answers the question, Does the application compile? At this point, a compilation error in any of the components can keep the integration testing from moving forward. Some projects use nightly builds to ensure that the product will always compile. If the build fails, the problem can be quickly resolved the next morning.

The most common build problem occurs when one component tries to use another component that has not yet been written. This occurs with modular design because the components are often created out of sequence. You solve this problem by creating stubs. Stubs are nonfunctional components that provide the class, property, or method definition used by the other component. Stubs are a kind of outline of the code you will create later.

When all of the build problems are resolved, integration testing really becomes just an extension of unit testing, although the focus is now whether the units work together. At this point, it is possible to wind up with two components that need to work together through a third component that has not been written yet. To test these two components, you create a driver. Drivers are simply test components that make sure two or more components work together. Later in the project, testing performed by the driver can be performed by the actual component.

In addition to adding stubs and drivers to your application, you might need to add a testing interface to help automate the integration testing of some components. A testing interface is a set of public properties and methods or other tools that you can use to control a component from an external testing program. For example, the following testing interface generates a log of the results displayed by the FlashCard application s Web form:

Visual Basic .NET

' From FlashCards.aspx Private Sub txtAnswer_TextChanged(ByVal sender As System.Object, _ ByVal e As System.EventArgs) Handles txtAnswer.TextChanged If txtAnswer.Text = FlashCard.Answer Then lblFeedback.Text = "Correct!" #If DEBUG Then LogResult() #End If ' Get another set of numbers. FlashCard.Shuffle() ' Refresh display to show new numbers. RefreshDisplay() Else lblFeedback.Text = "Oops! Try Again." #If DEBUG Then LogResult() #End If End If ' Clear answer txtAnswer.Text = "" End Sub #If DEBUG Then Public Sub TestUI() Dim strResult As String ' Check if correct. If lblFeedback.Text = "Correct!" Then strResult = lblFirst.Text + lblSecond.Text + "=" + txtAnswer.Text Else strResult = lblFirst.Text + lblSecond.Text + "!=" + txtAnswer.Text End If Dim filResult As StreamWriter = _ File.AppendText(Server.MapPath(".\results.log")) filResult.Write(System.DateTime.Now.ToString() + ", ") filResult.WriteLine(strResult) filResult.Close() End Sub #End If

Visual C#

//From FlashCards.aspx private void txtAnswer_TextChanged(object sender, System.EventArgs e) { if(txtAnswer.Text == FlashCard.Answer().ToString()) { lblFeedback.Text = "Correct!"; #if DEBUG TestUI(); #endif //Get another set of numbers. FlashCard.Shuffle(); //Refresh display to show new numbers. RefreshDisplay(); //Clear answer txtAnswer.Text = ""; } else { lblFeedback.Text = "Oops! Try Again."; #if DEBUG TestUI(); #endif } } #if DEBUG public void TestUI() { string strResult; //Check if correct. if (lblFeedback.Text == "Correct!") strResult = lblFirst.Text + lblSecond.Text + "=" + txtAnswer.Text; else strResult = lblFirst.Text + lblSecond.Text + "!=" + txtAnswer.Text; StreamWriter filResult = File.AppendText(Server.MapPath(".\\results.log")); filResult.Write(System.DateTime.Now.ToString() + ", "); filResult.WriteLine(strResult); filResult.Close(); } #endif

The preceding code uses the #If #End If/#if #endif preprocessor directives to prevent the testing interface from being compiled into the release version of the product. This is important because a testing interface exposes the inner workings of the application, which you generally want to hide in the final version.

Regression Testing

Unit and integration tests form the basis of regression testing. As each test is written and passed, it gets checked into the test library for a regularly scheduled testing run. If a new component or a change to an existing component breaks one of the existing unit or integration tests, the error is called a regression.

If you design your test properly, the unit and integration tests report their errors in a way that makes it easy to locate the cause of the regression. If the test doesn t provide an easy way to locate the cause, you might need to improve those tests or you might need to write new tests to target the regression.

The key to success with both integration and regression testing is to run the full set of tests frequently if possible, as part of the nightly build. Detecting problems early and resolving them as they occur prevents one error from hiding others.

Load Testing

When you deploy a Web application, it might be used by one or two clients at a time or it might get deluged by thousands of requests. To find out how well a Web application will work in these varying conditions, you need to perform load tests. Use the Microsoft ACT to create and run load tests on a Web application.

To use ACT, follow these steps:

Create a test by recording a user session with the Web application using ACT.
Set the load level and duration properties for the test.
Run the test.

In general, load tests are not created as early in the development process or run as frequently as unit, integration, or regression tests. The following sections describe how to create and run load tests in more detail.

Recording a Load Test

To record a load test in ACT, follow these steps:

From the ACT Actions menu, choose New Test. ACT displays the New Test Wizard.
Click Next. The wizard displays the Test Source page, as shown in Figure 10-2.

Figure 10-2. Recording a new load test
Select Record A New Test, and click Next. The wizard displays the Test Type page, as shown in Figure 10-3.

Figure 10-3. Selecting a language
Click Next. The wizard displays the Browser Record page, as shown in Figure 10-4.

Figure 10-4. Starting and stopping recording
Click Start Recording. The wizard displays a browser window, as shown in Figure 10-5.

Figure 10-5. Performing actions to record
Perform the actions you want to record. For instance, you might walk through all the pages in your Web application, performing tasks on each page.
On the Browser Record page, click Stop Recording when you have finished recording actions, and then click Next. The wizard displays the Test Properties page, as shown in Figure 10-6.

Figure 10-6. Naming the test
Enter a name for the test, and click Next. The wizard displays the Completing The New Test Wizard page. Click Finish to close the wizard.

Setting Test Properties

After you record a test, it is displayed in the Tests list in ACT, as shown in Figure 10-7.

figure 10-7 the recorded test

Figure 10-7. The recorded test

ACT tests can be written in the VBScript or JScript languages, but ACT records tests using VBScript only. After you record a test, set the load and duration of the test by following these steps:

Right-click your test, and choose Properties on the shortcut menu. ACT displays the test s load and duration properties, as shown in Figure 10-8.

Figure 10-8. Test load and duration
Set the Test load level to increase the number of users accessing the Web application at the same time. Set the Test duration to run the test for a certain amount of time or for a certain number of requests. Click OK when you ve finished.

Running Load Tests

To run a load test in ACT, select the test, and then choose Start Test from the Actions menu. ACT displays the Test Status dialog box, as shown in Figure 10-9.

figure 10-9 the test status dialog box

Figure 10-9. The Test Status dialog box

Running tests with ACT and analyzing their results is covered in more detail in Lesson 2.

Platform Testing

For Web applications, platform testing usually means verifying four main conditions:

Web forms are displayed correctly on all supported browsers and supported versions of those browsers.
The Web application appropriately handles unsupported browser versions for example, by displaying instructions for downloading the required version.
The client is prompted to install any required components, such as Microsoft ActiveX objects or plug-ins, if they are not already installed on his or her computer.
The Web application has acceptable performance over slower forms of network connections, such as modems.

To run platform tests, you need to set up a platform lab that contains computers representing the different hardware and software configurations that your application supports, as shown in Figure 10-10.

figure 10-10 platform testing lab

Figure 10-10. Platform testing lab

Verifying that each Web form is displayed correctly requires some human intervention. ASP.NET renders server controls differently on different browsers, so you should visually inspect your Web application as part of the final testing cycle.