Working with Test Results


In this section, you'll see how the Test Results window displays the progress and final outcome of test runs. You'll also learn about the secondary uses of the Test Results windows, including exporting and publishing test results.

Test Results window

When a test run is started, the Test Results window is shown, displaying the current status of the run. Each test will update automatically and you can use the control buttons to affect the run, even before it is complete. Click the Pause button to temporarily suspend test execution or click Stop to abort the test run. Whether you pause or stop a run, you will be able to view the results of any completed tests.

As an example, Figure 13-7 shows the Test Results window for a sample test run of four unit tests.

image from book
Figure 13-7

The Result Summary for the entire test run can be accessed by clicking the run status hyperlink, which is the first hyperlink in Test Results. In Figure 13-7, the hyperlink has the text "Test run: failed." The test run status hyperlink, may indicate "failed," "aborted," or "succeeded." The Result Summary will show details about the run start and stop time, the configuration used, and the user who submitted the run.

Simply double-click any test to see the detailed results for that test run. For tests that did not pass, you may see additional details such as the error message and a stack trace for the test failure. You can also right-click and choose Open Test to edit each test.

By default, the run's tests are shown as a simple list, but you can press the View Results by Result List button to show a test list pane. This pane functions much the same as it does with Test Manager. As with creating and managing tests, it is also convenient to view results from a large test run organized by those same test lists.

You may notice that the Failed and Inconclusive test results automatically have their checkboxes checked, while the two tests that passed do not. This is to enable the quick rerunning of tests that did not pass. Simply click the Run button and the checked tests will be executed.

As you have seen with Test Manager and Test View, the Run button has a drop-down to select other options, including Debug. In Test Results, the options also include a Run With selection. Use this to run the selected tests using an alternate Test Run Configuration.

Finally, if your run included any tests that did not pass, you will notice two additional hyperlinked options, "Rerun Original Tests" and "Debug Original Tests." Use these to execute the same set of tests again. Note that this will execute the tests as they were when initially run. If you have made any changes to the tests since the original run, those changes will not be included. To include new changes, run your tests with the Run button or by using Test Manager or Test View.

Sorting, grouping, and filtering tests results

The Test Results window supports exactly the same functionality for sorting, grouping, and filtering results that the Test Manager and Test View windows do for test cases.

Click on any column header to sort by ascending value, and again for descending. You can also right- click and choose Add/Remove Columns to modify which columns are displayed.

Notice the Group By, Filter Column, and Filter Text fields on the right-hand side of the Test Results window's toolbar. For details on their use, see "Sorting, grouping, and filtering tests" in the Test Manager section.

Exporting test results

By default, whenever a test is run, statistics on that run are automatically stored in a file on disk. These files are XML files with an extension of .trx. Team System uses these files to store information about which tests were run, when they ran, statistics on that run, and details about the results.

You can use the Test Results window to manually export test run results. This might be useful if you have disabled automatic test results storage or use the database option. To export all results from the current run, click the Export Results button. You can choose to export a subset of test results, perhaps only for tests that passed, by pressing the arrow on the Export Results button and choosing Export Selected Test Results.

Note, however, that these files can contain a lot of data, especially if you have many tests or have any long-running load tests.

Using Test Results XML (TRX) files

Test Results XML, or TRX, files contain all of the details for selected test runs. Team System establishes a Windows file association for the .trx extension, so double-clicking on such a file will automatically load those results into Visual Studio, where the details can be viewed.

In this way, you can easily copy TRX files between systems to view results with full details on a machine other than the one that originally executed the tests. For instance, you may find this useful for transferring details from your testers to your developers.

You can also use the Test Results window to manually import a TRX file to view its contents. Click the Import Test Results button on the Test Results toolbar, and then select the correct TRX file. If the contained run is already loaded in the IDE, then you will see an error message. Otherwise, you will be able to view the results and details from that exported test run.

The following section describes a more powerful option available, called publishing.

Publishing results

Exporting test results will store the data in a local TRX file. However, if you have a Team Foundation Server instance configured, you can send those test results to it through a process called publishing.

Publishing your run data is vital to your project's success because it enables your team to view and track testing metrics over time. Published data can be queried through various reports, such as Quality Indicators, Code Coverage Details, and Test Run Details.

Before you can publish results, you must be connected to a Team Foundation Server instance. In addition, that instance must have at least one Team Build configured because published results are associated with a specific build. Because run data is associated with a specific build, you can determine the quality of each build by reviewing the test run data for each to identify trends. Team Foundation and Team Build are described in detail in Part 4 of this book.

To publish a test run, click the Publish button located near the center of the toolbar in the Test Results window. When you click this button, you will see the Publish Test Results dialog shown in Figure 13-8.

image from book
Figure 13-8

Note

If the Publish button is disabled, be certain that you are connected to a Team Foundation instance and that your team project has at least one team build.

Choose one or more test runs to publish. You can then select the build with which to associate the run data. Finally, you can choose to include any code coverage data. Click OK and your results will be published. You will see the Publish View mode of the Test Results window, with details about each publication, including the status and any error message.

When your publication shows a status of Completed, your data has been saved to the operational store for your team project.

Note

Published data is not immediately available for reporting. The data can be queried once it has been pulled into the data warehouse. This is done automatically at a frequency you can set in your Team Foundation Server instance.

You can also publish test run data from the command line using the MSTest console application. See Chapter 15 for more details on command-line testing.



Professional Visual Studio 2005 Team System
Professional Visual Studio 2005 Team System (Programmer to Programmer)
ISBN: 0764584367
EAN: 2147483647
Year: N/A
Pages: 220

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net