Team System offers powerful collaboration features. Manual tests can be shared with any member of a development team using work items. The following section (and subsections) will show you how to run manual tests and export the results on Team Foundation Server.
To start the test run, go to the Solution Explorer on the right side of the interface, right-click on your project and select Debug Start New Instance. You can also run a test by going to the Test View window and selecting Run Tests. Right-clicking on any of the tests in the Test View window also gives you the option of running individual tests (as opposed to the entire test grouping). Simply select the Run Selection option to execute an individual test.
You can differentiate each test type using the Group By drop-down selector. This option is very useful for isolating the manual tests from the other tests. When you execute a manual test, the initial status is always Pending in the Test Results window. Figure 16-13 illustrates the Test Results.
To do a test run, simply check off the desired test (or tests) in the Test Results view, select the drop-down list above the test results and select Run (as shown in Figure 16-14). You can also individually right-click on any test in the Test View window and select Run Selection (which will have the same effect).
As with all tests, clicking Delete will remove the test from the Test Results window but not remove it from the project.
The edited tests will then appear in the IDE for running, along with a Pass/Fail window. Microsoft Word will only appear when you invoke editing from the Test View or the Test Manager and right-click.
Once you are done, you can evaluate whether or not the test was successful by clicking Pass or Fail, inserting any applicable comments, and then selecting Apply in the Test Results pane. Figure 16-15 shows you how you can select a pass or fail grade on the manual test. You can also add comments relating to the test results.
You can save your manual test results by closing the file. The Test Results window will confirm the Pass or Fail status of your test. If you want to cancel all manual tests in the Test Results window, right-click on any of the tests and select Delete Test Run.
Here is a handy tip: Group manual tests together using test projects to differentiate these tests from the others when viewing your test results on the Team Project portal.
If the test fails, the software test engineer can associate it with a bug report work item.
Once the manual tests have been executed, you can log any resulting test failures as bug work items. To do this, simply right-click on the test that failed and select Create Work Item Bug. You can also add the results to an existing work item by using the Add to Work Item option.
Many of your manual test details will be associated with the bug work item, including the following:
Physical path to the manual test
An attachment containing the manual test results in a Test Results XML file (.trx). The .trx file will re-create the test results for anyone wanting to view and diagnose the test results.
Figure 16-16 shows the bug work item instance that is automatically generated. The middle of the bug work item form enables you to change the assignment, iteration, priority, state, and so forth. The bottom half of the bug work item provides information about the test, including a summary, links, file attachments, and details.
The bug title is critically important: Notice in Figure 16-16 that the columns indicate information provided by default in a Work Item query including the unique ID, Work Item Type, State, Assigned to, and Title. When a work item is generated from a manual test, the name of the manual test is automatically added to the title, but if you don't add a more descriptive title, you will have a hard time finding a specific bug, especially if there are hundreds of nondescriptive titles like "manualtest1" in the work item database. Take the time to briefly describe the issue — for example: "Manual Test: UI Crash on Web Configuration Page."
Once your test run has been completed, you can share the results with the rest of the team by publishing them on Team Foundation Server. After the test results are published, the project manager or test lead can look at test reports (using the SQL Server Reporting Services — see Chapter 19 for more details). The project manager can also assign and manage any new bug work items.
You can publish test results by right-clicking on a test and selecting the Publish icon. The Publish Test Results window will appear (as shown in Figure 16-17), with the name of your test run, the owner, the file size of the results, and the date.
The Team System data store contains an OLAP database with cubes (in other words, multi-dimensional data constructs in SQL Server) that contains data-based representations for tests and work items. Several tables contain work item information, including tfsWorkItemTracking and tfsWorkItemTrackingAttachments.
As an administrator, you don't need to dig into the database to get this information. You can also query all work items using Team Explorer (or you can query the work items using the reporting site), as shown in Figure 16-18.
You can also export the test results in a Visual Studio Test Results File (.trx). This file contains an XML representation of the entire test run data, including the execution time, the name of the host machine, the test type, and much more.
Once the results are exported, it is easy to share them with other members of your testing team, developers, and project leads. The key here is to create reproducible results that anyone can use to eliminate bugs.
You can also import manual tests from Team Foundation Server. You can then open the tests within Test View, which enables you (or anyone else) to reproduce the steps and find the bugs or defects within your target application.