Managing Test Cases

As you start to use the variety of test types supported by Team System, you may quickly find yourself with many tests in your solutions. Team System projects often have dozens and in some cases hundreds of unit tests alone. Add suites of web, load, and other tests and you would quickly have difficulty finding the right tests at the right time.

Fortunately, Team System offers several ways to easily manage, categorize, and locate tests. In this section, we'll cover the Test View and Test Manager windows. These windows are your main interface for organizing and executing all of your tests.

Test Manager

Test Manager, available only in Team Edition for Software Testers, provides many features for organizing, editing, and executing your tests. You can use the Test Manager for all of your solution's tests, regardless of type. To access Test Manager, select Test Test Manager or click the Test Manager button on the Test Tools toolbar. You will then see a window similar to that shown in Figure 13-3.

Figure 13-3

Test Manager features two panes. The right pane displays your solution's tests. The left pane is for creating, selecting, and managing groups of tests called test lists. We describe these panes in detail in the next section.

Working with tests in Test Manager

To get started, click the All Loaded Tests node of the tree in the left-hand pane. Double-click any test to open it for editing. The appropriate editor will load, depending on the type of test selected. You can also modify properties by right-clicking on a test and choosing Properties. If you already have the Properties window open, simply single-click on a test to view its properties.

Selecting and running tests

Each of your tests has a checkbox next to it. To run tests, check the box next to each test you wish to execute and then press the Run Checked Tests button on the Test Manager's toolbar. This button features another mode, Debug Checked Tests, which can be accessed using the button's drop-down arrow. The debug mode causes execution to pause at any breakpoints you have set, at which point you can debug as you would any other application.

You can also right-click in the test list pane and choose Run Checked Tests or Debug Checked Tests.

Note

Note that Test Manager executes only checked tests. Tests that are highlighted but not checked will not be included in your test runs.

Running tests will activate the Test Results window, where you can track progress and view the results of all executed tests.

Disabling tests

To disable one or more tests, simply highlight the tests (checking them is not required) and then right- click and choose Disable. To restore them, repeat the process, selecting Enable. Disabled tests cannot be checked for execution, even if a containing test list is selected.

Disabling is a useful way to temporarily exclude a test, perhaps one that is flawed and needs correction, without having to remove it from any containing test lists. You can continue to use test lists to quickly select tests to run, knowing that your disabled tests will not be included.

Test lists

Another easy and effective way to organize your tests is by grouping them in test lists. Test lists are groups of tests that you wish to logically associate for any reason, perhaps by type, target, or purpose.

Think of test lists much like playlists of songs in an MP3 player, which are pointers to songs you want to play together. Just as you might load your "Upbeat" playlist for your Friday commute home, you could have a test list called "Offline Tests," which you can run without a network connection.

Creating test lists

To work with test lists, open Test Manager and note the pane on the left side. As shown in Figure 13-3, the left panel includes a tree view containing various groupings of tests. Right-click the Lists of Tests node and choose New Test List. The dialog shown in Figure 13-4 will be shown. You can also create a new test list without opening Test Manager by choosing Test
Figure 13-4

Give your test list a name, optionally a description, and then choose a location to place the test list. Click OK and you will see your list in the test list tree.

Note

Test lists and links to the tests they contain are stored as XML in your solution's test meta-data file, the file with an extension of .vsmdi located in the Solution Items folder. This makes it easy to reuse those tests in other solutions. We'll describe importing these files later in this section.

Note that test lists can contain other test lists. In Figure 13-4, we've created two main grouping lists, "Unit Tests" and "Web Tests." Under these, we created additional lists, such as "Album Tests."

Populating test lists

Once you have created a test list, you can drag tests from Test Manager onto the list. You can also create new tests directly in a test list by right-clicking on the test list and choosing New Test.

Keep two rules in mind when working with test lists. Each test list can contain only one instance of the same test, but tests can appear in more than one test list. By default, dragging tests into a test list is considered a move operation. If that test was dragged from one test list to another, it is removed from the source list and added to the target.

If you want a test to appear in more than one list, you can copy, instead of move, the test. To copy a test, right-click on any test and choose Copy. Then select the target test list and choose Paste. The test will then be in both the source and target lists. You can also hold the Ctrl key down while dragging a test to cause a copy operation.

Working with test lists

Test lists allow you to easily select all related tests for execution by checking the box next to the list name. In the Test List pane, click the checkbox next to one or more test lists you wish to run. All tests in that list, except disabled tests, will be checked. Tests in any child test lists will be selected as well.

As a convenience, if you happen to select all of the tests that comprise a test list through other means, such as the All Loaded Tests list, the containing test list will automatically be checked.

Special-purpose test lists

You've seen how to create your own lists in the Lists of Tests node in the Test List pane's tree. Below that are two other nodes, Tests Not in a List and All Loaded Tests. These are special lists that you cannot directly modify.

All Loaded Tests shows every test in your solution, regardless of their membership in test lists. This view is very useful for an overall picture of your tests. Dragging a test from this group to a test list has no effect on the list of tests in this group.

The other group, Tests Not in a List, shows any tests that have not been included in at least one test list. Use this view to quickly find any tests that might need categorization. If you drag a test from this group to a test list, it will automatically be removed from this group.

Choosing columns to display

By default, Test Manager displays the name, containing project, and type for each test. You can easily add or remove columns by clicking the Add/Remove Columns button or right-clicking in the list pane and choosing Add/Remove Columns.

The Test View window, described later, features similar support for changing which columns display.

Sorting, grouping, and filtering tests

Test lists are only one of the options for handling a large number of tests. The Test Manager and Test View windows both support grouping, sorting, and filtering of tests.

Note that once set, sorting, grouping, and filtering options affect all views of your lists and will remain the same as you switch between test lists or to the View All Tests mode.

Sorting

As mentioned before, you can customize which columns appear in the test listing pane. In addition, you can sort by the values of any of those columns. Simply click on a column name to sort the contents in ascending order. Click again to switch to descending order.

Grouping

By default, tests are displayed as a simple list, perhaps sorted by a column. These lists can get extremely long, especially when you're looking at the All Loaded Tests group in the Test List pane.

The Group By drop-down list enables you to choose a specific column. The lists will then be grouped according to their values for that column. This is especially useful because those groupings can be collapsed to make the list shorter and more manageable.

Note

Any column can be used for grouping. The column does not have to be one of the currently displayed columns.

For example, in Figure 13-5, all of the tests in the current solution are being displayed, but grouped by the name of the test list they are in, if any. Tests with no value for that column are placed together in the No Test List Name group.

Figure 13-5

To revert to an ungrouped listing, choose the [None] entry in the Group By field.

Filtering

Another powerful option, complementary to grouping and sorting, is filtering. Filtering restricts the tests to show only those that match a specified value for a column.

The second drop-down list, Filter Column, by default contains [All Columns], and enables you to select a filter column. You then specify the value to match by typing it in the Filter Text combo box, which contains <Type Keyword> by default. Press the Apply Filter button to the right of the Filter Text box and the list will be updated, showing only tests that match that value.

Tests are considered matching if any part of that column's value matches the text you entered. For example, if you enter "web," both "WebTest1" and "TestWebLogout" would match and be displayed.

The Filter Text box maintains a history of values. Use the drop-down list to quickly access previously created filters.

Filtering is an excellent complement to grouping. For example, if your solution contained multiple projects with many tests, already grouped by test type, you might find it useful to limit your view to just those tests in a specific project. Select the Project entry in the Filter Column list, enter the name of the project you wish to view, and press the Apply Filter button.

You could achieve similar results grouping by project name and sorting by test type. The final list would be longer, but you could collapse the groups of tests from other projects. You would also need to ensure that the Test Type column is added to the current view in order to perform the sort.

Loading and importing test meta-data files

As mentioned earlier, a solution that contains one or more test projects will have a test meta-data file with an extension of .vsmdi located in the Solution Items folder. This XML file stores details of the test lists in your solution's test projects as well as links to the tests those lists contain. These files can be useful when you want to reuse tests for other purposes — for example, in another solution or in a build environment.

A solution may have only one test meta-data file. If you have a solution without a test project, you can load a test meta-data file by clicking the Load Metadata File button on the Test Manager toolbar. If your project already has a test meta-data file, you can append other test lists by using the Import Meta-data File. Imported meta-data files are merged with the existing solution's meta-data file.

When you load or import a test meta-data file, Visual Studio displays the new test lists and attempts to locate any linked tests. Tests that could not be resolved will be marked with a warning icon. To help resolve these broken links, add the test project that contains those tests to your current solution.

Test View

The Test View window is available in both the Tester and Developer editions of Team System. It offers a streamlined interface very similar to that of Test Manager, but without some of the functionality.

To open the Test View window, choose Test Windows Test View or click the Test View button on the Test Tools toolbar. You will see a docked window as shown in Figure 13-6.

Figure 13-6

There are several differences between Test View and Test Manager. Perhaps most noticeably, listed tests have no associated checkboxes. Another significant difference is that the ability to select and create test lists is not supported in Test View. Finally, the Test View toolbar is slightly simplified, removing some options such as loading and importing test meta-data files.

Selecting and executing tests in Test View is simple and fast. Just highlight one or more tests and press the Run Tests button. You can also right-click and choose Run Selection or Debug Selection. To select multiple tests, hold down Ctrl while clicking on each test. To select a group of consecutive tests, click the first test, hold down Shift, and click on the last test.

Note

Unlike Test Manager, tests in Test View need only be highlighted to be executed.

As with Test Manager, you can double-click on any test to edit it. You may also right-click to access features such as deleting a test, creating a new test, and disabling/enabling a test. You may also choose which columns appear by right-clicking and choosing Add/Remove Columns.

Because Test View does not support test lists, the list of tests can be extremely long. Fortunately, Test View supports the same grouping and filtering options that Test Manager offers. For details, see the "Sorting, grouping, and filtering tests" section.