You've already seen how test lists can be used to easily categorize and organize your tests. However, sometimes you need more control over the members of a test group. Another type of test that Team System supports is the ordered test. An ordered test is simply a test that is composed of other logically related tests. As when using test lists, you can add one or more tests as members of an ordered test. Unlike test lists, you can also arrange those tests to execute in a specific sequence. In addition, the same test can be added to an ordered test multiple times.
Don't feel constrained by the term "test" when creating your test suites. There may be perfectly valid cases when a "test" doesn't actually test anything. Perhaps you've created a utility method that erases your customer table. Create a unit test to call this method and add it to your ordered test wherever you need that table reset.
Being able to specify the order of test execution as well as including a test more than once has a wide variety of applications. For example, you may have a Create User web test that, after execution, adds a new user to your database. Your next test, Log User In, may rely on the existence of that new user. By ordering your tests, you ensure that the first test successfully creates the user before the second test attempts to log that user in.
Another important reason to create an ordered test is for cases when you wish to profile more than one test at a time. Because you cannot add more than one test to a performance session, an ordered test, containing any number of related tests, is a great way to work around that limitation. See Chapter 12 for more information on profiling.
An ordered test is simply another type of test, so you create them in much the same way as other tests. Right-click on your test project and select Add Ordered Test. You will see the Ordered Test Editor, as shown in Figure 13-9.
Using the right and left arrow buttons, add one or more of the tests to the ordered test. The list of tests includes the tests from all projects in the current solution. Multiple tests can be added at the same time by holding the Ctrl key while clicking on each test. As mentioned before, the same test can be added more than once.
You may order the execution of the contained tests by adjusting their position with the up and down arrow buttons. The test at the top of the list will be executed first, proceeding sequentially down the list.
One of the key features of an ordered test is that tests run one at a time in a specified sequence. Using the Continue After Failure checkbox, you can indicate whether the ordered test continues to process remaining tests if a test should fail. By default, this is unchecked, indicating that the ordered test will abort when any test fails. Check the box to cause the ordered test to always execute all contained tests, regardless of success.
Find your ordered test in Test Manager or Test View. Right-click and select Properties. You will see the Properties window containing the values for your ordered test, similar to what is shown in Figure 13-10.
You can set the test's Description and temporarily disable the test via the Test Enabled property. Other properties include Timeout, which indicates how long the test should run before failing and the Continue After Failure property, described in the previous section. For descriptions of the Agent Properties and Deployment Items properties, see the "Distributed Load Tests" section of Chapter 15.
An ordered test is executed just like other tests. You can use the Test Manager and Test View windows. You can also use the command-line tool, MSTest.exe, as described in the "Command-line Test Execution" section of Chapter 15.
When executed from Visual Studio, the Test Results window will activate, displaying progress as the test is executing and results when it is complete. In the following example, we have an ordered test with four contained tests. As shown in Figure 13-11, the test has been run, but failed.
Notice that the Test Results window indicates "2/5 passed." The ordered test contained four inner tests, but the ordered test itself counts as a test with regard to success or failure. Right-click on the ordered test in the Test Results window and select View Test Results Details. The main window will display Common Results and Contained Tests.
The Common Results show the overall status of the ordered test, including the start and end time and the ultimate result.
However, in this case, we're interested in finding out what happened with the individual tests contained in the ordered test. The Contained Tests section displays each of the contained tests, along with their type, name, result, and how long the test ran.
You can see that the first two tests have passed. However, the third test, CompareAlbumsTest, did not pass. Because our project's Continue After Failure property was set to False, the remaining fourth test was not executed. Double-click on any inner test to see details for that test.
When you are ready to run the test again, you can repeat the selection process via Test Manager or Test View, use the Test Results' Run button, or run the original test(s) by clicking the "Rerun original tests" hyperlink.