Chapter 17: Generic Testing

Overview

Generic tests enable you to reach outside of the test framework by running external executables, which return results back to Team Test. Generic tests are useful for supporting current test processes that you may already have in place by wiring them to Team Test. They are also useful when there are complex steps to be performed by third-party tools, or when you want to create your own external testing tools.

Generic tests offer easy extensibility to the Visual Studio Team Test toolset, enabling easy integration of external tools to the test framework. No code is required to step outside the boundaries of Visual Studio and integrate with custom tools. Generic tests use the same test framework as the other test types, but instead of producing test results, they return the results of an externally run application to the test framework.

During the execution of a generic test, an external application is run. The application can return a simple pass or fail by setting its exit code, also known as the ErrorLevel, or it can return an XML file with the specific results of any number of tests that were run as part of the external application.

Figure 17-1 shows the integration of generic tests and Visual Studio Team Test. Notice that the generic tests are part of the Visual Studio Team Test system, but run external applications.

Figure 17-1

You can create your own test types using the test framework's extensibility, but this is such a complex process it deserves a book of its own. By using a generic test, you can obtain almost instant extensibility by creating your own external applications to perform any custom test or operation and then report the results back to the generic test.

Additional tools for extending the test tool framework are available as part of the Visual Studio 2005 Team System Extensibility Kit available for download from MSDN (http://www.msdn.microsoft.com/vstudio/extend/SDKDownload).