Creating an External Testing Tool | Professional Visual Studio 2005 Team System (Programmer to Programmer)

Here are some of the scenarios for which it may be useful to create your own external application, rather than write a unit test or other test type:

Supporting legacy test applications: You have already devoted resources to creating your own test tools. These tools now need to be modified to interact with Team Test. Using this code, you can easily modify your existing test environment.
Developing new external test applications: You may need a test application completely independent of Visual Studio. This is common when a test needs to be run on large amounts of code or when you want to provide custom testing tools for customers who do not have Visual Studio. For example, you could use this to create a tool to verify that assemblies are all marked with the same version number.
Logging test run data: If you just want to report data to the test run results log without actually putting any code under test, you can quickly create a console app that writes to the console. An example would be reporting all the files, dates, and times in the solutions' directory. The console output will be part of the test results.
Wiring other test systems to Team Test: If you already have a large test base in another test framework and you want to continue using these coded tests, you can create a simple conversion application to translate the results of the existing test framework into results that Team Test can understand. This is covered in the section "Wiring to Existing Test Systems" later in the chapter.

Scripting Host example

The Windows Scripting Host provides a quick and easy way to create specialized external applications for generic tests. You can use these scripts to set the stage for other generic tests or perform set up for unit tests that are to follow, such as copying files, installing an application, verifying that certain disk resources are available, or much more.

The advantages of using a script as a generic test target are that they are quick to create, are easily modifiable, do not require .NET expertise, and are easy to interoperate with many different objects on a system. They are also useful as a more advanced equivalent to DOS batch files.

Microsoft's Developer's Network has a great scripting reference with information on all the objects used here. See http://www.msdn.microsoft.com/library/en-us/script56/html/vtoriMicrosoftWindowsScriptTechnologies.asp for more information.

To use this example:

Copy the code into a file titled QuickTest.js under the C:\Temp folder.
Create a new generic test.
In the "Specify an existing program…" field, use the command-line Windows Script Host application C:\WINDOWS\system32\cscript.exe.
In the "Command line arguments…" field, provide the path to your script — for example, /nologo "C:\Temp\QuickTest.js."
5. Close and save the generic test editor document.
Run the test from the Test View window.
Double-click the test in the Test Results window. You will see the standard command-line output at the bottom of the test results.

Here is the code:

     // Instantiate scripting shell support     var WshShell = WScript.CreateObject("WScript.Shell");     // Intantiate file system support     var fso = WScript.CreateObject("Scripting.FileSystemObject");     // Obtain the collection of environment variables available to this process     var WshSysEnv = WshShell.Environment("PROCESS");     // Retrieve the env var that points to the tests' output directory     var TestOut = WshSysEnv("TestOutputDirectory");     // If the environment variable is not set, quit with an error     if (TestOut == "") WScript.Quit(1);     // Report the date & time the directory was created     WScript.Echo("%TestOutputDirectory% = " + TestOut);     WScript.Echo(fso.GetFolder(TestOut).DateCreated);     // Return Successful, this script is just for reporting data     WScript.Quit(0);

Managed Code example

The scenario here is that your company, which has 20 developers working on five different projects, wants to ensure that all of its assemblies are marked with the correct company name. Because projects come and go and all teams must run this verification, it is advantageous to place the code to perform the verification across .NET assemblies in a separate, standalone executable.

The standalone application can then be easily run from the command line, tied to a series of other such command-line applications with scripts, added as part of a build process, or added to a suite of tests with a generic test.

This particular code example uses the following command-line syntax:

Note

AssemblyCompanyNameVerifier "Your Company Name" file [file|directory…]

For this example, "Your Company Name" should match the line [assembly: AssemblyCompany ("Your Company Name")], usually in AssemblyInfo.cs, with quotes. file is a .NET assembly file to be verified, and directory can be a directory containing .dll .NET assemblies to be verified.

The possible errors and resulting ErrorLevel exit codes are defined in the ExitCode enumeration. An exit code of 0 indicates success.

     #region Namespace Inclusions     using System;     using System.IO;     using System.Diagnostics;     using System.Reflection;     using System.Collections.Generic;     #endregion     namespace AssemblyCompanyNameVerifier     {       class Program       {         // ErrorLevel codes to be set as the exit code         public enum ExitCode { Success = 0, WrongArguments = 1, FileNotFound = 2,                                WrongName = 3, CantLoadAssembly = 4,                                CompanyNameNotFound = 5 }         static int Main(string[] args)         {         // Verify the minimum number of arguments are passed         if (args.Length < 2) return ErrorLevel(ExitCode.WrongArguments);         // Obtain the company name from the command line which will be used to verify         // assemblies against         string CompanyName = args[0];         // Create a list to contain the files to be verified         List<string> AssemblyFiles = new List<string>();         // Traverse the remaining command line arguments         // If they point to a directory, add the *.dll files in the directory         // If they point to a file, add the individual file         for (int i = 1; i < args.Length; i++)           if (Directory.Exists(args[i]))             foreach (string file in Directory.GetFiles(args[1], "*.dll"))               AssemblyFiles.Add(file);           else if (File.Exists(args[i])) AssemblyFiles.Add(args[i]);           else return ErrorLevel(ExitCode.FileNotFound);         foreach (string file in AssemblyFiles)         {           // Track if the CompanyName attribute is found           bool FoundName = false;           // Attempt to load this file as a .NET assembly           Assembly a = null;           try { a = Assembly.LoadFile(file); }           catch (BadImageFormatException)           { return ErrorLevel(ExitCode.CantLoadAssembly); }           // Traverse each company name assembly attribute           // and make sure the name matches           foreach (AssemblyCompanyAttribute attrib in                    a.GetCustomAttributes(typeof(AssemblyCompanyAttribute), true))           {            if (attrib.Company == CompanyName) FoundName = true;            else { ValidName = false; return ErrorLevel(ExitCode.WrongName); }           }           // If the company attribute was not found, return the appropriate error           if (!FoundName) ErrorLevel(ExitCode.CompanyNameNotFound);         }         // The correct name was found in each assembly         return ErrorLevel(ExitCode.Success);         }         public static int ErrorLevel(ExitCode code)         {           // Create a simple little report           string report = "Result: " + code.ToString();           // Send the output to the console and debug terminal           Console.WriteLine(report);           Trace.WriteLine(report);           // Return the Exit Code           return (int)code;         }       }     }

The primary output from this app, other than the name of the ExitCode enumeration value, is the ErrorLevel (or exit code) returned from the Main method.