Test Tools

As we discussed in the previous chapter, the testability of your application will have a big impact on your need for test tools. In most cases, investing in the design of the system to allow you to run your tests below the user interface will provide a better return than purchasing a tool to run them through the user interface. Despite all the promises, test automation tools have historically struggled to be worth the effort of using, and probably more shelf space is tied up by test tools no one uses than by any other type of software, including CASE tools (remember those?).

If you do decide to go that route, make your list of necessary and desirable features and avoid being bowled over by vendors who want you to think they have the next test-automation silver bullet. Don't buy anything until you've made it work the way you want on your application.

Let's say you do need to run a lot of tests through a user interface, and your team can't figure out how to evolve a tool to do this or feels it would be too expensive. Commercial capture/playback tools look tempting, but captured scripts are murdered by any change to the application. Look for a tool you can interface your executable tests to, as described in Chapter 23. The more easily this can be done, the better. Ideally, the tool's scripting language would be the same as the one you're using for the system and the executable acceptance tests, or at least one in which you're fluent. At worst, it should have basic programming constructs for control flow, modularity, and parameter passing.

You can use the tool's capture facility to create the parts of the tests that interact directly with the user interface, but you'll need to program the control structures and interfaces with the executable tests in the tool's scripting language. By combining the tool with the executable tests under the JUnit framework, we've found it possible to develop, within the timeframe of an XP iteration, reasonably low-maintenance tests that run through the user interface.