Chapter 17: CapturePlayback Testing

Test automation enables you to get more testing done with the precious time you have. By leaving repetitive testing tasks to machines, testers can spend more time on creating new test designs, improving existing tests, and keeping up with the game code as it changes throughout the project life cycle. Choosing the right projects, people, and automation tools will prevent individual frustration and organizational rejection . Capture/playback testing is one automation approach that you can start working with right away. Ideally, testers should implement automation as a regular part of their job instead of treating it as a project of its own that requires additional effort and budget. Capture/playback automation will come "naturally" to testers who have practice, skill, and more practice.

Capture/Playback Automation Characteristics

Capture/playback test automation is not terribly suited for making "touchy feely" assessments of a game. Automated tests excel at comparing stored information against what occurs during the test, but they cannot judge ease of use, playability, or how fun a game is.

Most off-the-shelf test automation solutions are not suited for random events such as mob spawns, AI opponents, or dealing shuffled cards. To support capture/playback automation in these cases you need access to tools that give you control of the random number seed(s) or other mechanisms that drive the random behavior in a deterministic and repeatable manner. For example, a value can be provided in the game data file for testers to specify which seed to use for the AI, or a special pop-up in-game control panel can be provided for the same purpose. This kind of support needs to be identified ‚ in your test plan or through some other means ‚ early in the project so your automated tests can be developed in time for test execution.

Following are some real benefits to utilizing capture/playback automation:

• Retesting is faithfully repeated exactly as it was done the first time. You will discover intentional and unintentional changes to software this way.

• Problems get noticed and recorded every time, no matter how late in the day it is or how long the test machine has been working.

• Automated tests produce documentation of the test results every step along the way, every time the test is run. Typically other pertinent information is recorded as well, such as date, time, and identification of the software build being tested .

• Different automated tests can run concurrently on multiple computers. One person can easily manage two machines running automated tests.

• Automated tests can be run during non-working days and hours. In some game projects there is no such thing as "non-working days and hours," but perhaps automated testing can re-introduce that concept to your project ‚ .

On the other hand, capture/playback testing has some limitations and drawbacks:

• Capture/playback automation requires some overhead to keep automation scripts up to date with the game code.

• Capture/playback automation won't notice if something goes wrong outside of what is specifically checking for. A test that is checking one part of the UI may not notice when your health suddenly goes to zero or your sword disappears.

• You can't just grab anyone who says they do testing and expect them to be immediately successful at capture/playback test automation. Programming training and experience will make a difference, as will reading and working through this chapter.

Achieving even modest automation goals can have a big impact. If you expend the effort to get just 20% of your tests automated, that could save up to one extra work day a week for each tester.