Test Case Planning: The Headhunter Method


Quality assurance can be tiring, tedious work. Do what you can to make it fun, but also ensure that the job is getting done. One way to remove the tedium from tracking bugs down is to treat it as a challenge. The following four-step method can help you in this regard.

Step One: Focus on the Target

Test case design starts at the earliest design phases of a game. If the game design calls for lots of role-playing action, you can start gearing your test cases toward heavy database test cases. If it calls for a first-person shooter, then in the initial phases of game testing you can expect to work with more API and game engine testing.

You also want to orient your testing toward the current milestone's requirements and to quantify your results. By using metrics, you can quickly establish baselines for each milestone and see where the project stands from a quality standpoint at any time.

For example, if the milestone requirement states that the game engine needs to be in a "playable state," you quickly need to define "playable." It could mean that the game runs at 20 frames per second or more. It could also mean that 30 out of 40 planned features are functional. Work with the designers and programmers to quantify the expected results of your test.

Step Two: Lay Out the Bait

Now that you have selected a goal, plan the test's execution accordingly. The basic rule is: if something can be measured, it can be tested. Thus, it is often best to work backward when preparing a test case. Ask yourself: if the game were to fail in this area, how would I catch it?

In addition, don't be afraid to combine multiple test cases into one, but be careful not to make your test scenarios so long or complicated that you can't analyze the results.

For example, consider boundary testing. Here you are testing the input and output limits of the program. How many commands can you queue up before the interface breaks? Exactly how many items can you carry? What if your text input box contains more than 255 characters?

Focus on visible state transitions. Every time you perform an action that changes the range of choices available to the player or modifies the display, you have made a state transition. The category includes menu systems, chat windows, display screens, and GUI changes. Test each option in every menu.

Testing the input/output interfaces can also reveal bugs. When the game is accessing the hard drive or CD-ROM, send a heavy load of commands. Can the I/O interface handle the load? What about networking? If too many packets are dropped or a connection is lost, how does it affect the game? Check for malformed data instructions or packets. Seeing how your game can handle extreme cases can quickly show areas of weakness in your programming.

Step Three: Set the Trap

You've planned your test case and now you are ready to execute. The first consideration is your test environment. The machine you are testing on should be as generic and pristine as possible to avoid any potential contamination problems from previous tests and exotic hardware. Therefore, in a test lab, it is advisable to set up a "ghost hard drive" image to copy to other machines when needed to start a new test case.

Make sure you have the tools necessary to perform the test and properly record a log of the test. There are plenty of tools freely available on the Internet that can capture screens, network packets, and chart running API threads.

Set your test run's objectives ahead of time. If testing in a multiplayer environment, make sure everyone involved is trained in proper testing procedures and has received precise instructions concerning their roles. Nothing is more frustrating than one person forgetting what to do at a critical moment in the test.

Step Four: Capture the Bug

When a bug is found, make sure that you have enough information to document it. Were screen captures made? Are the logs complete? The more information you have, the better. If time permits, try to replicate the bug as many times as possible. Vary the conditions to see if you can narrow the variables that cause the bug.

If no bug was found, the game might be working, or your plan might be flawed. Remember, you test to find bugs, not to see if everything in the game is working correctly. Go back to the drawing board. Alter the conditions somewhat in the area that you are testing. Throw everything you have at that area of the game. If your tests continue to reveal no bugs, save your test cases for another day for regression testing.

Retracing Your Steps

The fact that a bug wasn't found one day doesn't mean that it can't be found the next—or in the next version of the game. This is where regression testing comes into play.

Regression testing means retracing your steps and rerunning all your test cases. Perform it at successive milestones, and especially when code is locked down, at which point you should be able to run all of your test cases without a single failure. If one does occur, a bug has been inserted since the last round of regression testing. Fix the code and try again until you are able to achieve a perfect success rate.

Don't Forget to Test the Installation Routine

Once you have built your gold master, don't forget to design a few test cases for the installation and removal routines. With console design, the installation routine isn't important, but it can cause major headaches on personal computers. More than one software recall has been caused by an uninstaller that erased the entire hard drive!

If you have any extra material being placed on the master, make sure that they conform to company standards and formats. While the inclusion of static art assets for the enjoyment of the user wouldn't require a lot of formalized testing, don't forget to test other dynamic content such as screensavers, videos, and even music to ensure that they work with most platforms and players.




Secrets of the Game Business
Secrets of the Game Business (Game Development Series)
ISBN: 1584502827
EAN: 2147483647
Year: 2005
Pages: 275

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net