Chapter 8: The Test Process

Programmers don't fully test their own games. They don't have time to, and even if they did, it's not a good idea. Back at the dawn of the videogame era, the programmer of a game was often also its artist, designer, and tester. Even though games were very small ‚ the size of email ‚ the programmer spent most of his time designing and programming. Little of his time was spent testing. The programmer of Astrosmash , a space shooter for the Intellivision system, made an assumption when he designed the game that no player would ever score 10 million points. As a result, he didn't write a check for score overflowing. He read over his own code and ‚ based on his own assumptions ‚ it seemed to work fine. It was a fun game ‚ its graphics were breath -taking (at the time) and the game went on to become one of the best-selling games on the Intellivision platform.

Weeks after the game was released, however, a handful of customers began to call with complaints. When they scored more than 9,999,999 points, the score displayed negative numbers , letters , and other non-numeric symbols. This was after the catalog described the game as having "unlimited scoring potential." The problem was exacerbated by the fact that the Intellivision console had a feature that allowed players to play the game in slow motion, making it much easier to rack up high scores. John Sohl, the programmer, learned a hard lesson: The user will always surprise you .

"Black Box" Testing

Almost all game testing is black box testing; testing done from outside the application. No knowledge of, or access to, the source code is granted to the tester. Game testers typically don't find defects by reading the game code. Rather, game testers try to find defects using the same input devices available to the normal player. Black box testing is the most cost-effective way to test the extremely complex network of systems and modules that even the simplest videogame represents.

Figure 8.1 illustrates some of the various inputs you can provide to a videogame and the outputs you can receive back. The most basic of inputs are positional and control data in the form of button presses and cursor movements. These can come from a variety of input devices: joysticks, keyboards, mice, and such esoteric devices as dance pads, bass fishing controllers, maraca controllers, and drum controllers. Audio input can come from microphones in headsets or attached to a game controller. Video input can come from USB cameras . Input from other users can come from a second controller, a local network, or the Internet. Finally, stored data such as saved games and options settings can be called up as input from memory cards or a hard drive.

Figure 8.1: Black box testing ‚ planning inputs and examining outputs.

Once some or all of these types of input are received by the game, it reacts in interesting ways and produces such output as video, audio, vibration (via force feedback devices), and data saved to memory cards or hard drives .

The input path of a videogame is not one-way, however. It's a feedback loop, where the player and the game are constantly reacting to each other. Players don't receive output from a game and stop playing. They constantly alter and adjust their input on the fly based on what they see, feel, and hear in the game. The game, in turn , makes similar adjustments in its outputs based on the inputs it receives from the user. Figure 8.2 illustrates this loop.

Figure 8.2: The player's feedback loop adjusts to the game's input and vice versa.

If the feedback received by the user was entirely predictable all the time, the game would be no fun. Nor would the game be fun if the feedback received by the player was entirely random all the time. Instead, feedback from games should be just random enough to be unpredictable. It is the unpredictability of the feedback loop that makes games fun. Because the code is designed to surprise the player and the player will always surprise the programmer, black box testing gets testers to think and behave like players.