Identifying Bugs

Game testing has two purposes. The first is to find defects that are in the game code or design. The second is to demonstrate which parts of the game are working properly. When the test doesn't find any problems, it "Passes." When a test finds a problem, it "Fails."

Another possible outcome of a test is "Blocked," which means an existing problem keeps you from getting to other parts of the test ‚ such as when the PC version of the American Idol game crashes after you reach the final 10 (see the following sidebar). This blocks you from doing any testing on the final rounds of the contest.

The test could also be "Not Available," meaning the part you are supposed to test has not been included in the version of the game you were given to test. It might be because the developers are still in the process of getting all the game elements put together, so a level, item, function, or character is intentionally left out of the test release. It could also be that the game version for the platform you are testing does not include what you are trying to test ‚ such as True Crime: Streets of LA multiplayer modes, which are only available for the PC.

Title > Onboard Sound Menu Crash Update

Date Uploaded > 28th November 2003

Description > To solve the problems with the American Idol game crashing back to the Windows desktop when navigating the menus at the start of the game OR once the player is through to the final 10 of contestants.

http://www.codemasters.com/downloads/?downloadid=12431

Here Comes the Judge

Not every tester will notice the same defects when testing the same part of a game. Likewise, not every tester will run the same test in the same way. Psychology may have an explanation as to why this happens, in the form of the Myers-Briggs Type Indicator (MBTI). One of the categories in this indicator rates a person as either a Judger or a Perceiver.

Judgers like to be scheduled, organized, and productive.

If you are a Judger, you prefer a structured, ordered, and fairly predictable environment, where you can make decisions and have things settled . You are serious and formal. You like to make decisions. You like to organize and make plans. You pay attention to time. You use schedules and timetables as a guide. You like to work first, play later. You like to finish projects best. You are settled, decisive , routinized, and predictable. You dislike surprises and need advanced warnings. You need issues settled. You get things done as soon as possible. Others may see you as concise , focused, and hard-working. You tend to use the directive communication style. (e.g. "Ask Jerry for specific instructions on balancing the budget.") You want to arrange things to happen as you want them to happen. You are planning for the future. You are good at listing tasks and developing timelines . You see the need for most rules.

Perceivers like to be flexible, curious , and nonconforming .

If you are a Perceiver, you prefer to experience as much of the world as possible. You like to keep your options open and are most comfortable adapting. You are playful and casual. You like to postpone some decisions, if you can. You like to keep plans flexible. You like to wait-and-see. You are less aware of time or late. You do whatever comes up. You like to play first, work later. You like to start projects best. You are flexible, spontaneous , unpredictable, and tentative. You are more carefree, leisurely, and disorganized. You enjoy surprises and spontaneous happenings. Others may see you as verbose and scattered . You don't like anything unalterable. You tend to use the informative communication style. (e.g. "Jerry has some information that might help you balance the budget.") You are interested in watching things unfold. You question the need for many rules.

http://www.boomspeed.com/zsnp/mbti.htm

Note ‚

Not sure if you are more of a Judger or Perceiver? You can take the temperament test at www.personalitytest.net/cgi-bin/q.pl to find out.

The tendency toward one of these behaviors versus the others will manifest itself in the way you approach testing, and the kinds of defects you tend to find. For example, a Judger is good at following step-by-step instructions, running through a lot of tests, and finding problems in game text, the user manual, and anywhere the game is inconsistent with historical facts. The Perceiver tends to wander around the game, come up with unusual situations to test, report problems with playability, and comment on the overall game experience. Judgers will verify the game's "authenticity" and Perceivers will verify its "funticity."

Conversely, there are things Judgers and Perceivers may not be good at. A Judger may not do steps or notice problems that aren't in the written tests. A Perceiver may miss seeing problems when running a series of repetitive tests. Although testing without written tests provides more freedom, Perceivers may not always have good documentation of how they got a bug to show up.

You are probably not 100% of one type, but you most likely have a tendency toward one or the other. Don't treat that as a limitation. Use that knowledge to become more aware of areas you can improve so you can find more bugs in the games you test. Your goal should be to use both sets of qualities at the appropriate times and for the right purpose. When you see a bug that someone else found and it makes you think "Wow! I never would have tried that," then go and talk to that person and ask her what made her think of doing that. Do this often and you can start to find those same kinds of bugs by asking yourself "How would Linda test this?". Make sure you share your own "bug stories" too. A couple of books on software defects that can give you some more insight are: Computer Related Risks , by Peter G. Neumann, and Fatal Defect , by Ivars Peterson.

Table 2.1 shows some of the ways that each personality type affects the kinds of bugs testers will find and what kinds of testing are best used to find them.

Table 2.1: Tester Personality Comparison
Judger	Perceiver
Runs the tests for ‚	Finds a way to ‚
Conventional game playing	Unconventional game playing
Repetitive testing	Testing variety
User manual, script testing	Gameplay, usability testing
Factual accuracy of game	Realistic experience of game
Step-by-step or checklist-based testing	Open-ended or outline-based testing
May rely too much on test details to see defects	May stray from the original test purpose
Concerned about game contents	Concerned about game context