List of Figures

Chapter 1: Two Rules of Game Testing

Figure 1.1: Resources balanced with project scope.

Figure 1.2: Budget reduction causes pressure.

Figure 1.3: Budget and headcount pressure caused by project scope increase.

Figure 1.4: Low trust means more testing.

Figure 1.5: More trust leads to less testing.

Figure 1.6: Our world (left) and the tester loving world (right).

Chapter 2: Being a Game Tester

Figure 2.1: Game testing activities.

Figure 2.2: Knights of the Old Republic character selection screen initial state showing Male Scoundrel.

Figure 2.3: DevTrack's new defect window.

Figure 2.4: Defect title and description.

Figure 2.5: Defect priority selection.

Figure 2.6: Defect type selection.

Chapter 3: Why Testing is Important

Figure 3.1: Mainline of a simple version tree.

Figure 3.2: A version tree with a branch.

Figure 3.3: Merging back to the mainline.

Chapter 4: The Game Team

Figure 4.1: 3D skeleton frame.

Figure 4.2: Super Street Racer game state flow.

Figure 4.3: Super Street Racer Dealership screen layout.

Figure 4.4: Job role/project assignment matrix.

Chapter 5: The Game Production Cycle

Figure 5.1: Train chase concept art from Everything or Nothing .

Figure 5.2: Story patterns of varying length and complexity.

Chapter 6: Software Quality

Figure 6.1: Sigma table excerpt for various sizes of delivered software.

Figure 6.2: Game code phase containment data.

Figure 6.3: Game code phase containment graph.

Figure 6.4: Game code phase containment data with expanded test categories.

Figure 6.5: Game test phase containment data.

Figure 6.6: Game test phase containment graph.

Figure 6.7: Pareto Chart of defects per KLOC for each game subsystem.

Figure 6.8: Control chart of weekly code change in KLOC.

Figure A.1: Control Chart for review rates.

Chapter 7: Test Phases

Figure 7.1: Console racing game timeline.

Figure 7.2: Typical entry in a bug database.

Figure 7.3: Test kickoff checklist.

Figure 7.4: Test creation kickoff checklist.

Chapter 8: The Test Process

Figure 8.1: Black box testing planning inputs and examining outputs.

Figure 8.2: The player's feedback loop adjusts to the game's input and vice versa.

Figure 8.3: The testing process feedback loop.

Figure 8.4: Portion of a test suite for Minesweeper .

Chapter 9: Testing by the Numbers

Figure 9.1: Planned and actual test execution progress data.

Figure 9.2: Planned and actual test execution progress graph.

Figure 9.3: Test completion rate per tester per day.

Figure 9.4: Tester participation rate calculations.

Figure 9.5: Test Effectiveness measurements.

Figure 9.6: Test Effectiveness graph.

Figure 9.7: TE measured for individual testers.

Figure 9.8: Defect severity trend data.

Figure 9.9: Defect severity trend graph.

Figure 9.10: Empty Star Chart.

Chapter 10: Combinatorial Testing

Figure 10.1: First two columns of Jedi combat test.

Figure 10.2: Complete three-way combinatorial table for Jedi combat test.

Figure 10.3: Adding Force choices for the Male rows.

Figure 10.4: Adding the first Force choice for the Female character tests.

Figure 10.5: Completed pairwise combinatorial table for three Jedi combat parameters.

Figure 10.6: ESPN NFL 2K5Game Options screen.

Figure 10.7: Starting the NFL 2K5 Game Options test table.

Figure 10.8: Adding the second set of Play Calling values.

Figure 10.9: Completing the Play Calling column.

Figure 10.10: Starting the Game Speed column.

Figure 10.11: Adding the second set of Game Speed values.

Figure 10.12: Completing the Game Speed column.

Figure 10.13: Starting the Challenges column.

Figure 10.14: Adding the second set of Challenges values.

Figure 10.15: Completing the Challenges column.

Figure 10.16: Starting the Coach Mode column.

Figure 10.17: Adding to the Coach Mode column.

Figure 10.18: Completing the Coach Mode column.

Figure 10.19: Starting the Multiple Routes column.

Figure 10.20: Restarting the Multiple Routes column.

Figure 10.21: Adding to the Multiple Routes column.

Figure 10.22: The completed Game Options test table.

Figure 10.23: HALO Advanced Controls option selection screen.

Figure 10.24: Look Sensitivity values placed into table template.

Figure 10.25: Invert Thumbstick values added to the table.

Figure 10.26: Completed Advanced Controls table.

Figure A.2: ESPN NFL 2K5 game options test table with seven parameters.

Figure A.3: ESPN NFL 2K5 game options test table with 2 min value combinations.

Figure A.4: ESPN NFL 2K5 Create Player pairwise combinatorial table.

Chapter 11: Test Flow Diagrams

Figure 11.1: Flow components .

Figure 11.2: Welcome to SmartDraw! window.

Figure 11.3: Create New Drawing window.

Figure 11.4: Arrowheads window.

Figure 11.5: Starting the Ammo TFD.

Figure 11.6: TFD after picking up a weapon.

Figure 11.7: TFD with HaveGun and HaveAmmo states.

Figure 11.8: Return flows added from HaveGun and HaveAmmo .

Figure 11.9: Flows added to get both gun and ammo.

Figure 11.10: Return flows added from HaveGunHaveAmmo .

Figure 11.11: Loaded gun and shooting flows added.

Figure 11.12: Flow added to shoot gun with no ammo.

Figure 11.13: The completed Ammo TFD.

Figure 11.14: Unreal Tournament 2004 Bio-Rifle AmmoEffects.

Figure 11.15: Unreal Tournament 2004Bio-Rifle GunEffects.

Figure A.5: Mobile game suspension TFD.

Chapter 12: Cleanroom Testing

Figure 12.1: NFL 2K5 offensive tendencies for players Haiken and Danny.

Figure 12.2: NFL 2K5 defensive tendencies for players Haiken and Danny.

Figure 12.3: Look Sensitivity values with usage percentages.

Figure 12.4: Invert Thumbstick values with usage percentages.

Figure 12.5: Controller Vibration values with usage percentages.

Figure 12.6: Invert Flight Control values with usage percentages.

Figure 12.7: Auto-Center values with usage percentages.

Figure 12.8: HALO Advanced Controls Cleanroom combinatorial table shell.

Figure 12.9: The first Advanced Controls Cleanroom combinatorial test.

Figure 12.10: Two Advanced Controls Cleanroom combinatorial tests.

Figure 12.11: Three Advanced Controls Cleanroom combinatorial tests.

Figure 12.12: Four Advanced Controls Cleanroom combinatorial tests.

Figure 12.13: Five Advanced Controls Cleanroom combinatorial tests.

Figure 12.14: Completed Advanced Controls Cleanroom combinatorial table.

Figure 12.15: Example flow with usage probability.

Figure 12.16: Unlock Item TFD with usage probabilities added.

Figure 12.17: Unlock Item TFD with altered flow 9.

Figure 12.18: Unlock Item TFD with altered flows 6 and 9.

Figure 12.19: Unlock Item TFD without usage probabilities.

Figure 12.20: Casual player usage table for Unlocked Item TFD flows.

Figure 12.21: Inverted usage percentages for the Look Sensitivity parameter.

Figure 12.22: Inverted flow usage table initialized with Enter and Exit flow data.

Figure 12.23: Fixed usage added for flows leaving NoPointsItemLocked .

Figure 12.24: Inverted usage added for flows leaving HavePointsItemLocked .

Figure 12.25: Inverted usage added for flows leaving HavePointsItemUnlocked .

Figure 12.26: Inverted usage added for flows leaving NoPointsHaveItem .

Figure 12.27: Completed table with inverted usage for NoPointsItemUnlocked .

Figure 12.28: Inverted Casual player usage and ranges for Unlock Item TFD.

Figure A.6: Unlock Item TFD with inverted usage values.

Chapter 13: Test Trees

Figure 13.1: Dawn of War two-level test case tree.

Figure 13.2: Skirmish Game Mode test case sub-tree added.

Figure 13.3: Dawn of War technology tree for Eldar Aspect Portal.

Figure 13.4: Blue Mage and Illusionist job trees for Humans in FFTA.

Figure 13.5: Black Pendant card.

Figure 13.6: Top level of Black Pendant test tree.

Figure 13.7: Two-level Black Pendant test tree design.

Figure 13.8: Black Pendant test tree design with causes added.

Figure 13.9: Expanded Deck-Graveyard branch.

Figure 13.10: Trimmed and completed Deck-Graveyard branch.

Figure 13.11: Expanded Hand-Graveyard branch.

Figure 13.12: Trimmed and completed Hand-Graveyard branch.

Figure 13.13: Test tree design portion for Field branch.

Figure 13.14: Host branch added to Black Pendant test tree design.

Figure 13.15: Host-Deck and Host-Hand branches of the Black Pendant test tree design.

Figure 13.16: Host-Graveyard branch of the Black Pendant test tree design.

Figure 13.17: Final Black Pendant test tree design structure, Part 1.

Figure 13.18: Final Black Pendant test tree design structure, Part 2.

Figure A.7: Expanded Multiplayer portion of test tree.

Figure A.8: Summoner job tree.

Figure A.9: Initial "Removed from Play" portion of the Black Pendant "Host" branch.

Chapter 15: Defect Triggers

Figure 15.1: Game software operating regions .

Figure 15.2: Sega Dreamcast block diagram.

Figure 15.3: Nyko multifunction Xbox game controller configuration.

Figure 15.4: Dawn of War online game mode connection exception alert.

Figure 15.5: Advanced Controls combinatorial table with Configuration triggers.

Figure 15.6: Advanced Option game controller configuration table.

Figure 15.7: HALO " game controller removed" attention notice.

Figure 15.8: Advanced game options with Configuration and Exception triggers.

Figure 15.9: Original Advanced Controls table with only Exception triggers added.

Figure 15.10: Ammo TFD template with Stress flows highlighted.

Figure 15.11: NoGunNoAmmo state mitosis.

Figure 15.12: PreMatch state and Startup trigger flows added to Ammo TFD.

Figure 15.13: Ammo TFD with SpectateAndJoin Restart flows highlighted.

Figure 15.14: Ammo TFD with AltFire Exception flows highlighted.

Figure 15.15: Ammo TFD with weapon skin Configuration flows.

Chapter 16: Game Test Automation

Figure 16.1: Should you automate?

Figure 16.2: Bash testing combinatorial table excerpt.

Figure 16.3: TFD excerpt for piƒata bashing test.

Chapter 17: Capture/Playback Testing

Figure 17.1: Control panel with test recording controls.

Figure 17.2: Region1 and Region2 images captured during recording.

Figure 17.3: Dawn of War main menu screen with Army Painter highlighted.

Figure 17.4: Playback options set to Speed = 100 and Terminate on Failure of CompareScreen .

Figure 17.5: Terminate on Failure pop-up window.

Figure 17.6: Initial Minesweeper game field.

Figure 17.7: Custom Field value area to capture.

Figure 17.8: New custom values to be captured.

Figure 17.9: Updated minefield using new custom values.

Figure 17.10: Revisited Custom Field window.

Figure 17.11: Screenshot compare list for failed Minesweeper test.

Appendix A: Answers to Exercises

Figure A.1: Control Chart for review rates.

Figure A.2: ESPN NFL 2K5 game options test table with seven parameters.

Figure A.3: ESPN NFL 2K5 game options test table with 2 min value combinations.

Figure A.4: ESPN NFL 2K5 Create Player pairwise combinatorial table.

Figure A.5: Mobile game suspension TFD.

Figure A.6: Unlock Item TFD with inverted usage values.

Figure A.7: Expanded Multiplayer portion of test tree.

Figure A.8: Summoner job tree.

Figure A.9: Initial "Removed from Play" portion of the Black Pendant "Host" branch.

Appendix C: Combinatorial Table Templates

Figure C.1: Three parameters, two values each.

Figure C.2: Four parameters, two values each.

Figure C.3: Five parameters, two values each.

Figure C.4: Six parameters, two values each.

Figure C.5: Seven parameters, two values each.

Figure C.6: Eight parameters, two values each.

Figure C.7: Nine parameters, two values each.

Figure C.8: Ten parameters, two values each.

Figure C.9: Three parameters, three values each.

Figure C.10: Two parameters with three values, one parameter with two values.

Figure C.11: One parameter with three values, two parameters with two values.

Figure C.12: Four parameters, three values each.

Figure C.13: Three parameters with three values, one parameter with two values.

Figure C.14: Two parameters with three values, two parameters with two values.

Figure C.15: One parameter with three values, three parameters with two values.

Figure C.16: Three parameters with three values, two parameters with two values.

Figure C.17: Two parameters with three values, three parameters with two values.

Figure C.18: One parameter with three values, four parameters with two values.

Figure C.19: Three parameters with three values, three parameters with two values.

Figure C.20: Three parameters with three values, four parameters with two values.

Appendix D: Test Flow Diagram (TFD) Templates

Figure D.1: Powerups TFD template.

Figure D.2: Craft Item TFD template.

Figure D.3: Heal Character TFD template.

Figure D.4: Create/Save TFD template.

Figure D.5: Use Points Buy Unlocked Item TFD template.

Figure D.6: Update Song List TFD template.

Figure D.7: Complete a Mission or Quest TFD template.

Figure D.8: Weapon and Ammo TFD template.