Classifying Defects

You don't have to wait for your next project to start using Defect Triggers in your tests. Use keywords to help classify new or existing tests and defects. With that information you can identify where the defects are coming from and what testing is missing. Not surprisingly, a lot of bugs that get released belong to Defect Triggers that received little attention, if any, during game testing. When defects of a certain trigger start to show up, that's your cue to beef up the tests to use that trigger even more. Table 15.1 provides lists of keywords you can use to classify your defects and tests according to each of the six defect triggers.

 

Table 15.1: Defect Trigger Keywords

Trigger	Keywords
Configuration	configure, model, type, version, environment, connect, disconnect, add, remove
Startup	startup, initial, first, un- initialized , creation, boot, warm up, wakeup , loading, setup, entry, transition, delay
Exception	exception, error, violation, exceeds, NULL, unexpected, recover, prevented, blocked, prohibited , unavailable
Stress	stress, load, rate, slowest, fastest , low, high, speed, capacity, limit, long, short, few, many, empty, full
Normal	normal, typical, common, usual, expected, planned, basic, allowed, available
Restart	restart, reset, cleanup, eject, power down, ctrl-alt-del, quit

Here are some examples taken from the Star Trek Elite Force II version 1.10 patch readme file. Remember, missing capabilities are defects as well as functionality that is in the game but doesn't work properly.

"Mouse 4 and 5 are now key-bindable in the Configure menu"

This is an easy one to start off with. The bug description mentions that the problem is with the Configure menu, so classify this as a Configuration defect.

"The sniper rifle now reloads automatically in all game modes. It no longer waits for the player to let go of the fire button."

In this situation, holding down the fire button prevented the reloading of the rifle. This should be considered an undesirable behavior of the weapon Restart process.

"Saved game issues with long computer names fixed."

The Stress keyword "long" is used here and it is in reference to a problem that occurs when trying to perform a function (saving the game) with the long computer name , so this is a Stress defect.

"The Ultridium Explosive damage has been increased."

Here is a function that wasn't quite operating the way it was intended. The damage increase is not limited to a particular configuration or condition and affects something that happens during gameplay (the In-Game operating region). This is simply a Normal defect.

"Death will no longer break chat in MultiMatch."

In this context, "death" is part of a player's life cycle: Spawn-Fight-Die-Spawn and so on. Death's place in this life cycle is to initiate a "restart" of the player's character. This restart breaks the chat function. The defect is a Restart defect.

"The visual graphic indicating that you have the flag in MultiMatch Capture the Flag matches has been enhanced."

Problems don't just have to happen in game logic to be considered bugs. Here one of the graphics effects was improved. The solution is not related to a particular condition or configuration of the game. This is a Normal defect.

" Teams are no longer automixed on map restart in MultiMatch."

Game maps can also have a life cycle: Start Map-Use Map-Change/Restart Map and so on. An undesirable function ‚ the automixing of teams ‚ takes place as a result of the map restart. Therefore, this defect is triggered by that Restart.

"When playing a Disintegration match in MultiMatch, the reload delay for the sniper rifle is now shorter."

Yet another "life cycle" reveals itself here. The cycle of shooting a weapon: a round of ammunition is ready to be fired , shoot weapon, wait for shooting effects to occur (shooting sound, projectile launched, cartridge ejected, ammo count decreased, and so on). Then the next round is ready, and so on. The problem is noticed ("triggered") when the player tries to shoot the weapon but the gun is spending too much time in the "ammunition is ready to be fired" part of the sequence. Think of the "loading" keyword and map this to the Startup trigger.

"The server will now inform the player if they vote to switch to a map that doesn't exist on the server."

This is a case where the player is referencing an "unavailable" resource by making a request that the game can't comply with. Apparently it was permitted prior to the patch without notifying the player that she was throwing away her vote. This is an Exception trigger defect.

"The point of view for demo recording will now follow the direction in which the player is looking."

Don't confuse a game "mode" with "configuration." Think of a mode as another feature or function of the game. Even though this bug only had to be fixed for demo recording, the problem was in the In-Game operating region and not dependent on any configuration or condition. It is another Normal trigger defect.

Sometimes you will come across defects that may seem to belong to more than one trigger category. An example might be the case where an exception is not handled properly during startup. What you must resolve is which situation was mostly responsible for triggering the defect. If the situation is only considered an "exception" during startup, then it is the exception that is triggering the fault. The rationale is that there is a particular piece of code that should exist to handle the exception during startup. The exception condition causes that code to execute and finds that it is missing or faulty. On the other hand, if the handling of that exception is common throughout the game and it only fails to operate properly during startup, then it is the fact that you tried it at startup that triggered the exception code not to run or to run improperly. Your responsibility as a tester is to test the handling of this exception in all operating regions of the game to help make this kind of determination.