How to Test

How you have your playtesters work on your game is as important as who you have testing and when you have them do it. Game designers will often ruin the effectiveness of their playtesters by making a number of fundamental errors in how they interact with them. These are all problems that can be easily avoided, as long as the designer is conscious of the way he deals with his testers and what he does and does not tell them.

The most important part of interacting with playtesters is to actually spend most of your time watching them play instead of telling them how to play. Let them play the game their own way and see how they fare. The temptation to correct playtesters actions is great and can be hard to resist. By the time the traditional playtesters start on the game, the designer has already played the game so much that he is intimately familiar with what the players are supposed to do in a given situation and how the game is supposed to be played in general. When watching over the shoulder of a playtester for the first time, the temptation is to say, Go over there next , or You want to use the strafe buttons for that, or Why don t you try to get the power-foozle? Watching someone stumble while playing a game the designer is intimately familiar with can quickly turn him into a teacher.

But the point of the playtesting is to see how players will actually play the game without the game s designer coaching his every move. Certainly, the designer cannot fit in the box the game comes in or even be downloaded over the Internet. A certain amount of stumbling about and learning the controls is to be expected, and the best way to playtest is to let the testers do this initial exploration on their own. And if the players truly do get stuck or if they never seem to be able to master the controls, the designer needs to ask himself what is causing these problems. Is the game too hard or too confusing? How can it be made simpler so that players have a fair chance of understanding it and learning how to play? These are the lessons a designer is supposed to take away from playtesting, but they are lessons the designer is never going to learn if he corrects the tester s playing at every step. Beyond that, when I sit in on a playtesting session, if the testers do not know any better, I try to pretend I am not on the development team to avoid having them color their opinions to be nice to me. Indeed, most gameplay focus tests are conducted using a one-way mirror, since people will behave differently when they think they are alone than if they have someone standing over their shoulders taking notes, regardless of how quiet that person is or who he claims to be.

While watching the testers play, the designer should try to observe the way in which they try to play the game. Players may not try the approach or solution the designer had thought of to a particular situation. The designer must then ask, does the game support what the tester is trying to do, and if not, could it and should it? The testing period, if started early enough, is a time when the designer can add a breadth of content to the game that will allow the game to truly be accepting of multiple playing styles. Up until this point, the people playing the game have been limited to the development team and the preliminary testers that may have been brought in. Now that there is a broader range of people playing the game, the designer will likely observe a broader range of playing styles than he had anticipated. The testing period is when the designer can make the game accepting of these playing styles, allowing players to truly play the game their own way on their own terms.

Of course, the designer cannot be present for all of the playtesting the game will undergo, not if the game is going to be thoroughly tested and released in a reasonable time frame. Often you will need to rely on what the testers report to you about their playing experiences. Though not as useful as watching the testers play firsthand, this information can nonetheless be quite helpful. When you do get this feedback, it is crucial to truly listen to what the testers tell you. This may seem obvious, but it is surprising how many designers prefer to ignore the feedback they get on their game. Often most of a game s testing, particularly that done by traditional testers, takes place late in the development process, after a good deal of work has gone into the project. At this point the designer is probably fairly confident that the game is working as he wants it to work. Therefore, it can be difficult for the designer to hear testers contradict this, perhaps pointing out fundamental problems in the game that the designer has overlooked for months of development.

The designer s first defense is often to claim that the testers do not know what they are talking about. Excuses can range from the tester being a fool to the tester not being the target audience for the game to the tester just complaining for the sake of it. Granted, often testers do make suggestions for changes to the gameplay that are best avoided, and if only one tester out of ten suggests that a certain piece of gameplay needs to be changed it may be because of that tester s personal preference. But when the designer hears the same complaint from a number of different testers, he needs to realize that there probably is something wrong with the game that needs to be addressed. The testers may not even be complaining about the right aspect of the game or suggesting an appropriate fix, but the fact that something is ruining their experience warrants investigation. The designer must avoid dismissing the complaints of testers and honestly look at each complaint to see if it has any merit. It is amazing the number of designers who will resist any and all suggestions the testers make. Often, these same designers come to regret their obstinacy later when the game is finally released, only to have players and members of the press complain about the same issues the testers had complained about earlier. Of course, once the game is released, it is too late to do anything radical about the problems.