Selecting Users

User testing is testing with real users. So, who is a real user? Clearly, you are not a real user, nor is anyone else on your development team. The ideal real user is a target user, the specific type of user that the user interface was designed for. A target user has specific needs and goals as well as a specific background. I discussed target users in detail in Chapter 8.

Usability testing typically focuses on first-time users performing a task once. This makes sense because testing for learnability is such an important goal, as I discussed earlier. There are usually more intermediate and advanced users than beginning users for a typical program in the real world, and thus most user testing misses a significant portion of the user base. However, beginning users are more likely to reveal problems.

TIP

---

Testing with beginning users is more likely to reveal problems.

It can be difficult to find suitable test users. For example, beginning users don't stay beginners for long, so you can't use an individual test user as a representative beginning user for more than one set of tests. It is even more difficult to test advanced users, since it is unlikely that a test user is going to use the program long enough to be a representative advanced user. If the program is new, you'll have to create advanced users yourself, since they don't exist yet, by giving some users sufficient exposure to the program. The difficulty in finding suitable test users is yet another reason why you need to perform programmer testing and QA testing before user testing. You don't want to "waste" users on easy-to-find usability problems.

What about user testing with colleagues that are not on your development team? Most usability experts recommend against this, since colleagues have different backgrounds, biases, expectations, and knowledge about the program than real users. However, target users are usually not readily available and colleagues are. I've tested with colleagues and found that it works quite well. I find application engineers make good test subjects. In keeping with the idea that some user testing is better than none, I think that testing with colleagues is far better than no user testing at all. After all, testing with colleagues is a waste of time only if they can't find any problems. If they are confused by a feature or can't perform a task, their input is probably just as good as that of target users.

On the other hand, some types of target users have no substitutes, such as:

• Noncomputer users
• Children
• Physically impaired users
• Highly trained specialists

For these types of users, everything you know is wrong. You must test with these specific target users to obtain useful information. If you're writing software for brain surgery, you had better test with brain surgeons.

How many users should you test with? For a design check of a typical mainstream program, testing with three beginning-level users is about the minimum. If you want to also test experienced users, you should test with at least three experienced users as well. Of course, the more complex your program is and the larger the market, the more users you should test with. Nielsen has found that you can get good coverage with as few as three test users per iteration. For a product check, you might want to use more than three test users to get broader coverage.