what you can learn from usability tests

The great advantage of usability testing is that it lets you see your site through someone else's eyes. And this helps you catch mistakes and pinpoint problems you can't see when you've designed the site.

Usability testing is "like a form of sensitivity training," says Michael Twidale, professor of computer science at the University of Illinois Urbana-Champaign. "It helps you figure out, 'What's this going to feel like for people who aren't me?'"

"The art of good usability design is that you can put on these spectacles," he says. "You can look at your design through someone else's eyes. Now that's very hard to do. But you can practice if you watch other people using the system."

And that's where usability testing comes in. "Sitting in on user testing sessions has changed my view of design," says designer Doug Bowman. "It helps you with the design that you're testing, but it also gives you insight that you file away in your head. So when you're designing a site, you just automatically think about what you've seen in past user tests, and you know instinctively how a user behaves when they arrive on a page."

But no matter how many times you've designed and tested sites, you still need to test each interface. Because the longer you work on a site, the harder it is to distance yourself from it. And no matter how well-versed you are in user behavior, you'll always be surprised by what you learn.

"You do find surprises," agrees Greg Dotson, Chief Information Officer for Guru. "Usability studies are really good at finding simple things you've overlooked. And they're also good at confirming things you've done well."

"I can't imagine a scenario in design where usability testing is not helpful, period," says designer Doug Bowman. "Your perception of how someone's going to use your site is entirely different from what happens when a user actually tries to use it."

Sometimes, your findings will seem obvious: As Greg Dotson says, "After you conduct a test, you think, 'How could I have been so stupid?'" But sometimes, they're completely unpredictable.

"That's what's interesting," says Steve Mulder, user experience manager for Lycos. "Sometimes you just don't know why something doesn't work."

In any case, the most important takeaways from usability testing are precisely the issues you couldn't see on your own because you're too close to the work.

"User testing reveals problems that people who know the brand and the product and the Internet can't see," said Lance McDaniel, VP of Creative for SBI and Company. "And when you get those big surprises that, to me, is a Godsend, because what you've done is avoid launching a site to ten million people with a problem that was obvious to the eight people who tested it."

What you can learn from usability tests:

• Problems with site organization. Sometimes a site will organize its content differently than the user might expect. This may slow the transaction down or even prevent the visitor from accomplishing what they set out to do. (See site organization, p. 98).

• Problems with names or labels. Often, users get confused by the names used for site sections and links. The features themselves may be perfectly usable, but if users can't understand the labels, they won't be able to find them much less use them. (See naming site sections, p. 122).

• Problems with placement. Often, users will intuitively look for a button or link in one part of the screen (the "shopping cart" link in the upper-right corner, for example). If a user hesitates and scrolls around the page, he's probably having trouble finding what he wants. (See how people see web sites p. 86.)

• Problems with grouping. Users will intuitively expect related items to be grouped together. They may get confused if seemingly unrelated items are grouped together or if an item they view as related is set apart. (See navigation systems, p. 114.)

• Problems with pacing or the order of events. When users are completing a transaction, they have expectations about what should happen next. Some are negotiable, and some aren't. For instance, they usually expect to enter a credit card number last in a shopping transaction. If you add an additional step after the credit card is charged, they may never even notice it.

So, by putting your site in front of users early in the design process, you can validate your assumptions and catch significant problems early enough to correct them. Some of the problems such as misleading labels are easy to change. Others, however, may be more involved. Usability testing sometimes turns up problems that are more than skin deep. You may discover problems with your site's functionality or organization that force you to rethink your entire approach.

and what you can't learn...

Although usability testing is powerful, it's not a panacea. It can't solve all your site's problems or prevent all its pitfalls. And it can't substitute for the earlier rounds of user research.

"You have to make time to understand customers, and usability testing is only part of understanding customers," explains Mike Kuniavsky, author of Observing the User Experience: A Practitioner's Guide for User Research.

"By the time you're doing usability testing, you should know with a very, very high degree of confidence that you have the right audience, that you know what problems they're having, and that you have a solution they will want and they will use," he said. "The only thing usability testing does is check whether the solution you've created is actually usable."

And just because a feature is usable, it doesn't mean it will be used. "The discount bins in record stores are littered with records that everyone can use, but nobody wants."

This last point can be hard to get across, because many proponents of usability testing confuse user ability with user interest.

"It's amazing how many bad decisions have been made on web sites thanks to usability tests," says Adam Berliant of Microsoft. "Why? Because usability has nothing do with whether or not people actually use something. There's a big difference between the question, 'Can you operate this coffee maker?' and 'Will you operate this coffee maker?'"

"So when I'm prioritizing my research, my first first question is 'Will people use it?' If the answer is 'yes,' then I want it to be usable."

What you can't learn from usability tests:

• Whether the features will be used. In usability tests, you're guiding volunteers through a task that requires them to use your site features. From this, you'll learn whether the features can be used, but you can't conclude anything about whether they will be used.

• Whether you've included the right features. Usability testing will occasionally turn up clues about features you should have included (if a user is unable to complete a task, for instance). But it can't tell you whether you've chosen the right ones. "Usability testing is very good for seeing if someone is able to complete a task," said Jeffrey Veen. "And that's about it, really."

Buy this book!

Don't Make Me Think!

by Steve Krug

A wise and witty manual on user-centered design and usability testing. Also one of the best books written about the web.