Chapter 1. Introduction: Nothing to Hide
4 Where We Got Our Data
17 Tell Me Again: Why Do I Need to Do
We'll begin by letting you in on a secret: There are no secrets to doing good usability studies. It's simply a matter of knowing how and where to look, and then documenting your observations.
Usability works because it reveals how the world works. Once you discover how people interact with your design, you can make it better than your competitor's.
As much as we try to teach companies to do their own user research and to
We are writing for sites that are not among the Web's Top Ten most-visited list. It's actually good news if you're not on the Top Ten most-visited list.
Fair enough. That's why we wrote this bookto bring the best of our
Where We Got Our Data
All of our findings and guidelines are based on empirical evidence from two sources. First we rely on our testing of 716 Web sites with 2,163 users around the world. Most of this research was
Other studies were conducted in the process of writing research
Our second source of information was a special study that we conducted for this book. When we talk about "the study" here, we're referring to this smaller set of data.
How We Did the Book Study
We tested 69 users, 57 in the United States and 12 in the United Kingdom, for this book. Slightly less than half (32) were male and slightly more than half (37) were
The users had a broad range of job backgrounds and Web experience. We screened out anybody working in technology, marketing, Web design, or usability because they rarely represent mainstream users. People who work in the biz know too much and have difficulty engaging with a design as regular users. Instead, they tend to criticize the design based on their personal design philosophy, which is invalid as usage data. In fact, whenever you hear a user throw around insider terminology like "information architecture" in testing, you probably have to discard most of what they say.
All the users had at least one year's experience using the Web. We almost never test people who are completely new to the Web because all we would find is that the Web in general and browsers in particular are difficult user interfaces that take some time to learn. We wouldn't learn much that would help us design better Web sites because completely new users wouldn't get very far into the sites.
The experience of
The standard rule for user testing is to
Sites Tested: Large Companies
Government and Nonprofits
For part one of the book study, we systematically tested 25 Web sites that cover a range of genresfrom industries like automobiles and financial services to entertainment sites and intellectually oriented medical and cultural sites.
You can see from their homepages that the sites also exhibited a wide variety of design styles, from somewhat primitive to overly glamorous. All in all, our basic goal was to test a good cross-selection of current Web sites.
We didn't pick any site because we wanted to dig into that company's Web strategy. Similarly, our comments on these sites should not be construed as criticism of the companies or
Homepages for the 25 sites that were tested systematically in the usability study we conducted for this book. The homepages are shown as they would initially appear in a browser window on a 1024-by-768 monitor.
It should be noted that the book study was not
This part of the study was a scavenger hunt of sorts. We gave each user three or four specific tasks to do on each site. While these
Some of the tasks we asked users to perform were:
Go to www.usps.com and find out how much it costs to send a postcard to China.
Go to www.sandiego.gov and find the
You are planning a family reunion at Sugarloaf Ridge, California. Go to www.parks.ca.gov and make a reservation for a campsite that can hold 35 people.
You are looking for a
Go to www.pixar.com and see if you can find out how they came up with the idea for the movie Monsters, Inc.
You want to visit the J. Paul Getty Museum this
You want to put $1,000 in the bank and keep it there for a long time. Go to www.bankone.com and find the accounts with the best interest rates.
You read an article about how fuel cell technology may change the world. Go to www.cummins.com and find the top two pros and cons of fuel
All of these tasks were eminently
In the site-specific testing, users were told where to go and were expected to stay there while performing their tasks. This is the way most usability studies are conducted, and is great if you want to find out how elements of a particular site's design works. Of course, that's not the way users work in real life. People have the entire Internet at their fingertips and they'll often jump from one Web site to another to complete a task.
In most usability studies, people are told what site to use, but that's not the way users work in real life. So we also gave ours a range of tasks and told them to go
For this reason, in part two of our testing we gave users a range of tasks and told them to go anywhere they pleased. We call these "Web-wide tasks" because users had the entire Web to choose from. These tasks represent a wide range of activities, from highly commercial pursuits to
The main downside of this approach is that users go to different sites even when working on the same problem, so we didn't get to systematically test those sites with a range of users. The upside is that we get to see how people construct their solutions across multiple sites, as they do when they are not in the lab.
For this section of the testing, we gave each user one of 12 tasks:
You and your family are interested in going on vacation to Mazatlan, Mexico. Find a travel offer that is both appealing and affordable for your family.
You have some extra time during the week and want to do something to help the community. Find a community service program that would be a good fit for you.
Uncle George is interested in getting a personal computer for his home. He's
Your nephew's birthday is coming up. He's going to be five
You're thinking about investing $10,000 in a retirement plan. Find the best way to invest your money.
You're planning to purchase a new home and need to finance it. Find a company that will give you the best service and rates.
There's been a lot of news lately about viruses on the Internet. Find the best way to protect your personal computer from viruses.
You're doing a report on Marie Curie. Find out who she is and some of her most famous accomplishments.
What are the three main reasons that Siberian
Find out what a "let" is in tennis.
In August 2003, the United States and Canada
A good friend complains about throbbing pain that usually radiates from one eye to the forehead, temple, and cheek. What might your friend be suffering from?
We include many screen shots in this book because it's much easier to
In either case, this means that if you check out a site after seeing it in this book, it is very likely to look different. Does this make the analysis in this book less relevant to your project? Not at all. The principles and guidelines that a screen shot illustrates are relevant long after a site has changed. In fact, many findings from our user testing in 1994 continue to be seen in studies in 2006 and will probably be found again by unlucky testers in 2020 and beyond.
An example of this is the Web site of the British Broadcasting Corporation (BBC). Jakob and Marie Tahir
The day before this screen shot was taken, the BBC site featured a great recipe for roast goose with sage and onion
As the second screen shot shows, the BBC was working on an archive in late 2005. If you visit the site now, the homepage may well feature the archive. If so, the BBC will finally have done what we suggested in 2001. Does that change the importance of the comments in Homepage Usability? It doesn't, because our guideline to provide an archive of homepage feature stories continues to be relevant for millions of other Web sites. Only 41 percent of corporate homepages follow this guideline, so 59 percent could improve their usability today if they pay attention to the mistake the BBC made in 2001.
Beta release of BBC's service to archive all stories that have run on the homepage so that users can find them later: Let's say that you remember seeing a mouthwatering photo of Rick Stein's Christmas goose when you visited the site