Card Sorting


Card sorting is a technique that's used to uncover how people organize information and how they categorize and relate concepts.

When Card Sorting Should Be Done

Card sorting is best done when you know what kind of information needs to be organized, but before an organizational solution has been implemented. It is usually done after the product purpose, the audience, and the features have been established, but before an information architecture or design has been developed, putting it somewhere in the middle of the design process. This differentiates it from contextual inquiry and task analysis, which both come at the beginning of the development process.

In addition, since it's a fast and easy technique, it can be used whenever you change your information structure or want to add elements to an existing structure.

The Card Sorting Process

Preparation

The goal is to get perspective on how your intended audience understands your proposed information space. So unlike other techniques, there aren't any additional constraints on the kinds of people you should invite, other than recruiting people who fit into your target audience. Typically, recruiting between four and ten people from that audience will give you a good perspective on organizing the information.

Schedule people one at a time so that the participants don't feel pressured to compete. You can schedule several people simultaneously if you have facilities where they can sit quietly and if there is someone nearby who can answer questions (though if you have only one person as a monitor, stagger the schedules about every 15 minutes so that the monitor has time to give each participant an introduction to the technique). An hour is more than sufficient for most card sorting studies.

Getting the Cards Together

The core of the card sorting process is, not surprisingly, the cards. On a deck of sturdy identical note cards, write the names of the things that you want to organize (or use a word processor mail merge and print on mailing labels that you then stick to index cards). These can be the names of specific sections, terms you're considering using, concepts behind the various sections of your site, images that you want to use, or even descriptions of individual pages. Use identical cards, except for the text, to minimize distraction.

You can have as few or as many cards as you want, though the size of a standard card deck (52) strikes a good balance between not providing enough cards to make adequate categories and providing so many that it's overwhelming. If you have hundreds of categories that you would like to try this technique on, consider breaking them up into more manageable chunks and doing multiple sets of tests.

The words on the cards should reflect what you're trying to test. If you're trying to uncover how people organize concepts, explain the concepts on the cards with a sentence or two. However, if you're trying to see how people understand a set of titles without necessarily knowing your definitions for them, you can just write the titles on the cards.

The Sort!

After bringing the participant in and going through all the initial formalities, introduce him or her to the concept. Say something along the lines of this.

start sidebar

This is a stack of cards. Each card has something that you might see on the Web site. I'd like you to organize them into groups that make sense to you. Take as much time as you need. There are no right or wrong groupings. Try to organize all the cards, but not everything needs to belong in a group. You won't have to provide a reason why cards belong in the same group, so if a group feels right, go with it. Focus on what makes sense to you, not what may make sense to anyone else.

end sidebar

Provide a stack of Post-it notes, several pens, and a pile of small binder clips or rubber bands. After they're done grouping, ask them to label the groups if they can, but not every group necessarily needs a label (don't tell them that they'll be labeling ahead of time since that tends to bias people to organize based on labels rather than on what they feel are natural groupings).When they're done, ask them to clip or rubber-band the cards and place the label on the groupings.

Finally, ask them to organize the groupings into larger groups without moving any cards from one group to another, naming the larger groups, if any names come to mind. Then, with larger binder clips or rubber bands, bind the metagroups together and wrap up the session.

Card Sort Analysis

There are two ways to analyze the output from card sorting, a formal way and an informal way.

The Informal Way

When you have the clusters from all the participants, look at them. Copy the clusters to a whiteboard. By eyeballing the trends in the clusters, you can infer how people intuitively understand the relationships between the various elements. For example, if people put "News," "About us," and "What we like" together, it tells you they're interested in putting all the information coming from your perspective into a single place. However, if they group "News" with "Latest Deals" and "Holiday Gift Guide," then maybe they associate all the information that's timely together. The difference between these two can mean the difference between an information architecture that matches its users' expectations and one that forces them to hunt for information.

You can look at the clusters as a whole or follow one card at a time through the participants' groups to see the kinds of groupings it's put into. Don't treat the clusters literally. People's existing organizations may not make a scalable or functional architecture. Instead, look at them for the underlying themes that tie them together. Pay attention to the cards that people didn't categorize or that were categorized differently by everyone. What about the card is giving people trouble? Is it the name? Is it the underlying concept? Is it the relationship to other elements?

When you've gone through all the clusters produced by all the participants and listed all the themes, go through the names they've assigned to the groups. Looking at the labeling and the labels' relationships to the clusters underneath, you should have the basis for creating an architecture that's close to how your user base expects the information to be organized (and even if it's not used by the information architect, the terminology can be useful to marketing when they're explaining the product to potential clients).

The Formal Way

Cluster analysis is a branch of statistics that measures the "distance" between items in a multivariate environment and attempts to find groupings that are close together in the variable space. This is exactly what a whiteboard-based card sorting analysis does, but it's a mathematical way to do it thoroughly and consistently. It allows you to uncover groups of objects that are similar across many dimensions, but may not be obviously alike in any one of those dimensions. Since people have trouble visualizing things in more than three dimensions, and there are often more than three variables that can determine similarity, the technique is used to "see" clusters that would have otherwise gone undiscovered.

The classic example of cluster analysis is from marketing research, and it's the grouping of target markets. People are different in many ways: their age, their income, their race, where they live, their gender, what they buy, and so on. Picking out groups along any one of these axes is relatively straightforward, but picking out clusters along all of them is much more difficult. How do you define a yuppie? Do they all drive Land Rovers? No, some drive BMWs, and some people who drive Land Rovers aren't yuppies. Do they all make $120K per year? No, some may make $30K and live as if they made $120K. Figuring out the clusters in society is hard, but cluster analysis can often extract useful distinctions.

In terms of card sorting, it works in reverse. It's used to find the underlying variables by looking at the clusters people make. Are certain things grouped together more often than other things? Are there hidden relationships between certain cards? These are all things that are hard to see by just looking at the cards.

Unfortunately, the mathematics of cluster analysis is nontrivial and can't be easily done without a computer. Statistical packages such as SAS and Statistica contain modules that can do cluster analysis, but these are expensive and require an understanding of the statistical procedures used in the analysis.

Fortunately, there is a piece of software that is designed specifically to do cluster analysis for card sorting data and is, as of summer 2002, free. IBM's User Involvement Group made EZSort available to the usability research community in 1999, and it's made the process of analyzing card sorting much easier. The program takes as input the groups created by the participants and produces tree graphs that show the relationship between groups of cards, revealing clusters of cards and clusters of clusters. These diagrams make it much easier to separate strong, common affinities from casual similarities and to see larger themes that would have been difficult to see through a haze of cards and groupings.

As of fall 2002, it's available from

http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410

The process of using it is straightforward: the names of all the cards and participants are entered into the program, then each person's clusters are re-created in the software by dragging the card names into piles; the names for the clusters are associated with these virtual piles, and the software is run. It produces a diagram that represents the relationship between all the items using a tree diagram (Figure 8.3). The more distant one "branch" is from another, the less related they are.

click to expand
Figure 8.3: EZSort diagram for a typical shopping site.

Note

For more information on EZSort, see the documentation or the paper by Jianming Dong, Shirley Martin, and Paul Waldo that can also be found at http://www-3.ibm.com/ibm/easy/eou_ext.nsf/Publish/410/$File/EZSortPaper.pdf

This diagram shows that information about the company strongly clusters (the tree at the very top of the diagram), which likely means that people expect all that information to be in the same place (or at least treated in the same way). Likewise, general product information such as recommendation, reviews, and "free stuff" are all considered similar in some way, with "free stuff" being a bit less similar to the others in the same group. The other elements also form smaller clusters. The two vertical lines represent "threshold" for membership—how close items must be in order to be grouped together—with the number below each one representing the calculated affinity that the line represents.

start sidebar
Card Sorting to Prioritize

Although card sorting is primarily an organization or naming technique, variations of it can also be used for other purposes. Jesse James Garrett, an information architect and one of my business partners in Adaptive Path, and I developed a technique that uses card sorting to understand how people prioritize features.

A group of 12 participants were asked to organize 50 cards that had descriptions of current and potential features for presenting current events on Web sites (this was done when researching how people read news online, as a break during a long one-on-one interview). Each card had a single sentence that described a way that information could be presented or organized on a Web site. The participants were first asked to place the cards into one of four piles describing how valuable they felt the feature would be to them, as they used the Web right now. The four piles were titled "Most Valuable," "Somewhat Valuable," "Least Valuable," and "Not Valuable." As participants placed the cards in these piles, they were asked to narrate their thoughts about the topic. This was done to understand why they made certain choices and how they thought about the features.

After the participants completed organizing the cards, they were asked to repeat the exercise, but this time only using the cards that they had put into the "Most Valuable" pile and with a different set of criteria for the placement. The new task was for them to organize the features in terms of how frequently they felt they would use each of the features. This was done in order to differentiate between the features that attracted people to a site and those that they felt were the most immediately useful.

To understand where the participants' values lay, each one of the categories was given a numerical rating indicating the strength of preference, from 0 to 5:

0 - Not valuable

1 - Least valuable

2 - Somewhat valuable

3 - Most valuable, rarely used

4 - Most valuable, sometimes used

5 - Most valuable, used often

The participants' choices were then rated, and the median value of their ratings was calculated. Since a number of choices had the same median, it was necessary to further organize the list. The standard deviation of the choices was calculated and represented the degree of agreement among all the ratings for a given category, with lower deviations representing greater agreement. Both median and standard deviation are defined in Chapter 11. The list was thus ordered first by preference and then by agreement, which gave the development team a much clearer idea of their customers' values and in turn helped them prioritize their own development efforts.

end sidebar

The card sorting process sheds light on people's existing understanding and preference, and it can show subtle relationships that may not be obvious by just examining a list of clusters. It also provides an idea of how concepts relate to each other since what may seem like a strong relationship when casually examined may turn out to be weaker when actually analyzed.

Contextual inquiry, task analysis, and card sorting can take you a long way toward revealing how your target audience understands their environment and what their needs are. As prescriptive techniques, they can focus a project early on, eliminating many wrong turns and reducing the need to ask major, fundamental questions about the product later on, when development should concentrate on honing the product's experience, not its purpose.




Observing the User Experience. A Practioner's Guide for User Research
Real-World .NET Applications
ISBN: 1558609237
EAN: 2147483647
Year: 2002
Pages: 144

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net