Schedules


Once you have the questions you want to answer, you're close, but it's not your research plan yet. Before it can be a research plan, it needs a schedule, and before it can have a schedule, you need to integrate it with the existing development system.

If you're lucky, the development process is well defined, with good schedules and specific deliverables. If you're not, then your research plan may become the development schedule (if you can convince the development team that it's worth their while to let it do so).

Often talking to the person immediately in charge of the product is the fastest way to determine the actual development process. This could be a project manager, a senior production person, or the marketing manager. Show him or her your list of prioritized questions, and rearrange the priorities to take advantage of the development schedule, moving research that's going to affect near-term development efforts up in the list. Concentrate on the big issues, but feel free to include secondary issues as techniques and schedules allow.

See Chapter 18 for more ways of integrating user experience research into your corporate culture.

The output of every research project not only provides information on how to improve the product, but it feeds into the next research project. Surveys describe the current audience so that you know whom to recruit. Contextual inquiry outlines what their problems are. Focus groups tell you which problems people feel strongest about, and so on. Every piece of information that research provides helps you know who your users are or what they want. You use that information, in turn, to focus subsequent research.

Thus the order in which you schedule the projects, although constrained by immediate priorities and the development schedule, should still be organized so that one project informs and reinforces subsequent projects. In practice, this means that procedures that gather general information fall before those that collect information about specific preferences and behaviors.

Starting Research in the Beginning

For an existing product in an early phase of redesign, the following order makes sense (see Chapter 4 for other thoughts about integrating research into development processes):

Early Design and Requirement Gathering

  • Internal discovery to identify the business requirements and constraints. (Internal discovery is described more in Chapter 18.)

  • A survey to determine demographic segmentation and Web use of the existing user base. (Surveys are described in Chapter 11.)

  • Log file analysis of their current behavior if such logs exist. (Log analysis is described in Chapter 13.)

  • Profiling of the ideal users, based on knowledge of the existing users or users of competitive products. (Profiles are described in Chapter 7.)

  • Usability testing of the existing product to uncover current interaction problems. (Usability testing is described in Chapter 10.)

  • Contextual inquiry to uncover problems users have (both with the product and with the task it's supposed to aid), and task analysis to specify how they solve them right now. (Both of these techniques are described in Chapter 8.)

  • Two to three rounds of focus groups to determine whether people feel the proposed solutions will actually help them and which features are of highest value. And competitive focus groups to determine what users of the competition's products find most valuable and where those products fail them.

Realistically, only a couple of these techniques are generally possible to do given release schedules, but this would be an ideal list to prepare for a major overhaul of the product, assuming there were four to six months of time devoted to this stage in the process.

Development and Design

  • Four to five back-to-back usability tests of prototypes that implement the solutions and test their efficacy. And competitive usability tests to determine strong and weak points in the competitors' products.

After Release

  • Surveys and log file analysis to compare changes in the product to past behavior.

  • Diaries to track long-term behavior changes.

  • Contextual inquiry to study how people are actually using it.

Starting in the Middle

A user research plan often may have to begin in the middle of a development cycle. Decisions have already been made about who the users are, what their problems are, and what solutions to use. These cannot be revised—at least until the next development cycle. In such cases, the research plan should begin with procedures that will give immediate benefit to the product before release and plan for more fundamental research to take place during the next development cycle. The following order may make sense for such a project:

Design and Development

  • Rapid iterated usability testing and competitive usability testing until release. (Competitive research techniques are described in Chapter 14.)

After Release

  • Log file analysis before and after release in order to know how customers' behavior changed.

  • A survey to determine the makeup of the user population.

Requirement Gathering

  • Contextual inquiry with the existing population to determine outstanding issues for the next release.

After this, the plan can continue as the first plan.

Organize Research Questions into Projects

Let's go back to the example. In the list of research questions, it appears that there are some quantitative questions about people's current behavior (the ratio of abandonees to those who complete a transaction, the page where the abandonment happens, etc.), and there are questions about their motivation and understanding of the site. The first set of questions can be answered through log analysis, and the second set can largely be accomplished through usability testing. There's one question, "How do they shop on the site?" that's probably too abstract to be easily answered through usability testing. It's one of those fundamental questions that will probably need to be researched over an extended period of time and with several different techniques. It can begin with a round of contextual inquiry.

start sidebar
CHOOSING AMONG THE TECHNIQUES

Picking the right techniques and grouping them can be difficult. The more experience you have with the methods, the better you will know which techniques best address which questions. If you don't have any experience with any of these techniques, start with the descriptions in this book, and pick one that seems right. Try it out. If it doesn't help you answer your question, note what kinds of information it was able to gather well and try a different technique.

Here is a table of the techniques with some trade-offs. It provides a basic overview of the techniques, but it's certainly not comprehensive.

Name

Stage in Development

Duration

Cycle Time

Profiles Chapter 7

Beginning of development process

Two to five days' work over two weeks

Once per major design, or when new user markets are defined

Description: Developers turn audience descriptions into fictional characters in order to understand how audience needs relate.

Benefits: Low-cost method that creates good communication tools for the product team. Focuses on needs of specific audiences rather than "The User."

Pitfalls: Based primarily on team's understanding of users, not external research.

Contextual Inquiry and Task Analysis Chapter 8

Initial problem definition

Two to four weeks, not including recruiting

Once per major set of features

Description: Observe people as they solve problems to create a mental model that defines users' current understanding and behavior.

Benefits: Creates a comprehensive understanding of the problem that's being addressed.

Pitfalls: Labor intensive.

Focus Groups Chapter 9

Early development feature definition

Two to four weeks, not including recruiting

Once per major set specification, then after every feature cluster

Description: Structured group interviews of 6–12 target audience representatives.

Benefits: Uncovers people's priorities and desires, collects anecdotes, and investigates group reactions to ideas.

Pitfalls: Subject to group-think among participants; desires can be easily misinterpreted as needs.

Usability Testing Chapter 10

Throughout design and development

One to two weeks, not including recruiting

Frequently

Description: Structured one-on-one interviews with users as they try specific tasks with product prototypes.

Benefits: Low-cost technique that uncovers interaction problems.

Pitfalls: Doesn't address underlying needs, just abilities to perform actions.

Surveys Chapter 11

Beginning of development, after launch and before redesign

Two to six weeks

Once before major redesign, regularly thereafter

Description: Randomly selected representatives of the audience are asked to fill out questionnaires; quantitative summaries of the responses are then tabulated.

Benefits: Quantitatively describes the audience, segments them into subpopulations, investigates their perceptions and priorities.

Pitfalls: Doesn't address the reasons why people have the perceptions they hold or what their actual needs are. Subject to selection bias.

Ongoing Research Chapter 12

Throughout life of product

Ongoing

Regularly after release

Description: Long-term studies of users; done through diaries and advisory boards.

Benefits: Investigates how users' views and use patterns change with time and experience.

Pitfalls: Labor intensive. Requires long-term participation.

Usage Logs and Customer Support Chapter 13

Beginning of development, after launch and before redesign

Varies

Regularly after release

Description: Quantitatively analyze Web server log files and customer support comments.

Benefits: Doesn't require additional data gathering. Reveals actual behavior and perceived problems.

Pitfalls: Doesn't provide any information about reasons for behavior or problems.

end sidebar

Based on what you know about the company priorities, group the questions into clusters by technique, and make a rough schedule. Log analysis can be started immediately, whereas the usability testing will take several weeks to prepare and recruit. The contextual inquiry can start at just about any time, but since, in this hypothetical situation, there are not enough resources to do a complete study immediately, a small round is begun immediately with the assumption that more can be done later.

start sidebar

What

When

Questions

Log analysis

Immediately

What is the ratio of people who abandon the shopping cart to those who complete a transaction?

On what page do people abandon it?

What pages do people open a shopping cart from most frequently?

Usability testing

Immediately (recruit now, test in two to four weeks)

Do people understand the instructions on the cart pages?

Do they know they're opening the cart?

When they abandon it, do they realize that they're abandoning it?

Under what circumstances do they open the cart?

How do they use the cart?

Contextual inquiry

Immediately (recruit now, interview people in one to two weeks) and ongoing

How do they shop on the site?

How do they shop for these things outside the site?

Do they know what a shopping cart is?

end sidebar

Continue through your issue list in order of priority, expanding all the items this way. As you're expanding them, look for similarities in the questions and places that research can be combined to simultaneously address a number of issues. In addition, look for places where competitive analysis can produce an interesting perspective.

You can also start with the list of techniques and use that to generate further questions and research ideas. So, for example, you could start by saying, "What can we learn with contextual inquiry? Will that address any of the research goals?" and work your way through the other techniques.

A single entry from the final list could look something like this.

start sidebar

What

Usability testing

When

Plan immediately, test in two to four weeks

Questions

Shopping cart questions

Do people understand the instructions on the cart pages?

Do they know they're opening the cart?

When they abandon it, do they realize that they're abandoning it?

Under what circumstances do they open the cart?

How do they use the cart?

How long does it take to make a purchase?

Navigation questions

How do people find specific items? (What tools do they use? How do they use them?)

How do people move from one major section to another? (What tools? How do they use them?)

Promo section questions

What order do people look at the items on the front door?

Do people understand the information in the promo section?

end sidebar

Consider this as a single unit for the purposes of scheduling. It's something that can be put into a scheduling or project management program. The specifics of what is tested may change when the test plan is being put together, but this will get you something that can be scheduled and a good idea of what is being tested and when.

Asking Questions across Multiple Projects

Because of the fast pace of most Web development schedules, the plan should concentrate on the short term, but it should not shortchange the long term. It's always tempting to focus on the goals for the next release and leave fundamental questions about a product and its users to long-term projects. But deeper answers are precisely what can make or break a product over the long run. These questions should not be put off because they're general or difficult to answer. In fact, they should be investigated as soon as possible since the earlier they produce results, the quicker that planning, design, and implementation of core product changes can be made.

However, just focusing on deep questions when there's a release coming up is rarely a wise plan, either from the perspective of the product or the role of the researcher in the company. Thus, the research plan should be structured as a set of parallel projects, with long-term goals cutting across several projects. Each project addresses whatever short-term questions need to be answered, but also asks a couple of key questions to nudge knowledge about the fundamental questions at each time, too.

This can be represented as a grid, with each fundamental question in a column, while research projects label the rows. Thus it's possible to keep track of which projects are asking questions about which fundamental goals they're addressing. This keeps long-term goals from being neglected by the research process. The following table shows one such representation, with the dark shaded cells representing which project gathers information investigating which goal (with project 4 an in-depth investigation of issues studied in projects 2 and 3).

start sidebar

Search Engine Results

Comprehensi-bility of Navigation

Shopping Cart Abandonment

Usability testing 1

Focus group 1

Log analysis

Usability testing 1

Etc.

end sidebar

Of course, not all projects are going to provide data for every goal, but keeping this structure in mind will allow you to keep the long-term needs in perspective while still getting the short-term work done.

Note

The research plan should be updated frequently: in between every round of research, with every major update of the software, and, in general, with every addition to the knowledge of the user experience and whenever company priorities change. It would not be unusual for there to be updates every week or two. Versioning every update helps keep track of all the changes.

The Format of the Plan

Until this point, I have intentionally avoided presenting a specific structure or look that the research plan should have since that will vary based on your needs and resources. It should be flexible and adapted to your environment. If you use project management or scheduling software, a lot of the plan can be represented in it. If you plan to show it to management on a regular basis, it can be in a more formal written form that can be folded into the product development plan. If your company culture prefers to organize with Post-its, it can be a wall of Post-its. Whatever. It should fit a style that is useful and comfortable for you, but that can be shared and integrated with the larger product plan. The plan is a document that you are going to use to communicate the structure of your research and to sell the value of your work to your product team and your company as a whole.

There are some things every research plan should do.

  • Set expectations. The people conducting the research and the recipients of the results should know what research is being done, how it's being done, and what results they can expect. Don't overload expectations for any single round of testing. A round of testing will not validate or condemn the entire project, and it should not be expected to. A research plan should also communicate that it is a flexible and ever-changing document. This can be done through version numbers or even expiration dates ("This research plan is valid until 12/2/2003").

  • Set schedules and responsibilities. Who is going to do what when? How does the research schedule integrate into the larger development process? This should be specific in the short term, but it can be more general in the long term, except for research that's directly tied to the larger schedule.

  • Specify goals. Every research project and the research plan as a whole should have specific goals associated with them. The goals collected at the beginning of the process drive the specifics of the research. It should be clear what they are.

  • Specify outputs. There should be outputs for every research project based on the needs of the stakeholders, specifying the information that's going to be presented. Ideally, the actual deliverables (report, presentation, etc.) should be described.




Observing the User Experience. A Practioner's Guide for User Research
Real-World .NET Applications
ISBN: 1558609237
EAN: 2147483647
Year: 2002
Pages: 144

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net