Quality Management

Concepts must be continuously tested throughout the development process. Showing concepts to human-computer interaction experts allows key problems to be pinpointed in the early phases of design. Later in the process expert evaluation can be used to complement user test for aspects that are hard to evaluate with users, such as checking usability in continuous use and assessing the influence of UI elements that are still missing. The main advantage of expert evaluation as a method of concept testing is that it is flexible with regard to presentation materials. Experts have knowledge about the product development and design process. Consequently, it is possible to get good readings from unfinished material supported by verbal qualifications, references to products and styles the experts know, and so on. An expert analysis of a concept can be done in 2 to 3 days. Since the results depend on the competence of the evaluators, the approach requires strong trust on the part of management in the expertise available to it. Without qualified evaluators, it just does not make sense to carry out these exercises.

Whenever possible, usability testing with interactive prototypes is preferred. They provide a much richer tool for capturing feedback. Even an extremely simple interactive simulation done with a low-fidelity prototyping tool can be highly effective.

User tests in themselves don't necessarily consume much time, but careful preparations such as recruiting users and planning tasks will increase the total duration. A proper usability test can seldom be done in under 4 to 5 days. (See Chapter 8 for the resources calculation for an international test round.) User testing should obviously be conducted with test users who represent the right target segment. Sometimes, however, the design team has to use company-internal personnel for confidentiality reasons, or when wider sampling is impossible because of tight schedules.

During a new mobile phone UI concept creation project at Nokia, which typically lasts about a year, we carry out three to four rounds of testing. When the design team is satisfied with the concept and it's getting close to implementation level, a design verification world tour is arranged. These are typically conducted in several countries, including a selection of the following: East and West coasts of the United States, Japan, Hong Kong, mainland China, and a few European countries. Rarely does a tour reveal anything radical; radical problems will have been ironed out before the designers packed the suitcase and set out. Regardless of their culture, people tend to approach logical usability problems and goal-oriented tasks much the same way throughout the world. Cultural differences have more to do with preferences in graphics and vocabulary than with the core interaction. Therefore the logical model of dialogue can be global, whereas displays designed for Europe and Japan have to be extremely different.

Usability tests, even when carried out with all necessary care, can be biased. There are several possible sources of error, including subjects' skills, subjects' motivation, the functionality of a simulation, and the inevitable differences between a laboratory environment and a natural one. Sometimes people involved in testing may interpret the results too blindly and propose a seemingly working solution that nevertheless has poor usability. The temptation to use usability as a lever for personal opinions and preferences has to be avoided by scrupulously critical interpretation of results, because biased usability is one of the world's worst sales tools. If it happens to a designer even once, his or her credibility will be destroyed forever. Think of marketing hype, where every product with a few keys and a display is pitched as 'easy to use' even as customers struggle with it. Ease of use can be claimed very easily, and it is hard to get caught claiming it falsely. At Nokia, we do regard our products as usable, but for just that reason we seldom cite usability as a key selling argument in marketing communications. We prefer to maintain our credibility by letting that message be conveyed through word of mouth from user to user.

When describing new concepts, the designers must always be more specific than saying that a concept is usable. For instance, one can sharpen the point just by saying that the design is 'more efficient' or 'easier to learn' than another one. Jakob Nielsen's well-known usability attributes have turned out to be straightforward and practical tools for focusing usability. In a way they can be seen as usability dials for adjusting user experience with a given concept; we want 20 percent more learnability and 30 percent more memorability, and we'll trade you 30 percent of efficiency to get it.