Response Time and User Satisfaction

Shneiderman posed the question best: "How long will users wait for the computer to respond before they become annoyed?" ^[19] Researchers say "it depends." The delay users will tolerate depends on the perceived complexity of the task, user expertise, and feedback. Variability also plays an important role in delay tolerance. Users can tolerate moderate levels of delay variability, up to plus or minus 50 percent of the mean.

^[19] Ben Shneiderman, "Response Time and Display Rate in Human Performance with Computers," Computing Surveys 16, no. 3 (1984): 265285. Keep it under 1 to 2 seconds, please .

A number of studies have attempted to quantify computer response times versus user satisfaction. Robert Miller found three threshold levels of human attention: ^[20]

^[20] Robert B. Miller, "Response Time in Man-Computer Conversational Transactions," in Proceedings of the AFIPS Fall Joint Computer Conference 33 (Montvale, NJ: AFIPS Press, 1968), 267277.

0.1s One tenth of a second was viewed as instantaneous.
1.0s A one-second response time was needed for users to feel they were moving freely through an information space.
10s A response time below 10 seconds was required for users to keep their attention focused on the task.

Miller proposed that the ideal response time is around two seconds.

Shneiderman agreed with Miller that a two-second limit is appropriate for simple tasks , as long as the cost is not excessive. Shneiderman found that users "pick up the pace" of computer systems, that they were more productive at shorter response rates, and that they "consistently prefer the faster pace," below 1 to 2 seconds.

Although users prefer shorter response rates, the optimum system response time (SRT) depends on task complexity. Fast SRTs cause input errors while longer response times tax short- term memory and frustrate users. Users want consistency in response times.

Because surfing the web is mainly a low-complexity activity, users prefer faster response rates. Usage studies empirically confirm this need for speed; most pages are viewed for less than a second and few for more than 10 seconds. ^[21]

^[21] Bruce McKenzie and Andy Cockburn, "An Empirical Analysis of Web Page Revisitation," in Proceedings of the 34th Hawaii International Conference on System Sciences (Los Alamitos, CA: IEEE Computer Society Press, 2001). Found that web page revisitation is over 80 percent, visits are short, and bookmark lists are long.

An Interview with Ben Shneiderman, Ph.D.

I talked to Dr. Ben Shneiderman, one of the leading experts on HCI, to find out more about the relationship between speed and user satisfaction on the web.

Andy King: How does speed relate to usability and success on the web?

Ben Shneiderman: Usability plays a key role in web success storiesdesign, graphics, navigation, organization, consistency, etc. all play important roles. Speed is also vital it's hard to get users to like a slow interface, and satisfaction grows with speed. Google is a good example of an excellent service that is even more valuable and appreciated because it is fast. Speed is the strongest correlate of user satisfaction.

King: Why do we prefer shorter response times?

Shneiderman: Lively interaction keeps the engagement high. For most people, wasted time, especially while just waiting for something to happen, is annoying.

King: What happens when we exceed our attention threshold (8 to 12 seconds)?

Shneiderman: Users not only grow frustrated, but they forget their next step, and have to reconstruct their intentionsoften making mistakes that only exacerbate their frustration.

King: What do you think of the flow construct for user satisfaction on the web?

Shneiderman: Rapid movement through complex sequences of actions that move users toward a desired goal contributes to the flow experience. Users should be working just at the right level of challenge, accomplishing something they desire . There is a great thrill of finding what you want, and getting it rapidly so you can move on to the next step. ^[22]

^[22] Ben Shneiderman, email to author, 24 August 2002.

System Response Time Guidelines

Shneiderman suggests the following guidelines for system response times: ^[23]

Users prefer shorter response times.
Longer response times (> 15 seconds) are disruptive.
Users change usage profiles with response time.
Faster is not always better. Users tend to increase the rate of interaction, which may cause corresponding increased error rates.
Users should be advised of long delays.
Modest variability in response times is acceptable (plus or minus 50 percent of the mean).
Unexpected delays may be disruptive.
Response time should be appropriate to the task:
- Typing, cursor motion, mouse selection: 50 to 150 milliseconds
- Simple frequent tasks: 1 second
- Common tasks: 2 to 4 seconds
- Complex tasks: 8 to 12 seconds

^[23] Ben Shneiderman, Designing the User Interface: Strategies for Effective Human-Computer Interaction , 3d ed. (Reading, MA: Addison-Wesley, 1998). An excellent HCI book for designers.

Negative Impressions and Perceived Quality

The speed at which your pages display can affect user perceptions of the quality, reliability, and credibility of your web site. Ramsay, Barabesi, and Preece studied the effects of slow-loading pages on user perceptions of web sites. ^[24] Using delays of two seconds to two minutes (with an interval of 19.5 seconds), they asked users to rate pages on eight criteria including "interesting content" and scannability. They found that pages with delays of 41 seconds or longer were perceived to be significantly less interesting and harder to scan. Note that the pages in this study loaded incrementally.

^[24] Judith Ramsay, Alessandro Barabesi, and Jenny Preece, "A psychological investigation of long retrieval times on the World Wide Web," Interacting with Computers 10 (1998): 7786.

Perceived Usability

Jacko, Sears, and Borella studied the effects of network delay and type of document on perceived usability. They found that perceived usability of web sites was dependent on the length of delay and on the media used in web documents. When delays are short, users prefer documents that include graphics. When delays lengthen, however, users prefer text-only documents because graphics are viewed as contributing to the delay. As users become more experienced , their sensitivity to delay increases , increasing the need for "delay reduction mechanisms." ^[25]

^[25] Julie A. Jacko, Andrew Sears, and Michael S. Borella, "The effect of network delay and media on user perceptions of web resources," Behavior & Information Technology 19, no. 6 (2000): 427439.

Perceived Quality of Experience

Morris and Turner found that perceived quality of experience (Shackel's utility dimension) affects the adoption rate of IT. ^[26] How users perceive the quality of a system can affect how much they will actually use it.

^[26] Michael G. Morris and Jason M. Turner, "Assessing Users' Subjective Quality of Experience with the World Wide Web: An Exploratory Examination of Temporal Changes in Technology Acceptance," International Journal of Human-Computer Studies 54 (2001): 877901.

They found that interface "enhancements" (graphics, animation, sound, etc.) had little effect on quality of experience "although these features may be aesthetically pleasingthey do little to remove actual barriers to the users' goal attainment ."

Perceived Quality of Service

The speed at which your pages display affects their perceived quality and reliability. Bouch, Kuchinsky, and Bhatti investigated the effects of delay on perceived QoS in order to find an acceptable QoS level for e-commerce transactions. They tested delays from 2 to 73 seconds for both non-incremental and incrementally loaded pages. ^[27] Users rated latency quality versus delay on a scale of high, average, to low (see Table 1.1).

^[27] Anna Bouch, Allan Kuchinsky, and Nina Bhatti, "Quality is in the Eye of the Beholder: Meeting Users' Requirements for Internet Quality of Service," in Proceedings of CHI2000 Conference on Human Factors in Computing Systems (New York: ACM Press, 2000), 297304.

Table 1.1. Web Page Quality Rating versus Delay
Quality Rating	Range of Latency: Non-Incremental Display	Range of Latency:Incremental Display
High	05 seconds	039 seconds
Average	> 5 seconds	> 39 seconds
Low	> 11 seconds	> 56 seconds

The results show a mapping between objective QoS and the users' subjective perception of QoS. Pages that displayed quickly (05 seconds) were perceived to be of high quality with high-quality products. Pages that displayed slowly (> 11 seconds) were perceived to be of low quality and untrustworthy. In fact, slower pages caused some users to feel that the security of their purchases may have been compromised, and they abandoned their transactions.

Figure 1.3 shows the actual data behind Table 1.1 for the non-incremental display. This figure plots the number of low, average, and high ratings versus latency. The range where high ratings turn to low is between 8 to 10 seconds for non-incremental downloads, closely matching what Nielsen and others have found.

Figure 1.3. Latency quality ratings show a drop-off at around 8 to 10 seconds.

Users tolerated nearly six times the delay for pages that displayed incrementally, although this tolerance decreased with usage. Test subjects rated pages as "average" with delays up to 39 seconds, and "low" with delays over 56 seconds.

The researchers also tested user requirements for speed by allowing them to click "increase quality" if they found the web page delay to be unacceptable. The average tolerance was 8.6 seconds with a standard deviation of 5.9 seconds. They attribute this large deviation in acceptable download times to contextual factors like web experience and user expectations. The longer users interact with a site, the less they tolerate delays.

Users will tolerate longer delays with tasks they perceive to be computationally complex. Users expect database access or complex calculations to take longer than displays of cached or static pages. Users form a conceptual model of system performance, which influences their tolerance for delay.

Credibility

Fogg et al. found that slow-loading pages reduce ease of use, which reduces credibility (or trustworthiness and expertise). Only difficult navigation was found to hurt credibility more. ^[28]

^[28] B. J. Fogg et al., "What Makes Web Sites Credible? A Report on a Large Quantitative Study," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (New York: ACM Press, 2001), 6168.