When we let users loose to go anywhere they wanted on the Web, they went to a search engine 88 percent of the time. Only in 12 percent of cases did they go straight to a Web site that they hoped would help them with their problem. For example, when the task was to research vacation travel to Mexico, only a few users went to a travel site that they already knew and trusted, such as Expedia or Travelocity. The vast majority employed a search engine to help them find sites.
The fact that search engines have become the dominant tool for users looking for solutions is clearly why search advertising has become such a profitable business. Search ads are the main way you get in front of prospective customers at the exact moment they are looking for new vendors.
Increasingly, the Internet user experience is becoming one of dipping a toe into Web sites rather than truly "visiting" them. Using search engines as their Web interface, people often grab query-related nuggets from sites without engaging with the sites themselves.
The search engine has always been an important tool for users. In 1994, when we were trying to understand why people used the Web despite its lousy usability, we asked everyone who came by our lab two questions: What are you doing online? What are your favorite sites? Their answers were strikingly diverse. People's pursuits ranged from golf to knitting to Linux to military history, and their favorite sites varied just as widely. In fact, the only commonality among them was that all users named a search engine among their top sites.
The conclusion was clear: The Web's strength comes from narrowly targeted sites that provide users with highly specialized information that they need or care about passionately. It was also clear that Search was a hugely important general-interest service because even back when the Web had only 30,000 sites, locating specialized ones was nearly impossible without help.
A major change over the years has been a decline in using Search to identify good sites as such. People are looking for answers. The Web as a whole has become one agglomerated resource for people who use search engines to dredge up specific pages related to specific needs, without caring which sites supply them. The job of search engines is no longer resource discovery but to answer questions.
It's a testament to the Web's growth that users now view it as integrated whole. This is good news for search engines and users but not for Web sites.
This changing behavior is explained by the theory of "information foraging": The easier it is to track down new resources, the less time users spend at each resource. Thus, improvement in Search quality over time is driving the trend toward answer engines. Always-on connections have a similar effect, because they encourage information snacking and shorter sessions. Finally, Web browsers' despicably weak support for bookmarks/favorites has contributed to the decline in users' interest in building a list of favorite sites.
It's a testament to the Web's growth that users now view it as integrated whole and don't bother with Web sites; they assume that anything they want to know is available somewhere. They just have to ask. "Web sites" weren't really a tangible concept until 1993 anyway. The pre-Mosaic Web in 1991 and 1992 was exactly that: a web of information where the fundamental unit was the article, not the server hosting a particular Web page. So this new user behavior is actually a reversion to the Web's original vision to some extentthough not completely because users still have some favorite sites that they treat as resources.
For search engines, becoming the user interface to the Web's embarrassment of riches is good news. It's also good news for users, who can find answers by visiting a few search hits rather than enduring the obscure design and poor navigation found on many sites. But is this good for Web sites? Unfortunately not. There is very little value in giving answers to users who don't know or care who provides the service.
E-commerce sites are something of an exception, because they often get a sale from users dipping a toe into their catalog. E-commerce sites differ from other Web sites in having confirmation and fulfillment stages that follow up on users' initial visits, and these steps can also grow the site's mindsharethe likelihood that the site will come to mind when the user is thinking about the type of products or services it offers. Thus, closing the first sale is one of the most important drivers of subsequent e-commerce sales.
It would be self-defeating for e-commerce sites to refuse shopbots (software that searches the Web for a product's lowest prices), prohibit deep links, or employ other tricks that require users to enter at the homepage and spend time navigating the site. Any barrier between the customer and the product translates into lost sales.
Even sites that don't sell must accept the trend toward users' answer-seeking behavior. Walling yourself off from the Web's web-like nature won't solve the problem. Tracking numbers of unique visitors is now irrelevant. Most such visitors are sampling a single page to get an answer, not engaging with your site. Instead of tracking them, count loyal users as a key measure for site success.
How People Use the Search Engine Results Page
The search engine results page is usually referred to as a SERP. It's fitting that this term is rarely used in plural because most users don't see more than one SERP per query. In 93 percent of searches, the users in our study only visited the first SERP, which usually held ten search results plus a number of ads. In only seven percent of cases did users page on to a second SERP, and the number who visited three SERPs for a single query was too small to provide a firm estimate, but it was likely less than one percent.
Not only did most users make do with a single SERP; most of them didn't even bother reviewing the entire page. Only 47 percent of users scrolled the first SERP, which means that 53 percent saw only those search hits that were "above the fold." (Originally a newspaper term, "above the fold" refers to what part of a Web page is visible on a screen without scrolling.) On the most widely used search engine, Google, users can only see four or five results above the fold, if they view the page on the most common screen resolution of 1024-by-768 pixels, like those in our study. In addition to the four or five "organic" search results, there are typically six or seven advertisements above the folda total of ten or so items to choose from.
This table shows how often users clicked the search hits at different placements on the SERP list. Since the first SERP only displays ten organic links, users needed to go to the second SERP to click links number 11 or higher. Of the seven percent who actually visited the second page, only five percent clicked therea discrepancy explained by the fact that a few users who visited the second page returned to the first page. It's no big surprise that the top links get more clicks than bottom links, but it may be surprising to see just how dominant the No. 1 spot is, with more than half of all the clicks.
Where Users Click on the SERP
With items 1 through 10 on the first search engine results page, 95 percent of clicks were there. This table only counts clicks on the organic links. Sponsored links follow a similar distribution with disproportionately more clicks on the top ads.
Using Keyword Pricing to Estimate Usability Improvements
It is very expensive and time consuming to conduct broad usability studies like ours of a wide range of Web sites. We can't do it every year, so we can't accurately track usability improvements on a steady basis through direct measurements. Fortunately there is an indirect measurement that's more feasible to acquire and that can be used as a proxy for Web site usability: the prices paid for keyword advertising on major search engines.
Over time, the price for keyword advertising has shown a distinct upward trend. Future increases in keyword bids will be tightly related to improvements in Web site usability.
On most major search engines, keyword advertising works as follows: First a company must consider how much it is worth paying to try to attract a visitor who has searched for a certain keyword and is presumably interested in a product or service associated with that word. The company then bids for each keyword, and the search engine usually displays the ads from the top eight bidders. (There are many twists to search engine advertising, but the basic principle is an auction in which a limited number of slots are sold to the highest bidders.)
For example, our own company, Nielsen Norman Group, usually pays 31 cents per user who searched for "usability training" when that user clicks through from the search engine to the page for our annual usability conference. Similarly, a company that sells vacation packages to Mexico would be willing to pay a substantial amount for users who are searching for "vacation Mexico" or "hotels Yucatan."
Over time, the price for keyword advertising has shown a distinct upward trend. Over the long term, increases in keyword bids will be tightly related to improvements in Web site usability. During the next several years, keyword prices will increase substantially as more companies realize the benefits of search engine advertising. Search ads are the best way to promote a Web site because keyword matching attracts users who are looking for the exact thing you're offering.
Currently most Internet marketing managers are clueless about keyword advertising and spend most of their budgets on advertising techniques that were appropriate in old media but don't work in an interactive medium like the Web. But one of the beauties of the Web is its accountability: You can track the results of advertising expenditures in terms of both click-throughs and the extent to which clickers turn into spenders. Because of this, even clueless managers will gradually reallocate their budgets to the best performing promotions, which will most often be search advertising.
As more and more companies discover the high return on investment (ROI) of search advertising, keyword prices will be bid up because of the increased demand for a fixed supply of advertising positions. Of course, a search engine could put more ads on each page but only at the price of diluting the effectiveness of each ad. What search engines are really auctioning off is the attention of motivated users, and as long as motivation stays the same, there's a limit to how much a page can be subdivided and remain profitable.
It's difficult to estimate when all companies will fully understand the benefits of search advertising, but 2010 might be a reasonable guess. When that happens, further growth in keyword pricing will be due to improvements in the target Web sites.
As we've said, each company should bid no higher than what it takes to get a positive ROI. So let's say that a company brings in an average profit of $2.00 for each new customer. It might bid up to $1.99 for a click, for marginal profits of one cent per new visitor. Bidding $2.01 would be too much and would result in losses.
Now let's assume that this company performed enough usability work on its Web site to double the conversion rate for new visitors, which is a likely result. The company now makes an average of $4.00 per new visitor, so it can bid up to $3.99 for each click on the keywords that generate the $4.00 visitors. Improving usability and doubling the conversion rate doubles the bid that the company can afford to pay the search engine.
Of course, our example company would hope that it wouldn't have to increase its search engine bids by quite as much. It would be preferable to pocket some of the increased profits from usability improvements instead of handing them over to a search engine. In the short term, the company would indeed be able to retain some of the profits because its site would be better than its competitors'. Until its competitors also increase the value of their visitors, they could not afford higher search bids. Sooner or later, however, they will conduct their own usability projects and improve their profits, allowing them to increase their bids. As soon as eight competitors have doubled their bids, it will be necessary for our example company to double its bid again. Since even one cent's profit is better than no profit, rational companies will increase their bids accordingly.
In the long term, keyword prices will tend to increase at about the same pace as Web site conversion rates. Since the conversion rate is a key measure of usability, bids for search engine advertising will continue to indicate the extent to which Web sites are improving their designs.
As Web sites improve, search engines will confiscate almost all the increased profits they gain from increased usability. In other words, search engines need do nothing but watch their incomes grow as mainstream Web sites do all the hard work of improving the Web. Is this fair? No. But the reality is that search engines drive much of the new traffic that a site can hope to attract.