Section 20.4. Benefits to Users

20.4. Benefits to Users

As Microsoft's intranet environment matured in the mid-90s, it began to suffer from the same afflictions as most enterprise intranets: too many clicks required to get to desired information, difficult site-wide navigation, and the best documents buried deep within search results. And, as mentioned earlier, users and their champions began to ask for taxonomies to make these problems go away.

The MSWeb team's response is a work in progress. What we've described represents only a brief moment in the lifespan of a large company and its information systems. The team is taking an evolutionary approach, avoiding unrealistic goals of fixing all problems for everyone in a few years. In this way, there are no false expectations. But even in a short time span, many concrete benefits have been realized, and taxonomies are at the forefront of these improvements. With category label taxonomy, for example, the labels are more representative and consistent, improving navigation within MSWeb and between Microsoft intranet sites.

Searching is also greatly improved. By encouraging resource record creation with UCS, MSWeb is able to identify valuable content in the intranet environment and therefore can do a better job of crawling remote intranet content. Better crawling leads to more comprehensive indexing. Users are now querying indexes that represent a much larger body of content and a higher-quality collection of content. More importantly, users' queries are more powerful than beforethey are able to take advantage of MSWeb's descriptive vocabularies to reduce the ambiguity of individual search terms. Consider a search on "asp," a very ambiguous term. During a search, the descriptive vocabularies stored in the MDR are automatically invoked to expand the search by including the different meanings ("Active Server Pages" and "application service providers"). These terms are also displayed as executable searches on the search results page to narrow or refine the search.

The MSWeb team has also helped pioneer a positive and increasingly common trend: "Best Bets." These are search results that are the product of manual efforts (they are also discussed in Chapter 8). Often displayed before other, automatically generated results, Best Bets link a user to documents that a cataloger has determined to be highly relevant to the user's initial search query. Best Bets are designed to address the "sweet spot" in searching, which consists of the few unique search queries that constitute the majority of all searches executed. Why not add value to the small number of frequently executed searches by adding Best Bets to their results?

Figure 20-21 shows the results for the search query "asp" from the MSWeb intranet, and you'll note that the first five are all Best Bets. The components of the search resultsresource title, URL, description, and categoriesare drawn from the metadata schema, since the query searched an index of the controlled vocabulary terms assigned to these Best Bet records when they were indexed with UCS.

Figure 20-21. Best Bets search results are drawn directly from resource records created using UCS

The MSWeb team uses a function provided as part of the SAS console to determine which searches merit Best Bet coverage. By invoking the console's "View Query Logs" command (Figure 20-22) and specifying a date range and collection, it's possible to determine how many documents each query retrieved. If the "Where Query Returned" option is set to "0 Best Bets," we can learn which of those high-retrieving queries do not have Best Bets associated with them, and create new Best Bets accordingly.

Figure 20-22. The "View Query Logs" function is useful in determining popular queries

Another SAS Console function is "View Metrics." Its "Ranked Hit Clickthrough" option provides a graphic representation of the rank of documents in a particular query's search results that are being clicked through (Figure 20-23). Typically, the Best Bets, ranked at the top, have a far higher clickthrough than other documents.

Figure 20-23. Best Bets are typically the most clicked-through documents

So, does this hybrid approachthe combination of manually and automatically generated resultsactually help users? It may be too early to tell, but the initial data is promising. Users are performing 18 percent fewer searches since Best Bets were implemented; this might suggest that the results of their initial searches are more successful, reducing the need to submit follow-up searches. And, as shown in Figure 20-24, users are clicking through the top results' links close to twice as much as they had before Best Bets were implemented. This may suggest that users are finding Best Bets results to be more relevant than automatically generated results.

Figure 20-24. Best Bets seem to have increased search-result clickthrough

Overall, the MSWeb team has attempted to measure the cumulative impact that better browsing, searching, and content have had on users. By performing a task-analysis exercise both before and after a major redesign, the team was rewarded with some hopeful results in terms of success rate, time on task, and number of clicks. The following table displays the results of the task analysis. The version 3.0 results were recorded in February 1999, prior to the implementation of the taxonomy-driven approach, and the version 4.01 results were recorded in July 1999, after the implementation of the taxonomy-driven approach.

Measure	v.3.0 Average	v. 4.01 Average	Change
Task success rate	68.30%	79%	+10.7%
Time on task	3 minutes 26 seconds	3 minutes 10 seconds	16 seconds
Number of clicks	13	5	8 clicks

Certainly, other factors may have had an impact on these numbers. But even if we discount them, there is still ample anecdotal evidence to demonstrate the value of the MSWeb team's efforts.