Scaling Up Your Search Solution

The most sophisticated search functionality in the world is no good if it cannot support the number of users and the intensity of the searching and indexing activity that you throw at it. Microsoft has addressed these scalability concerns with the latest generation of its search products in Windows 2003, Content Management Server, and SharePoint Portal Server 2003.

One of the design goals of SharePoint 2003 was to support one million users of a portal. For searching, the goal was to perform 95% of all queries in less than two seconds, and index up to 20 million documents (up from 5 million in version 1). This performance requires fast loading of thesaurus files and rapid crawling and indexing. SharePoint version 2 achieves at least double the indexing rate of version 1 in terms of documents per second. You can accomplish high availability and scalability by using a number of servers to perform different roles in the portal. Indeed, you may need multiple servers for the search function alone.

SharePoint lets you assign multiple servers to roles as web servers, search servers, and index servers. For instance, if you had a small number of users but a vast amount of data to index, you would propagate many index servers but only require a small number or perhaps a single search server. Figure 13.24 shows the notional architecture for SharePoint search.

Figure 13.24. SharePoint Search Architecture (Source: Steve Tullis, "Enterprise Search with SharePoint Portal Server V2.")

A search starts with a user request that hits one of the portal web servers. Search requests are balanced across search servers from the web servers and therefore directed to an appropriate search server. All search servers have identical copies of each index through index propagation.

Each indexing server is devoted to crawling and indexing up to four content indexes. For instance, you could devote an indexing server to competitive company web sites, and another to file shares within your organization. Separating the indexing from the search service means that the performance of searches will not be hindered by indexing activity, or vice versa.