Chapter 20: Stored Procedures for Web Search Engines

The search engine is a standard element of every web application. Many tools are available to help web developers create a search engine when information is stored in the form of web pages, but if the information is on a database server, everything has to be customized. This chapter demonstrates some typical problems you may encounter and solutions for them.

Characteristics of the Environment

The following are the characteristics of a typical web-based application environment:

• The system has impatient users. Pages must be served quickly or users will leave and go to another web site.

• A three-tier architecture is typically used to make the system more scalable.

• To be scalable, the system may be deployed on a farm of web and/or middleware servers. If more users need to be processed, you can simply add more servers.

• Since it may be deployed on a farm of servers, the application must use stateless connections to the database.

• The database server cannot be scaled in the same manner as web and middleware servers. Federated servers allow users to split a database among several servers, but one record will be stored on only one server. It is also not such a trivial matter to add another database server. Tables that are vertically split between servers have to be reorganized so that a new server gets its share of the table. Therefore, the database server is a more precious resource than the web and middleware servers.

• Network traffic could be an issue—both internally (between servers) and externally (users could be linked by modem).