Search Pages and Indexes

After your HTML documents are up and running on a Web server, you need to make sure that people can find them. Because most people look to search engines when they want to find something on the Internet, we'll start there.

Search Engines

Searchable indexes of Web resources. Some search engines (called indexes) also categorize information enabling people to search by categories and keywords.

Two types of search engines exist on the Web: spiders and indexes.

A spider (also called a Web crawler, or bot, which is short for robot) is an automated script that crawls through Web pages following hyperlinks to find related pages, and then builds a database with the contents of all the pages it visits. Google.com is an example of spider technology.
A search directory is a categorized list of sites on the Internet. The search directory's administrative personnel review the content of user-submitted Web sites and populate the categories. The Yahoo! directory is the best known example of this type of technology.

Search Bots

Search engine bots (also called robots, spiders, and crawlers) search through all Web pages and then index them according to the information they find. You can help the indexing portion be more accurate by using <meta> tags. Without <meta> tags, these bots treat every word in a document exactly alike. If you add keywords and descriptions to your documents, you increase the possibility that your Web pages will be found. You learned how to do this in Lesson 3, "Adding Text and More," but let's try a quick refresher. The following example shows the correct format for adding the <meta> tag to your documents. Figure 16.1 is an actual example of the <meta> tags used on the WebReference.com site.

<!DOCTYPE html      PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"      "http://www.w3.org/TR/xhtml1/DTD/xhtml1- transitional.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"        xml:lang="en" lang="en"> <head> <title>Your HTML Page</title> <meta name="keywords" contents="keywords that              people might use to search              for your page."> <meta name="description" contents="a brief              paragraph describing your              document."> <meta name="author" contents="your name"> </head> <style type="text/css"> </style> <body>     insert your document here. </body> </html>

Figure 16.1. The HTML source code for `http://www.microsoft.com` with the `<meta>` tags highlighted.

Tip

If you don't add any other <meta> tags, be sure to add the description. Not all search engines rank pages in the same manner, but they all display a description of the pages found. The right description can lure visitors to your site.

Search Bots

Figure 16.1. The HTML source code for http://www.microsoft.com with the <meta> tags highlighted.

Figure 16.1. The HTML source code for `http://www.microsoft.com` with the `<meta>` tags highlighted.