Can I Really Search the Whole Web?

Well, yes and no . (Don't you hate that answer?)

Although using the various search sites works similarly, each has its own unique search methods . But, more important, each has its own unique set of files ”a database ”upon which all searches are based.

You see, no search site actually goes out and searches the entire Web when you ask it to. A search site searches its own index of information about the Web ”its database. The more complete and accurate that database is, the more successful your searches are likely to be.

The database for a search site is created in either (or both) of two ways:

• Manually ” Folks who've created Web pages, or who've discovered pages they want the world to know about, fill in a form on the search site's Web site to add new pages (and their descriptions) to the database. If the search site's editors deem the site to be worthy of inclusion, it gets added.

• Through a crawler (or spider, or worm) ” All these creepy-crawly names describe programs that systematically contact Web servers (at regular intervals), scan the contents of the server, and add information about the contents of the server to the database. (They "crawl" around the Web, like spiders ”get it?) It takes the crawler a month or so to complete each of its information-gathering tours of the Web.

If a search site's database has been created by a crawler, the tool tends to deliver results that are more complete and up-to-date, whereas manually built databases tend to contain more meaningful categorization and more useful descriptive information. Also, most search sites with crawler-built databases do not offer you a way to search by browsing through categories ”a valuable technique you'll pick up later in this chapter. All search sites, however, support the main search method: entering a search term .

A search term is a word or phrase you type in a text box on a search site's main page, to tell the search site the type of information you're looking for. You learn all about search terms later in this chapter.

Because search sites search a database and not the actual Web, they sometimes deliver results that are out of date. You might click a link that a search site delivered to you and find that the page to which it points no longer exists. That happens when a page has been moved or deleted since the last time the search site's database was updated.

When this happens, it's no big whoop. Just click Back to go back to the list of results, and try another link.

How sites are ranked within their categories also varies from search site to search site. Some will simply list sites in alphabetical order. Some sell higher placement for a price, and then list the rest in alphabetical order. Some display results based on the likelihood that the site matches your search term. One site, Google, uses a unique page-ranking system that examines the Internet's elaborate system of links to determine a site's "value" based on the number of other sites that link to it.

Despite differences and strengths and weaknesses among the available tools, the bottom line is this: Any of the major search sites might locate a page or pages that meet your needs, or it might not. If you can't find what you want through one tool, try another. Because each tool has its own database, and each tool applies a different technical method for searching its database, no two search sites turn up exactly the same results for any given topic.