How Internet Search Engines Work

Each search engine uses a crawler or spider with its own set of rules guiding how documents are gathered. Some follow every link on every home page they find and then, in turn, examine every link on each of those new home pages, and so on. Some spiders ignore links that lead to graphics files, sound files, and animation files. Some ignore links to certain Internet resources, such as newsgroups, and some are instructed to look primarily for the most popular home pages. It can take a spider from several seconds to many minutes to crawl each site it finds, depending on the size and complexity of the site.


How the Internet Works
How the Internet Works (8th Edition)
ISBN: 0789736268
EAN: 2147483647
Year: 2004
Pages: 223 © 2008-2017.
If you may any questions please contact us: