Web surfers who know about your organization, products, or services but don't have your URL may rely on search engines and directories to find your Web site. Still others, who don't even know your company exists, may use search engines to enter search phrases that should apply to you. How do you increase the odds of having your site listed among those found? Search Engine An online utility that enables Web surfers to find Web sites based on search criteria they provide. This part of the chapter explains how search engines work and tells you what you can do to get your Web site found by the people searching for it. Tip Search Engine Watch is an excellent resource for getting more information about search engines ”including lists of search engines. Visit it at http://www.searchenginewatch.com/. Search Engine BasicsSearch engines are one of the most misunderstood features of the Web. Few people know how they work or why different search engines come up with different results for the same search phrases. Even fewer people know how the search engines know about all the Web sites they list on the search results pages. Here's a quick look at how search engines work. Tip To learn more about how to use search engines to find Web sites, check out Search Engines for the World Wide Web: Visual QuickStart Guide , a Peachpit Press book by Alfred and Emily Glossbrenner. You can learn more about it at http://www.peachpit.com/books/catalog/K5842.html. Robots and Spiders and Crawlers ”Oh My!Many search engines ”such as HotBot (http://www.hotbot.com/) and WebCrawler (http://www.webcrawler.com/) ”utilize software programs called robots , spiders , or crawlers . These programs venture out onto the Web when activity is low (whenever that is) and search for new pages. When they find a site they've never visited before, they follow all the internal links within the pages to learn about the pages on the site. They also occasionally (but not often enough) revisit sites to update the information they already have.
What kind of information do they gather? Well, that depends on the program. Some programs gather just Web page titles ”the information that appears in the Web browser title bar. (This is not the same as a file name). Others gather every single word in the page. Still others gather information stored in special meta tags embedded in the page (more on that later). DirectoriesWhile some search engines depend solely on robots, spiders, and crawlers, others ”such as Excite (http://www.excite.com/) and Yahoo! (http://www.yahoo.com/) ”start with a directory that entries can be manually added to. Some of these directories are static, containing only the information manually entered. Others use directory entries to send out robots, crawlers, and spiders to gather information directly from the Web sites entered. The DatabaseThe information gathered about Web sites ”whether gathered by robots, etc., or by manual entries into a directory ”is stored in a database that can be indexed and searched. Web surfers search the database based on words or phrases. Sites that match the criteria are displayed in a search results list.
Who's On FirstSearch engine result lists are usually very long ”sometimes with tens of thousands of matches. (Of course, not all of them appear on one page; you'll have to click through hundreds of pages to see them all.) The sites that most closely match the search word or phrase should appear at the top...right? Well, sometimes. It depends on the search engine and what you entered. Figure 11.1 shows the first page of results for a query on Search.com (http://www.search.com/) for horse camping in arizona . (I'm going camping tomorrow and it's on my mind.) If you can read the tiny print, you can see that not all of the pages seem to relate to the search phrase. (If you can't read the tiny print, please take my word for it.) Yet several pages further into the search results is information about the campground I'll be going to. Why didn't it come up on top? Figure 11.1. The first search results page for a search.com search.
The answer is: it depends. Most search engines list results based on how well the search word or phrase was matched in its database. (In my opinion, this is the way it should work.) Others ”such as Direct Hit (http://www.directhit.com/) ”list results based on popularity ”how many people click through to that page. Still others put paid-for listings near the top and others below it. And of course, whether a listing appears at all depends on whether the listing was added to the search engine's database to begin with. The Bottom LineThe bottom line is this: to be found by a search engine, your site must be included in that search engine's database. Your job is to get included in as many search engine databases and directories as possible. Tip Search Engine Watch's Search Engine Submission Tips page (see Figure 11.2; http://www.searchenginewatch.com/webmasters/) is a great place to learn more about how search engines work and how you can increase your odds of being near the top of the search results list. Figure 11.2. The Search Engine Submission Tips page on Search Engine Watch is a great way to learn more about getting your site found by search engines.
Using Meta TagsIf you were reading carefully , you probably caught my use of the phrase meta tag . I didn't define it then because I was saving it for now. Meta Tag An HTML tag for embedding certain types of information in a Web page document. Meta Tags, DefinedMeta tags are HTML tags that can be inserted in the HEAD part of an HTML document to embed document information. Now, in English: meta tags enable you to include information about a Web page where it won't be seen by site visitors but can be seen by robots, spiders, and crawlers. There are many uses for meta tags, but the ones you should be most interested in are for storing description and keyword information. This is the information most often picked up by robots, etc. Tip Be sure to include the description and keyword meta tags in all the pages of your Web site. Remember, not everyone visits your site starting with the Home page. DescriptionThe description information should be a short, concise description of the site or page. Don't repeat the page title; it's a waste of words. Since you're usually limited to 256 characters , make the most of them. The format for the description meta tag is as follows : <META name="description" content=" concise description of Web site or page "> What does the description meta tag look like in an HTML document? Figure 11.3 shows an example from the Home page of one of my sites, wickenburg-az.com (http://www.wickenburg-az.com/). Figure 11.3. Meta tags for keywords and descriptions are inserted in the HEAD part of an HTML document, along with the page's title.
KeywordsThe keyword information should be a list of key words or phrases, separated by commas. Again, there's usually a limit of 256 characters for keywords, but not all robots, etc., accept that many. If a robot, etc., is limited to, say, 150 characters, it will take the first 150 characters it finds. For that reason, you should put the most important keywords at the beginning of the meta tag. Also, don't repeat the same words over and over again because most robots, etc., are programmed to ignore repetition ”and a few will actually penalize you for it! The format for the keyword meta tag is as follows: <META name="keywords" content=" list of key words and phrases separated by commas "> Figure 11.3 also shows the keywords tag from the Home page of wickenburg-az.com. Note that it includes the words that best describe the site's contents ”and even some misspellings of those words! (When I analyze my site's logs, I often find that site visitors used an incorrect spelling of Wickenburg and still find my site.) Meta Tags in Search ResultsFigure 11.4 shows a close-up of some search results for the phrase horse camping . Figure 11.4. A close-up of some search results reveals how meta tags are not always properly used.
Most of the entries include the name of the page, followed by a brief description. The description comes from the description meta tag. This is how you want your pages to appear in search engine results. But look at the second entry, "Camping at Crazy Horse." The description text doesn't appear to be a description at all. Sure enough, when you click the link you can see that the "description" is really the first handful of words from the Home page (see Figure 11.5). Figure 11.5. When you omit the description meta tag from an HTML document, the first handful of words on the page appear as a description.
Now look at the last entry in the Figure. This entry is a real mess. It appears that the Web author included keywords in the page's title, as well as the description. Figures 11.6 and 11.7 partially confirm this; the title is entered three times in the HTML document's source code. Although the description meta tag appears okay, the error in the title's code may have caused the robot, etc., to pull up the wrong information for the description. Figure 11.6. In this example, the page title includes keywords,...
Figure 11.7. ...a fact that's confirmed when you inspect the page's source code. Perhaps the inclusion of the title code in triplicate is what messed up this page's listing in the search results; the description meta tag appears okay.
As you can see, when you set up meta tags correctly, your site's listing in search engines that use meta tags can be exactly what you want it to be. But when you do it wrong, the listing is wrong. Adding Your Site to Search Engine DatabasesOnce you've set up meta tags for all the pages on your site, you can help improve the chances of robots, etc. finding your site by manually adding your site's information to search engine directories. Then, if that search engine sends out robots, etc., your site will be found and indexed.
This really works. My Web server sits on a computer desk beside my main production computer. Sometimes I leave the monitor turned on so I can watch activity. (You think I'd have better things to do with my time, right?) I've noticed that within 10 seconds of adding one of my sites to a directory, a robot sometimes appears on my server to check out the site's Home page. Now that's service! To add your site to a search engine's directory, begin by visiting that search engine's main page. Now look for a link that says something like "Add a site" or "Add a URL" or "Suggest a Site." This link is usually in very small letters near the top or bottom of the Home page, but sometimes it can be buried on other pages. And sometimes it may not appear at all. Figure 11.8 shows how it appears at the bottom of the Excite Home page (http://www.excite.com/). Figure 11.8. You can find a tiny link for adding a URL at the bottom of Excite's Home page.
When you click the link, a page with instructions for adding your site will appear. Read the information and follow the instructions. Figure 11.9 shows Excite's Add URL page. It offers a pretty straightforward form. Fill it out and click the Send button. That's all there is to it. Figure 11.9. This page on Excite offers a simple form for adding your site to its database of Web sites.
In some cases, a message will appear with some kind of confirmation that your request is being processed . Sometimes you'll be told that it takes four to eight weeks (or some other silly period) to process your request. You might want to make a note of it, then try the search engine after a reasonable amount of time has passed. If you're not listed, try adding it again.
Adding Your Site to Directories & Other SitesYou can also add your site as a link to other sites, including directories of sites in your geographical or business area. This is especially useful if the other site is a good match for yours. For example, a state directory of accountants would be a good match for a CPA and a tourism directory would be a good match for a hotel or tour service.
To add yourself to one of these directories, begin by visiting the site. Make sure it's a good match for you business. Then poke around until you see some way to add a link or ask about adding a link. If all else fails, e-mail the Webmaster. Be aware that some sites charge a fee for links. Keep your advertising budget in mind when considering whether a link is worth the fee. (And please read the sidebar on the next page for my very strong opinions on the subject.) Other sites will only ask for a reciprocating link. Before agreeing to that, make sure the site is up to your standards. I discuss reciprocating links in more detail back in Chapter 6. |