Optimizing for Search Engines

---

Optimizing your site for a search engine is not difficult. The first thing to do is to start to think like a search engine-in other words, don't really think at all. Search engines look at pages and make educated guesses about their content by following a set of rules to try to understand what the page is about. For example, search engines look for word frequency, linking information, <meta> tags, and a variety of other things. However, when you get right down to it they really can't perfectly tell the difference between a page about the Miami Dolphins football team and a dolphin show in Miami, because search engines generally rely on keyword matching in conjunction with some heuristics such as the placement of words in a page or the number of linking sites. So if a page author knows what a search engine is looking for, it is easy enough to optimize a page for the search engine to rank it highly. The next few sections provide a brief overview of some of the things search engines look for as well as some tricks people have employed to improve their search rankings.

Relevant Text Content

Probably the best ways to get indexed is to have the keywords and phrases actually within the content of the page. Many search engines will look at text within a page, particularly if it is either toward the top of the page or within heading tags such as <h1> or <h2> . Search engines may also look at the contents of link text. Thus,

  <a href="specifications.html">  Specifications  </a>  

is not as search engine friendly as

  <a href="specifications.html">  Robot Butler Specification  </a>  

One problem with the fact that search engines focus on page text is that often page authors create home pages that are primarily graphic. Search engines might have little more to go on than the <meta> tag and page title and thus could rank the page lower. Consider first using the alt attribute for the <img> tag to provide some extra information; for example,

  <img src="robot.gif" alt="Butler-1000: Demo Company's industry   leading robot butler"/>  

Of course, putting the actual text in the page would be better. Some page authors resort to either making text very small or in a color similar to the background ¾ or both ¾ so that users won't see it but search engines will pick it up. For example,

  <font size="1" color="white">  The Demo Company Butler1000 is the best robot butler. The Demo Company Butler1000 is the best robot butler. The Demo Company Butler1000 is the best robot butler.  </font>  

Be careful with the small or invisible text trick. Many search engines will consider this to be spamming and might drop the page from the search engine.

Using <meta> for Search Engines

In addition to page content, search engines may look at the <meta> tags for keywords and descriptions of a page's content. A <meta> tag such as

  <meta name="Keywords" content="Butler-1000, Robot butler, Robot butler   specifications, where to buy a robot butler, Metallic Man Servant, Demo   Company, robot, butler" />  

could be used in a Demo Company page about robot butlers. Notice how the content started first with the most specific keywords and phrases and ended with generic keywords. This should play into how most users approach search engines.

Once a search engine looks at the <meta> tag, it can rate one site higher than another based upon the frequency of keywords in the content attribute. Because of this, some page authors load their <meta> tags with redundant keywords:

  <meta name="Keywords" content=" Robot butler, Robot butler, Robot butler,   Robot butler, Robot butler, Robot butler, Robot butler, Robot butler,   Robot butler, Robot butler" />  

However, many search engines consider this to be keyword loading and might drop the page from their indexes. If the keyword loading is a little less obvious and combinations of words and phrases are repeated like so

  <meta name="Keywords" content="Robot butler, Butler-1000, Metallic   Man Servant, Robot butler, Butler-1000, Metallic Man Servant, Robot   butler, Butler-1000, Metallic Man Servant, Robot butler,   Butler-1000, Metallic Man Servant" />  

the search engine might not consider this improper. An even better approach is to make sure the pattern of repeating words isn't quite as obvious as it varies its order, as shown here:

  <meta name="Keywords" content="Butler-1000, Robot butler, Metallic   Man Servant, Robot butler, Butler-1000, robot, Robot butler,   Democompany, Metallic Man Servant, Butler-1000, robot, butler,   Robot butler, Butler-1000" />  

However, be aware that search engines might still notice the heavy use of certain words or phrases and consider this spamming, potentially reducing the page's ranking or dropping it from the index completely.

Search engines also look at the description value for the <meta> tag. For example,

  <meta name="Description" content="The Demo Company Robot Butler is   the most outstanding metallic man servant on the market. The   Butler-1000 comes complete with multiple personalities and voice   modules including the ever-popular faux-British accent." />  

would be included on the robot butler page and could be examined by the search engine as well as returned by the search engine on the results page. Because it might be output for the user to see, you should provide some valuable information in the description that will help the user determine whether they want to visit your site. Preferably, keep the description to a sentence or two, and at most, three or four sentences.

Titles and File Naming

It's important for search engine ranking to make sure your page has a very good title. For example,

 <title>Robot Butler</title> 

is a bad title as far as search engine ranking goes. A better title might be

  <title>  Butler-1000: Specification of Demo Company's Robot Butler, the leading metallic man servant on the market  </title>  

Remember that people also look at page titles, and they are used for bookmarking , so a really long title might be more for search engines than for users.

The name of a file also can be important for search engines. Rather than naming a file butler.htm, use butler1000-robot-butler.htm. Consider that if you have a good domain name and directory structure, you can create a URL that almost makes sense. Consider, for example, if we named our server democompany.com as well as www.democompany.com. We might have a URL like

 http://democompany.com/products/robots/butler1000-robot-butler.htm 

Notice how this almost includes the same information as the title. This provides a secondary benefit of letting the user know where they are, rather than resorting to cryptic URLs such as

 http://democompany.com/products.asp?prod=robots&mod=butler1000 

Links and Entry Points

Search engine ranking may also depend on the number of links leaving a page as well as the number of pages that link to a page. Landmark pages such as home pages tend to have a lot of outgoing and incoming links. Search engines would prefer to rank landmark pages highly, so it is important that key pages in your site have links to them from nearly every page. Some search engines also favor sites that have many sites pointing to them. Because of this, people are already starting to create sites solely for the purpose of pointing to other sites.

Another approach to improving search engine ranking is to submit many pages in a site, or even off a site to a search engine. All of these entry pages, often called doorway pages , point to important content within your site. Unfortunately for many users, doorway pages are more like decoy pages, as they can be loaded with false content to attract the visitor and nearly always eventually deposit the user at a page they didn't really want to see. The problem with search engine promotion is that the distance from simple logical keyword loading and various tricks is a short one-particularly if page authors are obsessed with top-ten ranking.

Tricky Business

The tricks employed by search engine specialists are numerous and change all the time. Many ideas are simple add-ons to normal Web design techniques. For example, many page authors rely on invisible pixel shims to force layout. Search engine promoters say, "Why not put alt attributes on these images to improve things?" Imagine this

  <img src="pixel.gif" alt="robot butler robot butler robot butler" />  

all over your page. Then pity the user who pauses on top of one of these invisible pixels only to have a Tooltip pop up screaming about whatever the page is promoting. Spamming pages with invisible text, small text, and multiple images, or just loading the <meta> or <title> tags, are not the most sophisticated tricks, but they seem to work, at least for some sites.

Other tricks include the infamous "bait and switch," in which a special search engine page is created and then posted to a search engine. Once the ranking is high, the bait page is replaced with a real page built for users. A more complicated version of this is dubbed "cloaking." In the "cloaking" scenario, you write a program that senses when a search engine hits the site and "feeds" the engine the page it wants to see. Like a ravenous dog, it gobbles up the information with no idea it just ate the equivalent of informational pig snouts. As real users hit the site, they aren't served the search engine bait page, but instead get the real site.

Detecting search engines versus regular users isn't terribly difficult because the engines identify themselves and come from consistent IP addresses. In reality, "site cloaking" is just a modified form of browser detection. Search engines can do little to combat this approach because they would have to consider eliminating dynamically built pages-which is impossible given their growing importance-or not informing sites that they are search engines while indexing. A few search engines have already begun to provide a link to a page that shows what was indexed so users can determine if they are being shown something different than what a search engine indexed. Furthermore, if search engines do detect cloaking, it is a pretty sure fire way to get banned from being indexed.

The problem with all the search engine promotion business is that it tempts the page author to stop building pages for users and start building them for search engines. This is just another form of designing more for your own needs than for your users. One of the most interesting aspects about search engines is that many large organizations don't rely greatly on them for driving traffic. In fact, for many corporations, unless you type their name in directly, you'll be hard-pressed to find them in a search engine under generic keywords. Of course, if you try to look for them, they do come up. Despite what appears to be a major oversight by these organizations, these sites continue to get huge amounts of traffic. According to studies such as the GVU Internet Survey, people type in URLs directly quite often. How are they finding out about sites? Always remember that search engines aren't the only way to drive traffic and even when users rely on them significantly they will tend to trust the brands and organizations they are familiar with over the ones they are not.

---