Search Engine Optimization

While not an exact science due to the vagaries of search engine technologies, SEO is a set of methods that is said to improve a Web site's ranking on a search engine. As we saw in Chapter 2, users almost always turn to search engines with a new problem but almost never read beyond the first page of results, so SEO should be one of the most important elements in your Internet strategy. Having a good listing on the first SERP for all relevant keywords is essential if you want to be found by people who are looking for your kind of products or services.

Users almost always turn to search engines with a new problem but almost never read beyond the first page of results, so SEO should be one of the most important parts of your Internet strategy.

How do you get on the first page in major search engines? There is no sure way, and any consultant who promises to get you there is almost certainly a so-called "black-hat" SEO operator who employs unethical tricks that the search engines frown upon. In fact, search engines do more than frown: They usually remove a site they find guilty of trying to deceive them through black-hat SEO.

While working on this chapter, we serendipitously received this email. Of course, you should never buy anything from spammers. But in this case, there's another piece of basic advice that applies as well: If it sounds too good to be true, it probably is. Anybody who guarantees top placement on a leading search engine is almost certainly a black-hat SEO operator and is likely to get you booted from search engines instead.

The very first guideline for SEO is to refrain from using black-hat methods or hiring black-hat consultants. Only a few situations guarantee top placement: if you are a major company or government agency, or if you have an incredibly famous brand name that no other major organization in the world uses. In this case, you ought to be on the first SERP for your company name or brand name, and if you're not, it's usually because you are making some simple mistake on your Web site. An honorable SEO consultant can easily discover this and tell you how to fix it, and after some time you should see your site get better placement for this one keyword.

For all other query terms, it's not reasonable to expect to jump to the top of the listings in a week. If you are promised magic, you're probably getting deception, and it'll cost you dearly if you get bumped from the search engines.

Black-Hat SEO Tricks

Black-hat consultants attempt to manipulate search engines through techniques like these, all of which are considered foul play by the main search engines:

Cloaking. A Web site can send out different content depending on whether a page view is requested by a user or by the search engine's spider. This is usually called "cloaking" because the site hides its true content (what users see) behind a special page that's only made available to the search engine and is optimized to score well with that engine's software. Special pages that are created purely for search engines are called "doorway pages." It's possible (but not recommended) to have separate doorway pages with nonsense text that's optimized for each of your targeted keywords. These are not shown to users, who only see your "real" pages, which are not shown to search spiders.

Search spam: creating thousands or even millions of pages that have no real content but exist purely for the purpose of getting into search engines for every conceivable keyword. Similarly it's considered spam to create huge numbers of duplicates of the information in the hope that it will show up more often in a search listing.

Link farms. Some black hat operators establish thousands of bogus Web sites that have no purpose other than to link to their client's Web site in an attempt to make these sites seem popular.

Comment spam: posting loads of comments on blogs, discussion forums, and other places on the Web that allow outsiders to add content. These comments invariably include a link to the poster's site in an attempt to drive up its ranking. Of course, any normal commenter will include a link to his or her homepage, but comments turn into spam when they are posted indiscriminatelyoften by an automated script that posts the same comment on thousands of innocent sites.

Domain repurposing: buying a Web site that has a respectable ranking in search engines, throwing out all the content that won the site its placement, and then using it for something completely different (usually pornography).

In 2006 Google banned BMW's main site in Germany for employing cloaking. (The company was reinstated after it repented and removed the offending features.) Search engines don't like cloaking because it tricks them into giving a page different treatment than the one it "deserves" based on the information that's given to regular users. Presumably BMW was moved to employ cloaking because its pages are so heavily graphical that there is not much text to be indexed by search engines. For example, the main headline about the 5er Limousine is an image of the letters and not actual text. The same is true for the category header "Neue Automobile" ("new cars"), even though these words also appear as actual text in the far right column. Two big differences between images of letters and real text: The screen readers employed by vision-impaired users can't read image-based text, and search engines can't index pixels. Search engines only understand actual text, so they are pretty much the world's most influential blind users.

www.bmw.de

SEO has a dubious reputation because of the proliferation of black-hat operators in the early years, but it's really an honorable profession, and today any major SEO consultancy will limit itself to white-hat methods that are completely ethical and well within the bounds of good Internet citizenship.

There are three major classes of "white hat" SEO techniques:

• Linguistic SEO

• Architectural SEO

• Reputation SEO

You should apply all three to your site. Fortunately almost everything you do to improve your ranking in Web-wide search engines will also improve the search engine on your site because it will give the software better data to chew on.

After that, practice patience. After improving your site to make it more findable, you shouldn't expect to get high rankings the next day. It can take months or even years to rise through the ranks. Assuming that you are still in business two years from now, you'll reap plenty of returns from your investment in SEO. Some benefits will certainly come sooner, but very few will come immediately.

Naming Names

If you are in the lucky position of naming a new company or brand, one of your first steps should certainly be to search possible names on all the major search engines. If prominent sites appear as results for a name, you should avoid it, even in cases where it would be legal for you to use it according to old-fashioned trademark law.

Consider the situation for the labor union for senior managers in British civil service, First Division Discounters Association, or FDA. With 13,000 members, all of whom are important people in the government, you would expect the FDA to get good treatment by the search engines. When searching for "FDA" on Google.co.uk, this site comes in second, which is not bad, but why isn't it first for its own name? The first listing is the U.S. American Food and Drug Administration; even in the U.K., the three-letter acronym FDA is used more often to refer to the American government agency than the British labor union. On Google.com, the British FDA comes in at number eight, and all the top seven hits refer to the American FDA in one form or another.

When the British FDA was founded in 1918, probably no one ever expected it to be confused with a major American government agency, but today it is. The British FDA will almost certainly never reach the top spot for its own name; it's a lost cause. But keeping a reasonably high position in the British search engines and a page-one listing in the rest of the world are reasonable goals.

If all else fails, run search ads. Of course, this is expensive and is expected to become even more so each year, so Search advertising is not a great substitute for having a good site that performs well on its own. But Search ads are a good short-term fix because they can usually be up quickly and results are immediate.

Search ads work so well because search engines are the one type of Web site that people visit with the explicit goal of finding someplace else to go. If they see an ad for what they're looking for, they will very likely click it. Advertisers can satisfy a user's immediate needs because they target ads based on users' query terms. (This also explains why ads on search engine homepages don't work: It's impossible to target the ad to the user's current quest until the server knows what that quest is.)

Search ads work so well because search engines are the one type of Web site that people visit with the explicit goal of finding someplace else to go.

Tip: The Beauty of Using Text-Only Ads

Most search engine advertising comes as small boxes with nothing but textno big, colorful, blinking banners. Perhaps because text-only ads are a low-end media format, users actually take them more seriously. Also, having to express a message in a few words probably leads advertisers to be more focused and communicative. The text-only format exposes content-free messages as useless, and thus might save advertisers from the bad instincts they honed on old media.

Although there is no inherent reason that you can't use text for mindless chatterlike "where do you want to go today?"you won't find users clicking on such ads. Ignoring users' immediate needs is certain death on the Web.

Companies that run rich-media ads that disregard users can delude themselves into thinking that they're "promoting the brand"; in reality, they're being ignored because they don't connect with people's needs. After 13 years of watching Web users, we can clearly conclude that they are utterly selfish and live in the moment. Giving users exactly what they want right now is the way to Web success, and having to write small boxes of text encourages advertisers to travel it.

Tip: Tracking the Value of Search Ads

If you run paid advertisements on search engines, you should track the value of visitors who come to your site after clicking on these ads. This is the only way to determine how much to bid for clicks. Say you discover that the conversion rate for visitors from Search ads is one percent and that you make $10 profit from each converting visitor. Then you make ten cents profit from each user who clicks the ads, meaning that you can bid up to nine cents per click and still make a profit.

Avoid the following two classic mistakes when tracking the conversion rate and other return on investment data for search ads:

• Do not confuse clicks on ads that appear on a SERP with clicks on ads from the advertising network run by a search engine. Just because a search company manages a network of sites doesn't mean the sites are search sites. Typically the value of ads on content pages is much lower than the value of SERP ads because readers of content pages are just browsing and may click your ad out of idle interest. You should probably pay much less for non-Search ads; if you analyze them together with the Search ads, your ROI will be messed up.

• Do not use conversion trackers or other Web analytics offered by the search engines. Remember, search engines decide how much to charge you for your advertising, so you don't want them to know how much money you make from each ad. Always use site analytics packages from independent providers who do not have a stake in how much you are charged.

Linguistic SEO

Users typing queries into search engines rarely think to use the kind of words you might employ in your latest slogans or other approved corporatese. That's why writing with the words that your audience knows and uses is very important.

There are two reasons why you should also write plain-spoken headlines that include these keywords. First, search engines tend to assign extra weight to words in page titles and other headlines; second, users scan search listings and often do not read beyond the headlines. If some of the words users type for their queries appear in your headline, they are more likely to pay attention to your listing.

The Top Linguistic SEO Guideline

Speak the user's language, especially if you want to be found in search engines. Users type queries using their own vocabulary.

In the SEO field, finding out what words customers use is usually referred to as "keyword research." There are many Web sites and software services dedicated to helping you perform keyword research. For example, Yahoo! offers a free Keyword Selector Tool that lets you enter a term and get suggestions for related terms. Yahoo! also tells you how frequently its users searched for each of the suggested terms in the last month. This allows you a first cut at prioritizing what keywords to optimize for. (However, you ought to perform your own analysis in addition, since the true importance of keywords for you depends on their use by your target audience.)

Tip: Keyword Overuse Backfires

One of the few times that SEO guidelines conflict with usability guidelines is in the case of keyword density. SEO consultants often want you to stuff your pages with repeated use of targeted keywords even though this can reduce readability. Usually it comes naturally to use an important term several times on a page, but if you find yourself writing convoluted sentences in order to include a keyword once more, it's time to stop. After all, it does no good to attract more searchers to your page if all they do is bounce out immediately because the page makes no sense or is unhelpful.

Google offers its advertising customers a tool that suggests keywords relevant to their products and services. How can Google know your products? Actually, it just reads what you have already written about them on your Web site and then finds related terms that users frequently search for. This screen shot shows the top keywords Google suggested for promoting this book based on the Web page we made to sell the book to the readers of our own site. The list is not perfect (for example, it includes "intranet usability" even though we cover special guidelines for intranet usability elsewhere, but not in this book), but it's a great starting point for selecting the keywords to promote the book.

http://adwords.google.com

The two best methods for discovering your users' terminology are your own search logs and user testing. You can review the log files for the search engine on your site to discover the queries that your customers have entered. Obviously, you are not learning what people who never found your site might have typed, but your search logs are still the best source material for raw queries.

To get a broader idea of users' vocabulary, listen to what they say in user testing. How do they describe their problems and your products? Probably in completely different terms than the ones used by your marketing department. You can also use transcripts from focus groups for this, although you'll run into the problem of groupthink: Once one focus group member has started using a certain word to describe something, other members will too. This is one of the reasons we almost always prefer one-on-one sessions to group sessions for our studies.

Finally you can listen in on customer support calls and read blogs and discussion groups to get more information about users' language. One word of caution is that bloggers and people who participate in online discussion groups tend to be unrepresentative of the customer base at large. But their words are probably a better indication of customers' language than anything you'll hear inside your own company.

Tip: Think Phrases, Not Keywords

Most users only type two or three words into a Search box. The people who are the hottest leads often type the longest queries because they have progressed further down the sales funnel and thus have more specific interests. In any case, it's important to think of SEO beyond the level of individual keywords. Consider multi-word phrases, since they may be better at getting you business.

In thinking about what customer terminology to use on your Web pages, it is important to emphasize the words people use to describe their problem. It's a common mistake when you are selling a solution to focus on describing your features and maybe even your benefits without addressing the underlying problem. But people will often search for information about their current pain point without necessarily knowing what's out there to alleviate it.

Architectural SEO

Architectural SEO has two components: ensuring that your pages can be indexed and having an appropriate linking structure to guide the search spiders to your content.

To have a search engine index your content, that content must be actual text. Pictures of text don't work because a search engine spider is effectively blind to images. Search engines also currently can't index information that's presented in various multimedia formats such as video or audio files. It's fine to include multimedia on your Web site if it otherwise works for users (see Chapter 11 for guidelines on this), but for the information to be indexed by search engines, you must supplement the multimedia files with a textual description.

Search engines discover pages on your Web site by following the links. This means that the links must be encoded in plain HTML that can easily be decoded to tell the spider where to look next. It also means that your URLs must look like they describe regular Web pages instead of dynamically created pages. It doesn't matter whether the page is created dynamically inside your computer. What matters is that the search engine spider thinks that it's a regular page that will be there with the same content the next time a user comes around. For many spiders, this means ignoring anything after a question mark in the URL because this notation usually indicates a database query or other forms of one-time content.

Make sure that there is a clean path of links from your homepage to any page you want indexed in the search engines. The anchor text for the links should describe the most important function of each page on your site. Search engines often place extra emphasis on the text used to describe a link to a page. Having good link text is a primary usability guideline in any case because users often scan pages looking for links and need to see where they go without having to read too much.

Additionally, make sure that there are more links to the main page for a concept than to less important pages since search engines tend to give priority to heavily linked pages. If people search for one of your products and find a minor mention in a press release listed higher than the main product page, your architecture is faulty. One way to correct this is to make the press release link back to the product page. Also, if you have hierarchical information architecture, the link structure should reflect those levels, and provide more links to the higher ones. Using bread-crumbs as auxiliary navigation is one way to achieve this because a breadcrumb trail will always include links to the main pages and use their name as the anchor text.

Reputation SEO

Search engines try to give their users the best results by placing heavy emphasis on the reputation of sites. Sites that are widely considered to be good usually score higher than sites that are considered bad.

To achieve high ranking on the SERPs, you need to make other Web sites link to you. Furthermore links are given more points if they come from sites that themselves have high reputations, so you should particularly emphasize getting links from important sites. The main way to attract links is simply to have a good Web site that's worth linking to. If you have a high-quality product, people will link to it when they discuss it. If you have interesting content or helpful services, people will link to you.

How Search Engines Determine a Site's Reputation

At the time of this writing, the main way search engines determine reputation is by counting the number of incoming links to a site. The theory is that if many other sites consider you trustworthy enough to link to you, you probably are important. In the future, search engines will likely extend beyond this over-reliance on links and include other estimates of a site's importance and reputation. But even then, links will probably remain an important component of reputation ratings.

Of course, other sites can only link to you if you make this possible by having simple URLs for each page and by keeping the same URLs alive forever. If you have linkrot, you lose the search engine ranking that could have come from anybody who linked to an old URL. Conversely, if you keep the same URL year after year for the same concept or service, you will gradually build up its ranking.

In addition to having a great site that others naturally want to link to, you can initiate an active outreach effort to attract links to your site. For example, include links in any press releases you issue, and sites that carry the news will sometimes link to you. Without the URL in the release, you are depending on other sites to discover the appropriate URL, which is extra work and often not done. It's also good to ensure that trade associations and other groups you belong to link to your site from their sites.

You can initiate an active outreach effort to attract links to your site. For example, include links in your press releases and sites that carry the news will sometimes link to you.

Finally, you can submit your site to appropriate online directories such as switchboard.com. Some of these directories charge money to list sites, and such paid directories are rarely as highly rated as those that are editorially determined. However, depending on how highly your site is rated, it may be worth paying for inclusion even if it only gives you a small boost in ratings.