Nothing is more important to search marketing success than tracking your progress. If you regularly track your key measurements, they will prompt you to do everything else required to succeed at search marketing. Your measurements help you identify problems when they occur, causing you to take corrective action. But metrics have other value, too.
Strategic use of disappointing metrics can motivate your extended search team to do what you want them to, and motivate your executives to approve more campaigns to correct the problems. Sharing successful metrics can prove the value of search marketing to accomplish the same purposes. So, whether the measurements are good news or bad news, you can use them to make your overall pointthat we need to improve our search marketing.
Whether your new central search team tracks all your metrics for you or you can talk your existing Web metrics team into doing them, the tasks to perform are the same:
Search metrics help you to identify and diagnose problems as well as to prove success. We begin by putting your content under the microscope.
Assess Your Site's Content
For organic search, your site's content spells the difference between success and failure, as you learned in Chapter 12. Even though you have done the hard work of optimizing your Web pages, you must check your content frequently to ensure it does not deteriorate.
For paid search, you have a lot less to do. You are advised to check your paid search landing pages to make sure that their URLs have not changed. For both paid and organic search, your landing page copy affect conversion rates, but no easy way exists to check that.
Measuring the search friendliness of your content helps you identify problems and correct themproblems in content tagging, keyword prominence and density, links to or from your pages, and inclusion in the search indexes. You can decide that it is too much work, and that you would like to just react to problems as they occur. We advise against that approachyou will never maximize your conversions by just fixing what is wrong. Regular content monitoring will avoid problems and will improve content that is already successful. If you are ready to take a proactive approach, let's tackle the metrics for content tagging.
Content Tagging Measurements
Content tagging metrics report a wide variety of problems in the coding of your HTML pages. Here are some content standards that a content reporting tool can police, when run on a regular basis:
A content reporter can detect all of these problems and more. At least monthly, you should evaluate each page on your site that falls within your search marketing program's scope, checking each page for compliance with your content standards. Your content reporter should analyze each page automatically and generate two reports:
In Snap's case, each business area sells a separate product line, but in your case business areas could be divisions, countries, or something elseyour business areas should correspond to your organization's structure. Snap kept its scorecard simple, as shown in Table 15-1, with just five items to check. Later in this chapter, we explain what you do with these statistics.
Depending on your budget and your team's ability to do some programming, you can either purchase a content reporter or build one yourself. If you can afford the cost, Watchfire (www.watchfire.com) offers a specialized spider that crawls even the largest site and can find all sorts of searchability problems, even enabling you to customize the rules it uses for checking. It can roll everything up into a set of scorecards that you can use throughout your organization. For those on a budget, WebCEO (www.webceo.com) crawls your site and reports on your content for less than $300.
Keyword Prominence and Density Measurements
In Chapter 11, "Choose Your Target Keywords," you decided which pages would serve as organic search landing pages for each keyword in your campaign. You need to regularly inspect those pages even more carefully than the rest of the pages in the scope of your search marketing program. You worked hard on those pages when you started each campaign. Don't let them fall into disrepair.
You recall that a critical part of optimizing each landing page is improving keyword prominence and densitymaking sure that the keywords that page is designed to attract are present in titles and other prominent places, and sprinkled throughout the page. Your keywords should be found early and often.
It is worthwhile to regularly spot-check your search landing pages, using an auditing program as we did in Chapter 11. If you have only a few campaigns totaling several dozen landing pages, you should scrutinize every page once a month. If you have hundreds or thousands of landing pagestoo many to check each one monthlyyou might need to sample some of them each month. Regardless, regular checks of your landing pages will provide early warning for problemsyou will be alerted before you see marked drop-offs in rankings or search referrals. Obviously, if you do see a drop-off in organic search rankings or referrals for a particular landing page, you need to look at that page immediately. And you will see fluctuations in rankings, possibly because your competitors have improved their pages or their offerings. You might see changes because the search engines change their ranking algorithms. Following an ongoing process of checking your organic search landing pages will allow you to maintain and improve your content.
Some auditing programs, such as WebCEO (www.webceo.com, less than $300) and Site Content Analyzer (www.sitecontentanalyzer.com, less than $100) can automatically crawl a set of pages on your site and generate reports, taking some of the drudgery away, but you will still need to stare down each report to see how you are doing.
Links to and from your pages are critical to your search marketing efforts. The earlier you can detect problems with your links, the less impact they will have on your search marketing success.
Let's look briefly at links within your site. Each page on your site should be checked at least monthly for broken links. As you learned in Chapter 10, broken links stop the spider cold, just as they stop your visitors. Most content reporter tools, such as Watchfire, can detect broken links, but specialized broken link detectors such as Xenu can handle this, too. Broken links break the chain of pages that lead to conversion, for both organic and paid search, and must be diagnosed and repaired.
But for organic search marketing, you must also be concerned about broken links to your site. In Chapter 12, "Optimize Your Content," you learned about how important links to your pages are for organic search marketing. Whenever you change URLs on your site, you run the risk of breaking those all-important inbound links.
In Chapter 10, you learned how to redirect your old URL names to your new ones. You learned that the search engines transfer the value of inbound links to your new name when you do that correctly. But no one knows for how long search engines do that. Just as the post office eventually stops forwarding snail mail to a new address, eventually search engines must tire of treating links to your old URL as if they were to your new one.
For all of these reasons, you should regularly check your inbound links. When your URLs change, ensure that the proper redirects are in place, but also contact the important sources of inbound links and let them know they should change the URL in their link. (That contact is also a great opportunity to suggest better anchor text or an updated description in the link, if you want that.) Because you want to focus on the most important links first, you can prioritize them based on Google PageRank, using some of the tools profiled in Chapter 13. As an alternative, you can prioritize based on the referrals you get from each one, which your Web metrics facility can tell you. Whatever way you do it, you want each of those old links changed.
Even if you do not change your URLs, you still want to check inbound links to your organic search landing pages once per monthsearch experts often refer to these links as back links. If you are actively working on link-building campaigns, you should see progress every month. You also want to note any pages for which links are decreasingthey might be candidates for link-building campaigns. But you can track more than the sheer number of links to your site. You can analyze the links to your search landing pages compared to those to direct competitors for that same keyword.
You can decide to measure the efficiency of your link-building campaigns, too. For solicited links, how many requests actually result in links to your site? How can you improve the percentage? You might look at better targeting, improving your e-mail copy, or improving your site to see whether you can attract more links in your link-building campaigns.
Links are important for conversions and for organic search rankings, so check your link measurements each month to ward off problems.
Page Inclusion Measurements
No matter how well crafted your content is, it is useless to a search engine if your pages are not included in the organic search indexes. As you learned in Chapter 10, your pages can be missing from search indexes at any time and for any number of reasons. When you first began your search marketing program, you undoubtedly found a host of problems on your site, and you set to work correcting them. Regularly measuring your progress on getting your pages indexed is what we discuss here.
You probably recall the inclusion ratio metric that we introduced in Chapter 10, which shows the percentage of your Web site's pages that are included in search indexes. Two reasons stand out for why you should be checking your inclusion ratio at least weekly. The first is to track your progress at eliminating spider traps and enabling spider paths. A weekly check of your inclusion ratio can also reveal problems that have cropped up, however, before you have suffered long periods of harm to your search marketing program.
Springing the spider free from the traps that bedevil your Web site can sometimes take months. At times, you eliminate one spider trap just to find that there is another one in the same spot. By checking your inclusion ratio, you can track how correcting each trap (and opening each new path) allows the spider to include more and more pages in its index.
There is no need to exhaustively track inclusion in every search index. Many search marketers check just Yahoo! and Google, but you can check as many as you believe is necessary to monitor your search marketing program. (If you do not recall how to check the number of pages included in search indexes, Table 10-3 in Chapter 10 can jog your memory.)
In Chapter 10, we discussed how tough it is for some search marketers to estimate the number of pages on their sites. If this is an issue for your site, you can still easily monitor your inclusion ratio. All you need to do is choose a single, easy-to-calculate number that you can derive each weekthe total number of documents in your three content management systems, for example. Although this total probably does not accurately estimate the number of Web pages on your site, it will correctly reveal trends and problems if you calculate it the same way each time.
Here's how it works. Each week, as you add up the documents across your three content management systems, the number of documents will fluctuate, as content is added or deleted. If a new product line is announced, for example, you might find 209 new documents across your 3 content management systems, although just 193 new pages were published to your site. If your site is crawled weekly, you should expect to see a couple of hundred new pages reflected in the search indexes the following week. If they are not, it makes sense to inspect that new site for spider traps or missing spider paths. Note that it is not important whether the number of documents truly correlates to the number of pages on the site for this method to work. As long as you measure consistently, you will spot problems and trends from week to week.
Check Your Search Rankings
After verifying your content is search-friendly and that it is stored in the major search indexes, now you are ready to check your search rankings. Just because your pages ranked well when you did the initial campaign work does not mean that those high rankings will last forever. Regular measurement of search rankings is a key operational function.
Every campaign in your search marketing program has a list of targeted keywordsthe list you learned to choose in Chapter 11. To measure the success of each campaign, you need to check the search rankings for each keyword in that campaign at least once a month. (Most search marketers check their highest-priority terms weekly.)
You know by now that high search rankings are important, because the higher your ranking for a keyword, the more clickthroughs you are likely to get. A #1 result sometimes has a 70 percent clickthrough rate, whereas results that fall below #10 can draw clicks less than 1 percent of the time.
Measuring search rankings on a regular basis is also important, both to show success and to identify problems when they occur. Publicizing high rankings within your organization helps convince people that search marketing can work, and motivates marketers whose campaigns have not yet succeeded. People like hearing about search rankings, possibly because everyone can understand what it means to go from #19 to #8, and maybe because it is thrilling to pass your competitors. As your search campaigns are executed, you should see steady improvements in the keywords targeted for those campaigns.
Conversely, regular monitoring can uncover a sudden drop in rankings, enabling your team to take quick action to correct what is wrong before losing any more search referrals. Although you would eventually spot a ranking problem in your other metrics (as referrals and conversions drop), you can see the rankings drop sooner. That speed allows you to fix the problem more quickly so that your campaign gets back on track before costing you any more precious conversions than necessary.
The problem with checking your search rankings, however, is that it is time-consuming for you (and time-consuming for the search engines themselves, which we discuss in a minute). Search marketing campaigns, once started, do not stop. As long as you still have a need to attract visitors, the campaign continues. By the time you have cranked up several campaigns, you have developed a long list of keywords for which to track rankings.
Consider our fictitious company, Snap Electronics, which built on early success to have four search marketing campaigns running after just a few months. Each campaign averaged 25 keywords, totaling 100 keywords in all. Because Snap targeted Google, Yahoo!, MSN, AOL, and Ask Jeeves, it required 500 manual queries (5 search engines multiplied by 100 queries) to check their search rankings. And, for each of these queries, someone had to manually check the results, sometimes paging forward to multiple search results screens, to find what the highest ranking was for a page from Snap's Web site. This took days to do each time.
You should know that automatic rank checking is considered a necessary evil by search engines, because it consumes precious server resources for queries run by software programsnot by people who might click through on an ad to make the search engines some money. So, not only don't the search engines make money from rank checking, it costs them money to host the extra servers to run all of those extra queries. Google has been the most outspoken about this issue and has actually banned Web sites that check many of their keyword rankings frequentlydaily or even multiple times a day.
Google does understand that you and other search marketers need to check your rankings, but you are expected to use a rank-checking program that supports the Google application program interface, or Google API. (Table 15-2 above shows which programs do so, but you can also write your own program.) The API limits your rank-checking activity to 1,000 queries each day, which is sufficient for most search marketers.
One twist on search rank checking is competitor rankings. Sometimes you can motivate the extended search team to take the actions you need by showing where competitors rank, so it can be a useful metric to track regularly. There is no need to check competitor rankings on all of your keywordsmost of your high-priority keywords do not apply to your competitors because they contain your brand name. When Snap Electronics decided to look at competitor rankings, only digital cameras and easy digital camera were selected, because those keywords were critical to Snap's branding goals. Snap decided that tracking competitor metrics for the other keywords was not worth the effort. If you decide to track competitor rankings, restrict the keywords you select to your critical branding or conversion battles.
Whether or not you track competitors' rankings, you should create a rankings scorecard for every keyword (or at least the high-priority ones). Your search rankings scorecard might average the rankings for each keyword across all of your targeted search engines. So Snap Electronics would average its rankings for digital cameras across all five targeted search engines, as shown in Table 15-3. Whenever a Snap page was not found in the top 100 results for a particular keyword, it was assigned a rank of 100.
If you would like to be more scientific about it, you could use a weighted average that gives Yahoo! and Google more emphasis based on their share of searches, but you might prefer to use a simple formula that is easier to calculate. You can check out the rankings for each keyword, for each campaign, even for your overall search marketing program, tracking your progress over time. Later in this chapter, we show you another use for this scorecard.
Another example of a search ranking scorecard is shown in Table 15-4, where we can track the positions of specific search landing pages. (For space reasons, we have eliminated the leading www.snapelectronics.com from the URLs in the table.) As you have already learned, driving visitors to key pages with specific messaging and navigation helps increase conversions. The landing page scorecard is a good way to motivate content owners and business unit managers to make the necessary changes to the site for increased organic search success.
Search rankings are an important measurement because high rankings lead to more traffic to your site, in the form of search referrals. Next we measure those referrals directly.
Monitor Search Referrals
Search rankings are important, but high rankings are useless unless they lead to higher search referrals, a metric introduced in Chapter 7, "Measure Your Search Marketing Success." As you recall, search referrals are the visitors who come to your site from a search engine. We count overall search referralsthe number of visitors who come from search to any page within your search marketing program's scopebut we also slice and dice referrals to learn more. We examine the significance of referrals from each search engine, and you will learn to zoom in on your individual search campaigns to check referrals by keyword. And (say it with me now), we do it regularly. Checking referrals as a campaign first unfolds is exciting, but you need to continue to check them long after your enthusiasm has waned.
Overall Search Referral Measurements
The first way we track search referrals is across the scope of your entire search marketing program, from both organic and paid search.
Almost any Web metrics facility can report on search referrals, but you might need to customize your reports so that they fit your search marketing program. For example, Snap Electronics chose just its U.S. Web site as the scope for its search marketing program, so it wanted to look at just the search referrals to its U.S. site, ignoring those to the other country Web sites at snapelectronics.comthose referrals will become important when Snap increases the scope of its program. Snap customized the referral reporting by its Web metrics facility to examine its U.S. site only. Table 15-5 shows Snap's results for the first three months of its search marketing program.
As the table shows, search referrals increased markedly over the period, as the digital cameras campaign (and later others) made a dramatic difference. You will notice that search referrals did not increase each month, but that a decrease in referrals for the entire site hid the improvement. That is why we also calculate the search share, the percentage of search referrals compared to the total referrals. This measure reveals that search marketing was still improving even during a downturn in overall traffic to the site.
If your site has seasonal ups and downs, you probably already have ways of interpreting your statistics to compensate. Some organizations use rolling averages, others use "year-over-year" comparisons, whereas your company might do something else entirely. Whatever method you use to smooth out seasonality should work just as well for search referrals as it does for everything else.
Search Referral Measurements by Search Engine
Although most organic search marketing work is not directed at any particular search engine, sometimes your efforts work better in one engine than in another. Segmenting search referrals by search engine can show you how your success differs in each engine. Moreover, for paid placement, your efforts are highly targeted to specific paid search vendors working with specific engines, so analyzing each engine separately is required.
Examining referrals by search engine will help you track progress as your campaigns take shape, but it will also aid in identifying problem areas that might affect one search engine more than the others, or one search engine before the others. A sudden drop in traffic from a single search engine might by caused by a decrease in rankings or fewer pages included in the index.
As before, most Web metrics facilities can separate referrals by search engine, but they might need some tweaking on your part to accurately categorize certain referrals as being from the correct search engine. For example, Google contextual search referrals are usually shown as coming from googlesyndication.com no matter what site they were actually from. Know that sometimes you will get clicks with no referrer at alljust remember that counting most of the traffic is better than nothing. Table 15-6 shows how Snap Electronics measured search referrals by each of the five search engines targeted by its search marketing program.
Beyond your need to track referrals to make decisions about your campaigns, you also need to track paid referrals to audit your per-click fees. But don't expect your referral reports to match the bills from paid search vendors, for several reasons:
Given all of these discrepancies, you might wonder why you should bother auditing your bills at all, but it is important. You should expect your referral reports to show as much as 10 percent variance from vendor ledgers. However, you should look for trends. If counts have varied by 3 percent to 4 percent for months, but this week they are 7 percent off, you should investigate.
Search Referral Measurements by Keyword
High search rankings typically create high search referrals, but occasionally a top search result does not garner heavy clickthrough by searchers. You will detect those situations by carefully tracking your search referrals by keyword for every search marketing campaign. Each month, you can calculate your referrals for each keyword, both for organic and paid search. Table 15-7 shows a sample report from Snap Electronics for July.
As the months go by, you can examine the referral trends to see how your keyword referrals are trending. Snap Electronics used the simple form shown in Table 15-8 to track its monthly search referrals.
Tracking referrals is important, but you must keep your focus on the real prize: conversions.
Calculate Web Conversions from Search
None of the measurements we have discussed will make more of a splash than counting conversions. We have emphasized all along that search marketing is only useful insofar as it furthers your Web site's basic goalscoaxing more searchers to visit and more visitors to convert. You have chosen what events on your Web site constitute conversions. Regardless of what they are, the value of search engine marketing is to produce more visitor conversions. Now you need to count them, and count them regularly. Strong early results count for a lot less if they are allowed to atrophy from inattention.
Why would your conversions change over time? Because everything that causes conversions changes over time:
You might be wondering exactly what an acceptable conversion rate would be. You might be tempted to research averages for your industry, but that is the right answer for you only if you are an average business in your industryaverage in every way. You know that across your product line, you have wide swings in conversion rates that are all rolled up into one homogenous average across your business. Some products convert at 4 percent and others are under 1 percent, perhaps. So what does it tell you that your average conversion rate is 2 percent? Not much.
Similarly, getting an average conversion rate across your whole industry is even less helpful. The only way to assess your success is to compare your present against your past. You should see that you are improving over timenot across the board perhaps, but overall. You should see that the conversions for each product tend to go up over time.
Now that you are convinced that you need to track conversions, we have bad news. Your Web metrics facilities might or might not enable you to accurately track the behavior of visitors who began their visits through a search referral. You should consult your local metrics expert to see how your system works, because there might be ways to get your existing measurement system to track conversions for you. Here are the kinds of devices that are most frequently used to track search conversions:
If you can use one of these methods to calculate your Web conversions of searchers, you can show how many referrals you are getting, the number of conversions (and the conversion rate), and the value in business terms of those conversions, just the way we did in Chapter 7. And you can track search conversions at any level of detail: a keyword, a group of keywords, an entire search marketing campaign, or even your overall search marketing program. (With paid placement, you can set up your categories or ad groups to simplify reporting.) So, if you can track that last month your site received 1,000 extra visitors from search for a particular campaign's keywords, and 2 percent (20) of them bought your $500 product, you generated $10,000 in extra revenue from that search marketing campaign. Table 15-9 shows the revenue scorecard for Snap Electronics.
When you can pinpoint the value that precisely, you can learn a lot. Suppose you find that 1 percent of visitors who come to the site by other means convert, whereas searchers convert at a 2 percent rate? That might justify a shift in marketing resources to search. If you find the search conversion rate lower, perhaps you need to examine whether there are ways of raising conversion, or whether there might be other keywords that convert at a higher rate. Remember that it is crucial that you analyze your conversion metrics at this detailed level to make good decisions. Looking at averages across your whole program is fine for a snapshot of progress but does not help you decide which organic search landing page to tweak or which paid placement keyword should be dropped.
Some companies, especially large companies, have complex infrastructure issues that make tracking search conversions difficult or impossible. If you find yourself in this position, it's not the end of the world. You can still estimate your business impact using the same method used in Chapter 7 to forecast our campaign's impact. Merely take the number of incremental visitors being referred from search for your campaign's keywords and multiply by your site's conversion rate. Using the example above, if we assume those same 1,000 extra visitors converted at a rate of 1 percent (the site-wide rate), we would estimate that 10 of them purchased the $500 product for $5,000 in incremental revenue from our campaign. It is not exact, but it is far better to estimate the value than to be silent about it.
If you can accurately tie Web conversions to the search referrals, you can track your conversions as granularly as your referrals. You can examine the revenue driven by each keyword in a campaign, as well as check conversion rates across your program for search referrals from individual search engines. This kind of analysis can help you gradually improve your campaigns, as you learn what works and what does not.
Review Your Measurements with Others
Well, we sure calculated a big pile of statistics in this chapter. Let's see what you can do with them.
As noted throughout, there are two main reasons to compile these statistics: to prove the value of search marketing and to identify problems that require action. When these metrics show your success, we suspect you will know what to do. Have a party, review the numbers with executives, and bring them up every time you want to do more. You will get the attention you need when things go wellcongratulations.
But most people do not know what else they can do with these measurementscompel changes in behavior. That is what we talk about here. You can follow a simple a step-by-step process to change organizational behavior:
For some of these metrics, you need to see where you stand to decide what a good benchmark should be. If your site is riddled with "meta refresh" redirects on 15 percent of your pages, for example, you might choose a goal for 90 percent refresh-free pages earning a "yellow" score with 95 percent earning "green." On the other hand, if 95 percent of your pages are already clean, you might want to shoot for 97 percent and 99 percent.
For search rankings, you might decide that any keyword that averages a ranking below #30 should be scored "red," and that anything that drops out of the top ten is "yellow," because that is simple and everyone understands it. If you want to be more precise, you could have higher benchmarks for queries with your brand name in it (such as "snap digital camera") because it is so much easier to attain high rankings for those keywords.
For organizations that focus on organic search, benchmarking content and rankings might be enough. You can use those metrics to drive the actions you want to improve content tagging, keyword density, page inclusion, and any other behavior needed. If you have problems driving Web conversions on your site, or you have significant paid placement activity, you will want to focus on referrals and conversions, too. You can develop your own benchmarks for those metrics and assess each business area against them.
One study showed the average Web site gets about 13 percent of all referrals from search engines. So, for every 100 visits to your site, 13 percent of them are searchers clicking a search result; the rest are following links, using bookmarks, and directly entering the URL. Remember, 13 percent is the average. Well-known Web sites might have far more visitors that remember the URL or have bookmarked the site from a previous visit. Don't fall into the Lake Wobegon trap of needing all sites to be "above average"take the baseline for your site and try to keep improving.
If you can track conversions, you have the best information of all. You can use conversion data to address pages that attract high search referrals but do not lead to high conversions. Perhaps those pages are at fault, or perhaps other pages between the search landing pages and the conversion page are the problem. Regardless, you know there is something wrong, and you can work on all of those pages and track improvement. Moreover, tracking conversions allows you to test which keywords convert at the highest rates. It is those keywords that should consume your organic optimization resources and command your highest paid placement bids. Management by conversions focuses your resources at the points of greatest returns.
This step-by-step approach using scorecards can prove quite successful when you stick to it, gaining attention for your search marketing program and forcing organizations to pay attention to the actions required. And year after year, it is the gift that keeps on giving, because every time your business areas reach the "green" benchmarks, you can raise them higher. Each year, your content gets a little cleaner and your rankings a little higher, because these metrics become part of everyone's job.
You can do even more with these metrics, however. In Chapter 9, we made a number of predictions to convince senior management and the bean counters that we would spend the money well. The metrics we have demonstrated should reveal whether your projections are coming true. Creating a simple matrix to share with key executives will keep them happy that they have invested wisely.