Hack93.(Don t) Benchmark Your Site

Hack 93. (Don't) Benchmark Your Site

Except in very controlled situations, attempting to compare your conversion rate information to published data almost always creates more problems than it solves and should be avoided.

Many business people seem to have an intense desire to benchmark themselves against market leaders, even if they aren't selling to the same market. Especially for a relatively complex key performance indicator like conversion rate, one dependent on so many variables (for example, marketing strategy and spend, product assortment and pricing, usability of site, or audience), making any comparison except in the most controlled situations is an exercise in futility. I'll explain two situations where you can compare your data to that gathered from another site in a minute, but first here are a few reasons to avoid benchmarking yourself.

First, unless you know exactly where the data is coming from, you might be comparing apples to bowling balls. Given the differences in accuracy observed when comparing different data sources, are you sure you want to make business decisions if you're not sure about the accuracy of someone else's data? If my key comparison is "page views per visit," but I have no idea whether a competitor is rigorous in excluding robot and spider activity [Hack #23] from their data set or whether they use accepted standards to define a visit [Hack #1], how can I be sure their number is any good? The easy answer is that you can't, so comparisons may be misleading.

Second, even if you know the data sources are the same, you still might be comparing apples and oranges. Conversion rate is the most commonly compared measurementat least it's the one most people ask about most oftenbut there are a handful of definitions for "conversion rate" [Hack #82]. Depending on whether you're using "order conversion rate" or "buyer conversion rate" and which timeframe you're looking at, a number that sounds like an apple can easily be an orange or worse.

Finally, even if the numbers are the same, you can rarely make anything more than a "gee whiz" comparison. Unless you're able to get data about companies in your vertical market (for example, online sales) and unless you're able to get data about companies in your specific sub-vertical (for example, online booksellers), and unless you're able to get data for sites that have roughly the same average order value (for example, between $25 and $50), and unless you know pretty well how these companies are attempting to attract new visitors, and unless…you get the picture. While you may be able to make a "valid" comparison without a tremendous amount of information, it is extremely unlikely that the comparison will be useful in any meaningful sense.

7.4.1. The Foolproof Benchmarking Hack for Anyone

The easiest way to benchmark your site in a meaningful way, one guaranteed to generate actionable information that can be used in future comparisons, is to benchmark against your own site and track your own success.

If you establish key performance indicators that are both significant and useful to your business, then you can easily compare those numbers month over month and determine whether the time and effort you've allocated is money well spent. Well-defined key performance indicators [Hack #94], by definition, force you to make comparisons over time, allowing you to view your metrics in the right context.

If in June your order conversion rate is 2 percent, but in July you spend a million dollars in time and effort to improve your site's usability and the quality of your advertising reach, you had better hope that your July conversion rate is better than 2 percent! If your AOV is $35 and you dedicate hundreds of hours to improve your ability to cross-sell and up-sell visitors in the purchase process, you hope that your AOV will go up.

It's all about effort versus improvement and the continuous improvement process [Hack #2]; compare yourself to yourself, and you'll always be making a valid and useful comparison.

7.4.2. The Only Way to Benchmark Against External Data

If you absolutely must have benchmark data for competitive purposes, there are a few sources for data. Unfortunately, these strategies are vendor dependent, but if your need to benchmark is so overpowering and you're not satisfied with benchmarking against yourself, you may want to give these vendors a look.

7.4.2.1 Fireclick Index (http://www.fireclick.com).

While the page describing the application is chock-full of hyperbole ("The most valuable report ever produced for online businesses"), the Fireclick Index (Figure 7-3) was the web measurement industry's first truly useful benchmarking tool.

Figure 7-3. Benchmarking report from the Fireclick Index

By allowing customers to compare meaningful metrics like order size, first time and returning visitor conversion rate, and shopping cart abandonment rate within a handful a handful of verticals and industry segments (such as fashion and apparel, electronics, home and furnishings, and catalog), Fireclick mitigates the problems described above and delivers at least nominally useful data. More information about the Fireclick Index is available at http://www.fireclick.com/solutions/fireclick_index.html.

7.4.2.2 Coremetrics LIVEmark (http://www.coremetrics.com).

Released in early 2005, Coremetrics introduced LIVEmark to their customers as a focused strategy for comparing data to vertical and sub-vertical segments. By enforcing implementation standards within participating customers, Coremetrics is able to provide a great deal of meaningful benchmark information over traffic, transactions, on-site search, and marketing campaigns (Figure 7-4).

The applications from Fireclick and Coremetrics both provide carefully controlled environments for making comparisons. Both vendors go so far as to double-check implementations for participating web sites to ensure data accuracy and apples-to-apples comparisons.

Figure 7-4. Benchmarking data from Coremetrics LIVEmark