Hack41.Measure Email Marketing

Hack 41. Measure Email Marketing

Email marketing is among the most popular methods for customer acquisition and retention, and a key factor driving conversion for many companies. Fortunately for you, email provides a plethora of interesting, easily measured data.

Email is one of the most important methods you have to communicate with your customers, but one often neglected nearly completely by the people sending the message. Regardless of whether you're prospecting or communicating valuable information to existing customers, the volume of email most people receive each day prevents us from spending enough time with any one message, unless the message is well timed and well crafted. Fortunately there are a number of things that can be measured that, when combined with fundamental marketing measurements, yield an incredibly powerful strategy for testing and measuring your email marketing efforts.

3.6.1. Things That Can Be Measured

By itself, email has a limited number of metrics you can track. Measuring the resulting web site visits, online and offline interaction, and purchases tells the whole story. In addition to total sends, the number of email addresses to which you send your message (and perhaps the most common email metric), there are a handful of other metrics you need to understand.

3.6.1.1 Hard bounces.

A hard bounce is a message deemed undeliverable, usually because the address was mistyped or has been canceled. It's also likely to be an address that was entered by somebody who wanted to access content that required registration but didn't want to be in your database.

3.6.1.2 Tracked opens.

Tracked opens are the number of recipients who opened the message using an HTML-capable email client. The inclusion of a nonresident graphic (a web bug [Hack #29]) or script can trigger a server call, indicating that the message has not only been received, but viewed.

With the proliferation of spam, most email applications now reject scripts and nonresident images and warn recipients that something odd was included in the email. The use of scripts in email is a bad idea.

"Unique" opens help eliminate the confusion caused when one person opens the same message multiple times. This can be accomplished by relating the name of the called element to the email address.

3.6.1.3 Tracked click-through.

Tracked click-through is the number of responses measured from an email message. This number may be greater than the number of opens or recipients if individuals click multiple times or if there are multiple links in your message.

3.6.1.4 Unsubscribes.

The number of unsubscribes tell you how many people no longer want your messages. When measuring customer satisfaction, it is common to assume that one complaining customer might represent many unhappy customers who couldn't bother to complain. Given how easy it is to filter messages or simply delete them, unsubscribes are significant events.

Taken together, these metrics can tell you a lot about the quality of the email you're sending and provide the basis for testing your customer lists, the offers you make, and the way you represent those offers. Keep in mind that the aforementioned metrics are usually found in your email delivery and list management application. Once visitors successfully receive, read, and respond to your email by clicking to your web site, then the fun begins!

3.6.2. The Fundamental Email Response Metrics

Measuring the effectiveness of anything on the Internet begins with defining success. What is the desired outcome? Start there or you have no chance of success at all. A handful of reasonable measurements for success exist, including response rate, conversion rate, and a handful of relative value assignments.

3.6.2.1 Response rate and basic response metrics.

Response rate is very similar to the click-through rate and is often a more practical calculation, since clicks can sometimes be difficult to measure accurately. Your email response rate is simply the number of responses divided by the number of total sends. In addition to measuring your response rate, you should look at the number of visits and unique visitors driven by the email message as a rough indicator of success.

3.6.2.2 Click-through rate.

If you're able to import or input data about the number of total sends, you can use that value as the denominator to calculate your click-through rate. The calculation is simply the number of total clicks divided by the number of total sends. You should calculate click-through rate on a campaign-by-campaign basis and also for individual links contained in the email message, if there are more than one. Note that this calculation depends entirely on your ability to count the number of clicks generated, something that not every email delivery application and delivery strategy allows you to do.

3.6.2.3 Landing page "stickiness".

Getting an email recipient to click is only half the battle. Once you get people to a web page, you then have to get them tokeep moving into your site. Unless your landing page is the ultimate goaland it almost never isyou need to get the recipient (who is now a visitor) to click more deeply into your site. An excellent KPI for keeping track of this on a landing pagebylanding page basis is the "stickiness" calculation: the number of single access page views divided by the number of respondents to the email campaign. A low "stick" from the campaign can mean that your landing page is poor, the message in the email is poorly matched to the landing page, or the page was slow to respond. If you're really ready to measure email, you might want to measure "stickiness" tied to different messages and links in each message, helping you better assess which variations work and which do not.

3.6.2.4 Conversion events and rates.

Email conversion should be measured a variety of ways, looking at success on both a grand and granular scale (Figure 3-5). You should build an email conversion rate, defined as the total number of success events divided by the number of respondents, but you should also look at those conversion events on an individual basis. Put another way, you want to know that your email campaign had a 5 percent conversion rate but you also want to know that 85 percent of the conversions were purchases, 10 percent were downloads of some kind, and 5 percent were newsletter registrations.

You may also want to calculate a "relative respondent interest rate"the percentage of email respondents who view more than five pages or reach a specific milestone page as a result of your messagethus relaxing the definition of "conversion" to include a variety of conversion events [Hack #39]. Each rate can be used to define key performance indicators, helping you understand how all your tests translate into improved visitor activity on your site.

3.6.2.5 Value metrics.

When you attach costs and revenues to your email campaign measurements, you can begin to really explore the return on investment for your email marketing. Common KPIs include cost-per-email sent, cost-per-click, cost-per-respondent, cost-per-acquisition, revenue-per-click, revenue-per-respondent, and revenue-per-acquisition. Keep in mind that you don't have to be an online retailer to benefit from these calculationsthere is always a cost associated with creating and sending an email, and you can also always assign some reasonable value to your conversion events.

3.6.3. Things That Should Be Tested

Assuming you're all set to measure the effect of each email you send through to a variety of conversion events, you should explore the following aspects of your email composition and delivery, looking for the "right" mix that speaks best to your particular recipients.

Figure 3-5. The visualization for email marketing email opens through to conversion

3.6.3.1 Format and layout.

By mixing up the format and layout of the messages you send, you can look for differences in how recipients respond to changes in visual composition. You want to explore whether HTML, text, or rich media emails perform better, especially since the cost of producing and sending text-only emails is significantly lower.

3.6.3.2 Length and tone.

Conventional wisdom states that short messages will perform better than long, rambling ones, but this is not always the case. By testing longer and shorter messages, you can experiment with your recipients'response. By varying the tone in your email, you can determine whether people respond to a more direct approach or a softer sell.

3.6.3.3 Date and time of delivery.

Many marketers believe that Monday morning is a bad time to receive email offerstoo much spam to sort through at the start of the week. Some suggest that Friday afternoon is also a bad timepeople are trying to finish up their week, and your message may wait until that dreaded Monday morning before it is seen. But different audiences react differently.

Your subject matter might be just the thing your customers want before they begin their work week or just the thing before they start their weekend. So test time of day, day of week, and even the day of the month to see which helps you best reach your goals.

3.6.3.4 Return email address.

The address from which your messages come may also have an impact on results. Emails could come from [e-mail], [e-mail], or simply [e-mail]. This is especially important to explore so you're sure that recipients are not simply treating your email as unwanted spam.

3.6.3.5 The subject line.

You'd be surprised at how much the subject line makes a difference in whether or not your email is opened. You should experiment with different subjects to determine the most effective for a particular offer and also the most effective' tone for your company. Some messages need loaded, direct subjects ("free if you respond NOW!"). Others 'benefit from a softer approach ("An update for Eric Peterson from Fred at Company.com").

3.6.3.6 The call to action.

The call to action might be an invitation to see more information, enter a contest, or simply "Buy Now." Each different possibility fetches different results, but you need to test how your specific audience responds to different calls to action. Differ the means of response (click versus call versus email) and you increase the number of scenarios you need to test (but you still need to test them all!).

3.6.4. How to Put This Data to Work!

Now that you have the basic metrics, you can use them to measure the effectiveness of your email messages. In fact, you can use them to measure the effectiveness of all the elements of your email messages. When you sit down and compare results for different emails, you should keep the following in mind, going so far as to create a matrix of the "things that can be tested" juxtaposed against your tracked opens, tracked click-through, response, and conversion rates, and any relevant revenue or value metrics (Figure 3-6).

Figure 3-6. Comparison matrix

To really take advantage of everything you've read in this hack, do the following every time you send a new email:

• Make sure you have access to the "things that can be measured," as they are important to making the rest of the calculations.

• Make sure you understand the "fundamental email response metrics" and that your web measurement application is set up to make them.

• Build a matrix like that shown in Figure 3-6, in which you identify which "things that should be tested" you're going to experiment with.

• Partition your potential audience into reasonably sized buckets based on the number of tests you're going to run. If you have enough recipients, plan on testing against only a subset of recipients, and then sending the "winning" message to the bulk of your audience.

• Send the different emails, being careful to tag each so that the "things that can be measured" and fundamental metrics can be partitioned easily for analysis.

• Fill in the appropriate fields in Figure 3-6, making the necessary calculations.

• Analyze carefully and send the balance of emails using the best strategy.

While following this strategy will obviously take more time than just throwing together an email and sending it out, the effort is definitely worth it. There are hundreds of examples of companies using this exact approach, called "digital Darwinism" by some (because only the strongest email strategy will survive), and this approach is widely credited with dramatically improving many companies' email communication efforts.

Jim Sterne and Eric T. Peterson