Hack15.Improve Data Accuracy with Cookies

Hack 15. Improve Data Accuracy with Cookies

Cookies are a fundamental component in any web measurement solution and they come in several flavors. Because of the explosion in use of anti-spyware applications, you need to understand how cookies are commonly used and make an active decision about how they'll be used on your site.

In theory, one of the simplest ways to improve the accuracy of your analytics data is to use cookies as a data tracking mechanism. A cookie is a piece of information that is stored by your web browser and comes in two minor variations: session cookies and persistent cookies. Session cookies last only as long as the visitor is on your site and are deleted after the user closes her web browser or after some period of inactivity (typically 30 minutes [Hack #1]). Persistent cookies last beyond a single visit and have an expiration date some time in the future. Session and persistent cookies use identical technology but differ in how they're treated by security and privacy applications like the Platform for Privacy Preferences (P3P) [Hack #26].

2.3.1. Session Cookies for Short-Term Accuracy

Session cookies are typically set by web server applications and allow your analytics solution to group interactions with your web server at the visit level. With logfile-based solutions, you should enable your web server to set session cookies and configure your analytics solution to track these session cookies in your logs. Tag-based solutions [Hack #3] will typically set their own session cookies, so you should get this functionality for free. Once session cookie tracking is enabled, you can start to analyze a number of useful visit level statistics, including total visits, pages per session, entry pages, exit pages, and clickstream data.

2.3.2. Persistent Cookies for Long-Term Measurement

Persistent cookies allow your analytics solution to track visitor behavior across multiple visits, which is absolutely critical in web site measurement. This is useful when you are trying to understand customer retention information, such as repeat visit and purchase activity, or understand the frequency of visit and lifetime value [Hack #84] of your visitors and customers. Persistent cookies come in two flavors: first party and third party, depending on how they're set.

2.3.2.1 First-party cookies.

Cookies set by the business from their own web servers and domains are called first-party cookies. For example, if eBay is setting their own tracking cookies from the ebay.com domain, these cookies are said to be set by the "first party." While not exclusively so, the use of first-party cookies is most common in software-based web measurement solutions that rely on web server logfiles for data.

2.3.2.2 Third-party cookies.

Any cookie set from a domain other than that of the accessed web site are third-party cookies. For example, if Disney were using a WebSideStory tracking domain, ehg-disney.hitbox.com on their www.disney.com web site, the WebSideStory cookie is a third-party cookie. More and more, the hosted application providers are moving to provide a first party cookie option to their customers, a reflection of accuracy issues associated with third-party cookies.

2.3.2.3 When a first-party cookie is a third-party cookie.

The determination about whether a cookie is first or third party is made by comparing the domain for the web page being served to the cookie domain. Some large organizations have several "brand" web sites that need to be measured as if they were a single site, so sometimes the lines between first-party and third-party cookies become blurred. Consider the following scenarios:

• If Microsoft sent data to and got a cookie from SageMetrics sageanalyst.net domain from www.microsoft.com, this cookie is definitely a third-party cookie. Microsoft.com domain visitors are making third-party requests to SageMetrics.net domain, from which a third-party cookie is set. This practice is somewhat deceptive, because the customer did not explicitly ask to send data to SageMetrics.

• If Microsoft sent data to and got a cookie from SageMetrics by modifying DNS and creating a sageanalyst.microsoft.com domain, when the data is sent from www.microsoft.com, this cookie is said to be a firstparty cookie set from microsoft.com domain. Microsoft owns and controls the domain and contracts management of the subdomain to Sage-Metrics as its vendor.

• If Microsoft sent data to and got a cookie from SageMetrics by modifying DNS and creating a sageanalyst.microsoft.com domain, when the data is sent from www.msn.com to sageanalyst.microsoft.com, this cookie is said to be a third-party cookie. The cookie is sent to the microsoft.com domain from the msn.com servers (different servers in a different domain.)

The third example is usually limited to very large organizations with multiple web sites trying to gather data across multiple domains.

2.3.3. Improve Accuracy with Persistent Cookies

While there are alternatives to cookies [Hack #17], they remain the most popular strategy for determining the uniqueness and visitation history in web site measurement. Unfortunately, because of the proliferation of anti-spyware applications, many designed to remove or disallow third-party cookies, as well as legislation around the world designed to limit the use of "information gathering applications" (which a cookie arguably is, depending on your perspective), the use of cookies is increasingly at risk. Data recently published by JupiterResearch indicates that third-party cookies may be inaccurate as much as 28 percent of the time, and that all cookies might fail as much as 15 percent of the time.

If you are using cookies to track your online visitorsand it is very likely you arethe following recommendations will help improve accuracy.

2.3.3.1 Use true first-party cookies whenever possible.

Because anti-spyware applications are much less likely to block or delete tracking cookies from non-tracking domains, whenever possible, set cookies from your own servers. The logic here is that visitors and anti-spyware applications are less likely to delete your cookies than those of a third party they don't know (or one that is known to be a tracking domain). Most log-based tracking solutions offer some type of web server add-on that will handle this for you, as will any tag-based solution that runs in your data center (as opposed to an external location).

2.3.3.2 Use DNS to make it look like you're using first-party cookies.

One trick that a number of vendors use is mapping a first-party tracking domain to an externally located IP address to give the appearance of a first-party cookie, when in fact the data is flowing to a third party. A simple change in the DNS servers, stating that requests for tracking.yourdomain.com should be sent along to your external tracking servers and telling those servers what to do with this traffic when it arrives, is all that is required.

2.3.3.3 Make sure your privacy and P3P policies accurately reflect what you're doing.

People block cookies because they're concerned about their privacy and security online, period. The best way to mitigate this problem and help your visitors understand why you're using cookies is to tell them via your privacy policy [Hack #26] and P3P compact policy [Hack #27]. If you're clear with your visitors, hopefully they will trust you and allow cookies to be set from your site.

The most important thing you need to know about cookies is to not take them for granted. Data suggests that the number of Internet users who are deleting and disallowing cookies is still increasing and that as many as 15 percent of your web visitors are blocking all cookies as a rule. By working closely with your measurement vendor to implement the "right" data collection strategy, you will increase data accuracy and the overall quality of your web data.

Xavier Casanova and Eric T. Peterson