Hack90.Build Your Own Web Measurement Application: Commerce Data

Hack 90. Build Your Own Web Measurement Application: Commerce Data

Fundamental to online retailers successful with web measurement data is the ability to get non-traffic data into the analysis. In this hack, wel show you how to add the value of a transaction to the mix and tie it back to your referring sources.

In this hack, we'll return to our sample program and demonstrate how to extend it to collect additional data. We'll use the example of measuring revenue, but the same method could be used to collect other types of data. Any data you can access using JavaScript can be passed in the page tag, written to the logfile, and then analyzed by the logfile reader. For example, you could report visitors' screen sizes or visitor segments by the same method.

When a visitor to the web site makes a purchase, we want to record that information. We do this by putting additional JavaScript on the page after the transaction is completednormally the "thank you" page on the web site.

In this hack, we'll assume that you have some way of accessing the revenue amount using JavaScript. How to do that is very dependent on the shopping cart software you use, and we won't cover it in any detail here. Some shopping carts pass the amount back to you when the transaction is finished, whereas others seem to try and make it as hard as possible. As a last resort, you can approximate by using a fixed amount, such as the amount of the average transaction.

6.11.1. The Code: Page Tag and Tag Processor

Assuming that you somehow have a revenue amount available to the JavaScript, you must modify your JavaScript page tag to pass the data to the logging script. Previously the page tag looked like this:

 document.write('<img src="/books/4/263/1/html/2//cgi-bin/readtag.pl?url='+escape(document. location)+'&amp;ref='+escape(document.referrer)+'">'); 

To pass the new data, simply modify the code on your "Thank You" page where you confirm the transaction to add the additional field:

 rev=0.00; <!-- Change 0.00 to the actual value of the transaction --> document.write('<img src="/books/4/263/1/html/2//cgi-bin/readtag.pl?url='+escape(document. location)+'&amp;ref='+escape(document.referrer)+'&amp;rev='+rev+'">'); 

This should be done only on the thank you page; we'll leave the page tags on the other pages alone.

The next step is to update the script that processes and slogs these requests. We next need to modify the CGI program that reads the page tag and writes the logfile (readtag.pl). This is straightforward. We previously wrote five fields using the command:

 print LF "$time\t$client\t$url\t$ref\t$cookie_val\n"; 

Now we simply modify the script to write either five or six depending whether revenue is present or not. To implement the new functionality, replace the original code line in readtag.pl file with this new set of statements:

 my $rev = $cgi->param('rev'); if (defined($rev)) {   print LF "$time\t$client\t$url\t$ref\t$cookie_val\t$rev\n"; } else { print LF "$time\t$client\t$url\t$ref\t$cookie_val\n"; } 

Once these changes are complete, some of the logfile lines will include an additional field on the end, like the second line here:

 1104772080 192.168.17.32 /checkout.php http://www.example.com/addtocart. php?prodid=17454 192.168.17.32.85261104772101338 1104772091 192.168.17.32 /thankyou.php http://www.example.com/checkout. php 192.168.17.32.85261104772101338 39.99 

6.11.2. The Code: readlog.pl

The next step is to modify the logfile reader to read these types of lines. The following lines should be added to the appropriate classes in this script. The fist step is to modify the parser in the Request constructor to understand the optional additional field. Modify the Request (Request.pm) class to look like this:

 package Request; use strict; my $case_insensitive = 0; sub new {   my ($invocant, $str) = @_;   return undef   unless (my ($time, $host, $file, $referrer, $cookie, $revenue) = $str =~ /^ # start of line (1\d{9})\t # time: ten digits starting with 1 ([^\t]+)\t # host: non-empty string ([^\t]+)\t # file: non-empty string ([^\t]*)\t # referrer: possibly empty string ([^\t]*) # cookie: possibly empty string (?:\t([\d\.]+))? # optional revenue (tab followed by number) $/x); # end of line   $file = lc $file if $case_insensitive;   return bless { time => $time, host => $host, file => $file, referrer => $referrer, cookie => $cookie, revenue => $revenue } } 

The Session class needs to know how to calculate the revenue of a session. If there are several revenues in the session, we'll use the last one, rather than adding them, to avoid duplicates if the visitor reloads the thank you page. The risk is that a visitor who does make two purchases will be counted only once, but this is usually a small problem. Modify the Session (Session.pm) class to look like this:

 package Session; … sub Revenue { my $self = shift;  for my $i (reverse 0..$#$self) { # count backwards through array   my $req = $self->[$i];   if (defined ($req->{revenue})) { return $req->{revenue}; } } return undef; # revenue never defined } 

Finally, the Data (Data.pm) class needs to report some statistics from the revenue. There are many useful things that we could report, but as examples, we'll show the total revenue, the conversion rate (i.e., the proportion of sessions with positive revenue), and the amount of revenue generated by each search term. Modify the Data class to look like this:

 package Data; … sub new {   return bless { … total_revenue => 0, sessions_with_revenue => 0, search_terms_revenue => {},   }; } sub AddSession {   …   my $revenue = $sess->Revenue();   if (defined($revenue) && $revenue > 0)   {     $self->{total_revenue} += $revenue; ++$self->{sessions_with_revenue}; if ($search_term) {   $self->{search_terms_revenue}->{$search_term} += $revenue; }   } } sub WriteReport {   …   $self->WriteHash('Search Terms with Revenue', 'search_terms_revenue'); } sub WriteSummary {   …   printf "Total revenue: \$%.2f\n", $self->{total_revenue};   printf "Sessions with revenue: %d (%.1f%%)\n", $self->{sessions_with_revenue}, $self->{total_sessions} == 0 ? 0: $self->{sessions_with_revenue} / $self->{total_sessions} * 100; } 

And that's it. We've successfully retrieved an additional piece of data using JavaScript, passed it to the server in the page tag, written it to the logfile, and reported it in our analysis. Other additional data could be added to the program just as easily.

6.11.3. Running the Code

As always, from the command line, assuming that page.log is in the same directory as readlog.pl, all you need to do is type:

 perl readlog.pl page.log 

Assuming you're collecting data using the rev variable, you will see a report similar to that in Figure 6-11 when you run the code.

Figure 6-11. Search terms with revenue analysis

Hopefully, this hack will suggest other ways you can easily use web measurement applications based on page tags to grab nontraditional web data for processing and analysis.

Dr. Stephen Turner and Eric T. Peterson