Hack34.Measure Your Mistakes

Hack 34. Measure Your Mistakes

One of the historical uses for web measurement tools that retain value is the ability to identify and diagnose error messages that your visitors may be seeing.

Nobody is perfect, they say, and web programmers are often less perfect than most. Constrained by tight deadlines, scope creep, and poorly defined standards for web page development, web programmers do the best they can. Unfortunately, the best is sometimes not good enough, and errors arise. One of the most valuable uses of your web measurement application is the identification of errors so any problems can be quickly corrected.

2.23.1. Types of Mistakes

There are many types of mistakes that can happen, but not all of them can be traced with typical web measurement tools alone. Here are a handful of the most common that can be tracked.

2.23.1.1 Web site is slow or fails to respond.

There are services that specialize in web site monitoring designed to ping your web site regularly to ensure that it is accessible and that your web servers are handling page requests in acceptable amount of time. You can set up alerts for when something is wrong. Typical web measurement tools on the other hand do not provide adequate support for measuring these aspects, but it is possible to integrate response and availability into your web measurement application to provide a more holistic view of the situation [#Hack Measure Site Performance].

2.23.1.2 Broken hyperlinks.

If a visitor requests a URL that is unavailable, the web server will return the classic HTTP 404 error (we suspect you've seen this error before). The most typical cause for this is that there is a broken hyperlink somewhere that points to an incorrect URL, one that does not exist. Web measurement tools can capture and report on this event after it happens.

Monitoring your broken hyperlinks report is probably one of the most important things you can do with your web measurement application.

Run your broken link report every day looking for problems on your web site. Some of the time, the problem will be yours and easily corrected. Other times, the links will be coming from another site. In this case, you can either write the site asking them to correct the link or put a redirect in place of the broken link designed to push the visitor along to the right page.

2.23.1.3 Aborted page views and downloads.

Tracking abandonment and interrupted downloads is more difficult than tracking HTTP errors, however. It requires using web server logfiles for your data collection because page tags can capture neither incomplete downloads nor the download time. The Microsoft IIS web server records a server status code of 64 when a request is terminated. Some web measurement tools supply such plug-ins and can report on page abandonment rates and incomplete downloads (Figure 2-23).

2.23.1.4 Client-side script errors.

One of the most frustrating types of errors your visitors encounter are script errorsthe type that prevent links from working, forms from being submitted, and otherwise result in a very poor user experience. While difficult to actively measure, creating a strategy for measuring this type of error can explain a great deal about why your visitors fail to move from page to page on your site.

2.23.2. How to Measure Your Mistakes

Depending on whether you are using web server logfiles or page tags for collecting traffic data, there are different methods to capture errors.

Figure 2-23. Report showing the percentage of pages and downloads delivered completely

2.23.2.1 Using logfiles as a data source.

Web server logfiles record page requests with errors the same way they record successful page requests. The only difference is in the HTTP status code that is recorded along with the request in the logfile. Table 2-5 shows some of the most frequent HTTP protocol status codes:^[6]

^[6] A complete listing can be obtained at http://www.w3.org.

Table 2-5. Frequently encountered HTTP status codes

HTTP code	Interpretation
200	Successful hit; the request was fulfilled.
302	Redirection. The data requested actually resides under a different URL.
401	Unauthorized request. The request requires user authentication.
404	The server has not found anything matching the given URL.
503	Service unavailable. The server is currently unable to handle the request.

2.23.2.2 Using page tags as a data source.

Many hosted application vendors have custom error page code that can be used to specifically capture errors, provided there is not a specific JavaScript error on the page. This strategy usually requires that you have a custom error page (usually a 404 or "File Not Found" page) that can be coded.

2.23.2.3 Measuring client-side JavaScript errors.

Page tags cannot detect many types of errors directly because the problem does not by default trigger an event. Placing a page tag on a custom error page and passing the error message in a custom variable [Hack #31] will record the incident for you and make the error information available to your reports.

One way to do this is to use page tags and error handlers in your scripts. For example, in JavaScript you can specify an onError handling function for your document. In onError, you can send additional information via the page tag to a custom variable [Hack #31]. The following code can be used to build list of client-side errors to pass to your web measurement page tag.

    <SCRIPT>     ' Tell JavaScript how to handle any client-side errors    window.onerror = myOnError     ' Define arrays to capture the error messages     msgArray = new Array()     urlArray = new Array()    lnoArray = new Array()    ' When an error is discovered, build an array of messages, urls and line    numbers    function myOnError(msg, url, lno) {    msgArray[msgArray.length] = msg    urlArray[urlArray.length] = url    lnoArray[lnoArray.length] = lno        return true    }    ' Build a string that can be sent to the data collector    function sendErrors() {        var errorList=""    for (var i=0; i < msgArray.length; i++) {       errorList = errorList + urlArray[i] + '|' + lnoArray[i] + '|' +    msgArray[i];    }    '    ' CALL TO THE PAGE TAG/DATA COLLECTOR GOES HERE!    '    }    </SCRIPT> 

Depending then on how your particular web measurement application works, you'll replace the CALL TO THE PAGE TAG/DATA COLLECTOR GOES HERE line with an image request (or whichever strategy your vendor recommends) that will send the string generated in errorList to a custom variable that can be examined later.

2.23.3. Investigating Your Mistakes

Often, you'll see errors report and have no idea why they're being generated. Rather than sit and scratch your head, you should actually drill down into the problem and see what else you can learn about the source of the problem. By investigating your mistakes, you can both correct the shortterm problem and prevent similar problems from arising in the future.

2.23.3.1 Investigate errors in relation to traffic loads.

Segment the errors by time of day and day of week. If your web measurement tool can report on activity loads by time of day, see whether the error instances coincide with traffic rush hours. It may be that the system runs fine when idle but that errors arise with slower performance during times of heavy traffic.

2.23.3.2 Investigate errors in relation to web browsers.

Segment the errors by the web browser and platform [Hack #71] that visitors used. You may find that the error is specific to certain web browsers. Browser version issues are the most common culprit when you're investigating script-related issues.

2.23.3.3 Investigate errors in relation to your servers.

If you have a server farm and your web measurement tool can report on traffic segmented by each of your load balanced web servers, investigate whether one of your servers is responsible for errors while others may be running fine. A good indication of a problem would be significantly fewer pages being served from one machine relative to the volume of traffic distributed to the server (e.g., if a machine is served 50 percent of the traffic but only serves 30 percent of the page views, something is wrong).

2.23.3.4 Investigate errors in relation to dynamic URLs.

If your web measurement tool can provide the necessary detail, investigate whether errors only occur with selected query strings among your dynamic URLs. If you're using a tag-based measurement solution, you may want to ask your vendor if it's possible to pass the full value of the requested URL along with the generated error message, perhaps in a custom variable, allowing you to complete this type of analysis.

Since you cannot watch people surf your site in appreciable numbers, it is essential to track your mistakes and act upon the data you generate. Nothing is more frustrating to a web visitor than error messages, especially when it is well within your power to prevent and correct problems.

Akin Arikan and Eric T. Peterson