Server-Side Browser Sniffing | Speed Up Your Site[c] Web Site Optimization

You can use flow control to test for different browsers and display conditional code. This is especially useful for external JavaScript files that are optimized for various platforms. Using conditional SSI and the set command, you can create environment variables that closely mirror common client-side JavaScript sniffing techniques. Let's take a real-world example, using our own DHTML hierarchical menus (see Figure 17.1).

Figure 17.1. Peter Belesis' DHTML hierarchical menus.

For his hierarchical menus, Peter Belesis uses an external JavaScript "loader" file to conditionally load two other JavaScript files, for a total of three HTTP requests . This ingenious method avoids some compatibility problems Belesis found with older browsers. Here is the JavaScript code that filters out browsers that support the level of DHTML he needs for his menus (available at http://www. webreference .com/dhtml/hiermenus/latest/loader.html):

 HM_DOM   = (document.getElementById) ? true : false; // includes Opera     HM_NS4   = (document.layers) ? true : false;     HM_IE   = (document.all) ? true : false;    HM_IE4   = HM_IE && !HM_DOM;    HM_Mac   = (navigator.appVersion.indexOf("Mac") != -1);   HM_IE4M   = HM_IE4 && HM_Mac;  HM_Opera   = (navigator.userAgent.indexOf("Opera")!=-1);  HM_Konqueror = (navigator.userAgent.indexOf("Konqueror")!=-1); HM_IsMenu   = ((HM_DOM   HM_NS4   (HM_IE4 && !HM_IE4M)) && !HM_OPERA);

Let's try and replicate that logic with conditional SSI (see Listing 17.1).

Listing 17.1 XSSI Browser Sniffing

 <!--#if expr="${HTTP_USER_AGENT} = /Mac/" -->     <!--#set var="isMAC" value="true" --> <!--#endif --> <!--#if expr="${HTTP_USER_AGENT} = /Opera/" -->     <!--#set var="isOPERA" value="true" --> <!--#endif --> <!--#if expr="${HTTP_USER_AGENT} = /Konquerer/" -->     <!--#set var="isKONQUERER" value="true" --> <!--#endif --> <!--#if expr="${HTTP_USER_AGENT} = /MSIE [4-9]/" -->     <!--#set var="isIE" value="true" -->     <!--#if expr="${HTTP_USER_AGENT} = /MSIE [5-9]/" -->         <!--#set var="isIE5" value="true" -->         <!--#if expr="(${isIE5} && ${isMAC})" -->             <!--#set var="isDOM" value="true" -->         <!--#elif expr="${HTTP_USER_AGENT} = /MSIE.5\.[5-9]/" -->             <!--#set var="isDOM" value="true" -->         <!--#elif expr="${HTTP_USER_AGENT} = /MSIE [6-9])/" -->             <!--#set var="isDOM" value="true" -->         <!--#endif -->     <!--#endif --> <!--#elif expr="${HTTP_USER_AGENT} = /Mozilla\/[4-9]/" -->     <!--#set var="isNS" value="true" -->     <!--#if expr="${HTTP_USER_AGENT} = /Gecko/" -->         <!--#set var="isDOM" value="true" -->     <!--#endif --> <!--#endif --> <!--#if expr="(${isNS} && !${isDOM})" -->     <!--#set var="isNS4" value="true" --> <!--#endif --> <!--#if expr="(${isIE} && !${isDOM})" -->     <!--#set var="isIE4" value="true" --> <!--#endif --> <!--#if expr="(${isIE4} && {isMAC})" -->     <!--#set var="isIE4M" value="true" --> <!--#endif --> <!--#if expr="(${isDOM}  ${isNS4}  (${isIE4} && !${isIE4M}))" -->     <!--#set var="isMENU" value="true" --> <!--#endif -->

Whew! That's a fair amount of code, but it closely approximates the client-side JavaScript. We know that MSIE 5.0 (Mac) and MSIE 5.5 and above support the W3C DOM (at least enough for Peter's menus), and that anything using the Gecko engine by AOL/Netscape supports the DOM. By first checking for MSIE4+ and then Mozilla4+, we filter out older browsers that don't support DHTML and group non-MSIE browsers into the isNS camp. Then we use some AND NOT operators to assign all non-DOM Netscape, Explorer, and Mac Explorer browsers to isNS4 , isIE4 , and isIE4M variables, respectively. Note that IE 4 to 5.4 is grouped in IE4, because W3C DOM support begins with IE5.5.

Opera can use a slash or a space before the version number, depending on whether it is spoofing:

 Opera/6.0 (Windows NT 4.0; U) [en]  as Opera  Mozilla/5.0 (Windows NT 4.0; U) Opera 6.0 [en]  as Mozilla 5

To add Opera, you could use the following logic:

 <!--#if expr="${HTTP_USER_AGENT} = /Opera *\/* *[5-9]/" -->      <!--#set var="isDOM" value="true" --> <!--#endif -->

Next , you would use these new environment variables to conditionally include external JavaScript files, like this:

 <!--#if expr="${isMENU}" -->      <!--#include virtual="/scripts/ARRAYS.js" -->     <!--#if expr="${isDOM}" -->         <!--#include virtual="/scripts/DOM.js" -->     <!--#elif expr="${isIE4}" -->         <!--#include virtual="/scripts/IE4.js" -->     <!--#elif expr="${isNS4}" -->         <!--#include virtual="/scripts/NS4.js" -->     <!--#endif --> <!--#endif -->

However, in this case, we found that Opera 5 and 6 do not support all the features we need for the menus. The code is simplified somewhat for illustrative purposes, because we'd normally filter out Opera 0-6 here. Because Opera can disguise itself as MSIE or Mozilla, your first test should be for Opera , and then check for MSIE/Mozilla and not Opera , like this:

 <!--#if expr="${HTTP_USER_AGENT} = /Opera/" -->      <!--#set var="isOPERA" value="true" --> <!--#endif --> <!--#if expr="${HTTP_USER_AGENT} = /MSIE [4-9]/" --> <!--#if expr="!${isOPERA} -->     <!--#set var="isIE" value="true" --> ...

One improvement would be to merge the arrays file with each of the API files to reduce the number of includes from two to one. You learned about shrinking JavaScript in Chapter 9, "Optimizing JavaScript for Download Speed."

Faster Browser Sniffing with BrowserMatch

A better way to assign custom environment variables is to embed the regular expressions in the httpd.conf file for speed. Apache can assign environment variables with the BrowserMatchNoCase or BrowserMatch directives, which apply a regular expression check to the environment variable HTTP_USER_AGENT . As with the XBitHack and Options directives, you can add this code to the general section or virtual host section of your httpd.conf file. Listing 17.2 shows the same sniffing logic using BrowserMatchNoCase .

Listing 17.2 Setting Environment Variables with BrowserMatchNoCase for Speed

 BrowserMatchNoCase Mac isMAC BrowserMatchNoCase Opera isOPERA BrowserMatchNoCase Konquerer isKONQUERER BrowserMatchNoCase "MSIE [4-9]" isIE BrowserMatchNoCase "MSIE [5-9]" isIE5 BrowserMatchNoCase "MSIE.5\.[5-9]" isDOM BrowserMatchNoCase "MSIE [6-9]" isDOM BrowserMatchNoCase "Mozilla\/[4-9]" isNS BrowserMatchNoCase Gecko isDOM

NOTE: These directives are the same as setting isXXX=1 or or true or false.

With these new environment variables embedded in your server, you can then use in your HTML files an abbreviated version of the code used earlier (see Listing 17.3):

Listing 17.3 Improved SSI Browser Sniffing

 <!--#if expr="(${isNS} && !${isDOM})" -->     <!--#set var="isNS4" value="true" --> <!--#endif --> <!--#if expr="(${isIE} && !${isDOM})" -->     <!--#set var="isIE4" value="true" --> <!--#endif --> <!--#if expr="(${isIE4} && {isMAC})" -->     <!--#set var="isIE4M" value="true" --> <!--#endif --> <!--#if expr="(${isDOM}  ${isNS4}  (${isIE4} && !${isIE4M}))" -->     <!--#set var="isMENU" value="true" --> <!--#endif -->

You then can conditionally include external JavaScript files like this (see Listing 17.4):

Listing 17.4 Conditionally Including JavaScript Files

 <!--#if expr="${isMENU}" -->      <!--#include virtual="/scripts/ARRAYS.js" -->     <!--#if expr="${isDOM}" -->         <!--#include virtual="/scripts/DOM.js" -->     <!--#elif expr="${isIE4}" -->         <!--#include virtual="/scripts/IE4.js" -->     <!--#elif expr="${isNS4}" -->         <!--#include virtual="/scripts/NS4.js" -->     <!--#endif --> <!--#endif -->

By embedding the HTTP_USER_AGENT string comparisons within the server configuration file, you shrink and speed up your SSI code. Of course, once Microsoft or AOL revs their browsers above 9, you'll need to tweak the regular expressions to handle versions 1099, but that won't happen for a while.

PHP Browser Sniffing

Embedded server-side scripting languages like PHP and JSP also can be used to sniff browser environment variables and capabilities. PHP has been called "SSI on steroids," because it allows you to use a real programming language within your pages.

To do simple sniffing for conditional CSS in PHP, you'd do something like this (see Listing 17.5).

Listing 17.5 Conditional CSS in PHP

 <?php if (preg_match("/Opera (\d)/i",$HTTP_USER_AGENT,$v)) {     if (($version=$v[1])>4) $type=1;     else $type=2; } elseif (preg_match("/MSIE (\d)/i",$HTTP_USER_AGENT,$v)) {     if (($version=$v[1])>4) $type=1;     else $type=2; } elseif (preg_match("/Mozilla(?:\/\s)(\d)/i",$HTTP_USER_AGENT,$v)) {     if (($version=$v[1])>4) $type=1;     else $type=2; } else {     $version="unknown";     $type=2; } switch ($type) {     case 1: $StyleSheet=compliant.css; break;     case 2: $StyleSheet=older.css; break;     default: StyleSheet=older.css; break; } ?>

A number of pre-written PHP browser sniffers are available:

The Horde Project: http://cvs.horde.org/co.php/ horde /lib/Browser.php
phpSniff: http://phpsniff. sourceforge .net/
SniffBrowser: http://www-student.eit.ihk.dk/instruct/php-sniffer.php

The open source Horde Project provides a browser sniffer as part of their application framework in PHP. Their browser class provides capability information for the current browser using the HTTP_USER_AGENT string.

Let's try and duplicate our JavaScript code with Horde's PHP browser class (see Listing 17.6).

Listing 17.6 Sniffing with the Horde Project's PHP Browser Class

 <?php require_once 'Browser.php'; $browser = new Browser(); $isOPERA     = $browser->isBrowser('opera'); $isKONQUERER = $browser->isBrowser('konqueror'); $isDOM  = $browser->hasFeature('dom'); $isNS4  = $browser->isBrowser('mozilla') && $browser->getMajor() == 4; $isIE   = $browser->isBrowser('msie') && !$isOPERA; $isIE4  = $isIE && !$isDOM; $isMAC  = stristr($HTTP_SERVER_VARS['HTTP_USER_AGENT'], "Mac"); $isIE4M = $isIE4 && isMAC; $isMENU = ($isDOM  $isNS4  ($isIE4 && !isIE4M)); if ($isMENU) { include($DOCUMENT_ROOT . "/scripts/ARRAYS.js "); if ($isDOM) {     include($DOCUMENT_ROOT . "/scripts/DOM.js");     } elseif ($isIE4) {         include($DOCUMENT_ROOT . "/scripts/IE4.js");     } elseif ($isNS4) {         include($DOCUMENT_ROOT . "/scripts/NS4.js");     } } ?>

Of course, we'd have to make sure that the browser class matched our definition of DOM support. (They say it mirrors the document.getElementById test.) Note that we've added in an Opera exclusion test for MSIE here for safety (even though the Horde sniffer doesn't identify Opera as being DOM compliant). Once Opera provides sufficient DOM support for our menus, we could add it back to the isMENU test.

PHP also provides a built-in function called get-browser.php . This function matches the HTTP_USER_AGENT string to the browser's capabilities using the browscap.ini file, which we'll discuss next. The function returns an object, which contains various data elements representing browser capabilities. Unfortunately, you've got to be sure you keep your browscap.ini file up to date. For more information on PHP's get-browser function, see http://www.php.net/manual/en/function.get-browser.php.

Faster PHP Scripts

Zend Technologies offers software tools to optimize your PHP code at http://www.zend.com.

Advantages and Disadvantages to Server-Side Browser Sniffing

The main advantage to server-side browser sniffing is that both these techniques save three HTTP requests over the client-side technique of loading external JavaScript files. For high-traffic pages, this can make a big difference in response rate.

Unlike external files, conditionally included scripts and CSS are not cached for subsequent pages. Some webmasters use a hybrid approach, using SSI on their home page and external files for the rest of the site.

Unlike JavaScript browser sniffing, this technique is limited by the information the browser sends to the server; in this case, the HTTP_USER_AGENT string. The problem with this approach is that the DOM category is under-represented. The DOM test is not as elegantly inclusive as the JavaScript DOM test ( isDOM = (document.getElementById) ? true : false; ), which catches all browsers that support this technique of element referencing. This test is often used as a surrogate for DOM support. It is close enough for our purposes. I've approximated this by testing explicitly for MSIE 5/5.5+ and Gecko browsers that we know support the DOM. MSIE, Mozilla, and Gecko variants account for nearly 90 percent of the users who are browsing WebReference.com. ^[8] The other 10 percent are Lynx, robots, iCab, and the like.

^[8] WebReference.com, "WebReference.com Statistics" [online], (Darien, CT: Jupitermedia Corporation, 2002 [cited 13 November 2002]), available from the Internet at http://webreference.com/stats/.

There are two ways to include any DOM-compliant agents in this group: through brute force and by sending JavaScript tests from the server.

browscap : The Browser Capabilities File

The good folks at cyScape, Inc. created a browser capabilities file ( browscap.ini ) that recognizes many of the browsers in use today by way of HTTP_USER_AGENT strings (http://www.cyscape.com/browscap/).

The file maps these strings to various browser capabilities. Be sure to get the latest version, because the file isn't updated very often. Juan Llibre keeps his version of browscap pretty up to date. You can check out that version at http://asp.net.do/browscap.zip.

By correlating these strings with the browsers that support the DOM, you could in theory make a brute-force XSSI DOM test that exactly matched the JavaScript equivalent. You could create a list of browser id strings based on the HTTP_USER_AGENT environment variable and embed them in your httpd.conf or php.ini file.

When new browsers enter the arena, however, you've got a problem. You would have to update your browscap.ini file and your regular expressions.

A better brute-force approach would be to flag all the browsers that don't support the DOM and negate them to find those who do. This would be a matter of finding all non-MSIE/Mozilla/Gecko browsers that support the DOM, finding the version number where they started, and listing these browsers just below that version, along with those browsers (Lynx, for example) that don't support the DOM.

Let's take a generic example. Suppose that you have a new DOM-compliant Mozilla competitor called Mothra. At version 7, Mothra starts to support the DOM, so you would do something like this:

 <!--#if expr="${HTTP_USER_AGENT} = /Mothra [0-6]/" -->      <!--#set var="notDOM" value="true" --> <!--#endif --> ... other non-DOM-compliant browsers here ... <!--#if expr="(!${notDOM} && !${isIE4} && !${isNS4})" -->     <!--#set var="isDOM" value="true" --> <!--#endif -->

For either brute-force technique, especially the first, you'd throw in a safety check for DOM support in the JavaScript file, like this:

 isDOM = (document.getElementById) ? true : false;  if isDOM {     //DOM code goes here }

This would filter out any browsers that weren't listed in the database but nevertheless squeaked through and didn't support the DOM. The !notDOM approach would require fewer updates of the browscap.ini file. This approach would match the JavaScript DOM sniffing technique, even after a new DOM-compliant browser came out. The disadvantage to this technique is the need to update the browscap.ini file as new browsers appear (although less often than the previous approach). Wouldn't it be nice if there were a way to do this automatically? That's where products like BrowserHawk come in.

BrowserHawk: Advanced Server-Side Sniffing

BrowserHawk from cyScape, Inc. is the next step up in server-side browser sniffing. The brainchild of Richard Litofsky, BrowserHawk can detect almost everything about your browser on the server, including JavaScript capabilities.

BrowserHawk is available for Active Server Pages (ASP), Coldfusion (CF), and Java Server Pages (JSP) and servlets (as a Java Bean), and it can be used on any web server that supports Active Server Pages, JavaServer Pages, or servlets such as Microsoft IIS and Apache. BrowserHawk uses an object-based binary file for its browser database. This provides faster browser detection than browscap.ini .

The database includes over 120 properties, compared to 15 or so in browscap . It can be updated automatically with the Professional and Enterprise Editions (see Figure 17.2).

Figure 17.2. BrowserHawk detecting environment variables.

With BrowserHawk, you can use Java Server Pages (JSP) to do sophisticated browser sniffing on the fly. For our browser-sniffing example, the BrowserHawk database has all the browser details it needs. Listing 17.7 shows the example browser sniffing code in JSP.

Listing 17.7 Sniffing with JSP and BrowserHawk

 <%@ page import = "com.cyscape.browserhawk.*" %> <%   BrowserInfo b = BrowserHawk.getBrowserInfo(request);   // This code sets some basic browser properties   boolean isMac = b.getPlatform().startsWith("Mac");   boolean isIE  = b.getBrowser().equals("IE");   boolean isNS  = b.getBrowser().equals("Netscape");   boolean isOpera = b.getBrowser().equals("Opera");   boolean isKonqueror = b.getBrowser().equals("Konqueror");   // This code sets the isDOM and isMenu vars   boolean isIE4Plus = isIE && b.getMajorver() >= 4 && b.getVersion < 5.5;   boolean isIE5Plus = isIE && b.getMajorver() >= 5;   boolean isIE55Plus = isIE && b.getVersion() >= 5.5;   boolean isIE5Mac = isIE5Plus && isMac;   boolean isIE55NoMac = isIE55Plus && !isMac;   boolean isNS4Plus = isNS && b.getMajorver() >= 4 && b.getMajorver() < 5;   boolean isNS6Plus = isNS && b.getMajorver() >=6;   boolean isMozilla = b.getBrowser().equals("Mozilla");   boolean isGecko   = b.getGecko();   boolean isDOM = isIE55NoMac   isNS5Plus   isIE5Mac   isMozilla;   boolean isMenu = isDOM   isNS4Plus   (isIE4Plus && !isMac); %> <% if (isMenu) { %> <%@ include file="/scripts/ARRAYS.js" %> <% } %> <% } if (isDOM) { %> <%@ include file="/scripts/DOM.js" %> <% } else if (isIE4Plus && !isMac) { %> <%@ include file="/scripts/IE4.js" %> <% } else if (isNS4Plus) { %> <%@ include file="/scripts/NS4.js" %> <% } %>

This JSP code performs the same function as the PHP and XSSI examplesonly faster. Using JSP with BrowserHawk gives better performance than JavaScript includes and conditional SSI, assuming that you don't mind the additional overhead of running JSP or ASP.

What About Standards?

Of course, if we all wrote for the W3C's DOM, we wouldn't need this elaborate browser detection. This is true for (X)HTML and CSS in large part, but JavaScript is a special case. Accommodating older browsers and the DOM can be done in one file or separate files. DHTML developers typically either separate their code into three parts (IE4, NS4, and DOM), use an API that abstracts these differences, or ignore older browsers altogether.

The problem with the multi-file approach is that you've got at least two external files to load (data arrays and the browser APIs, for example) adding two HTTP requests to every page. By using server-side sniffing and conditional includes, you can eliminate extra HTTP requests and dramatically speed up high-traffic pages.

For More Information

For more information on the topic of client-side versus server-side browser sniffing, visit http://cyscape.com/developer/workshop/article/bh_vs_js.asp.

Conditional meta Tags

For maximum speed, you can optionally omit meta tags from your high-traffic pages. Here is an example from WebReference.com's home page:

 <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">  <html> <head> <title>WebReference.com - The Webmaster's Reference Library - Web Authoring Tips &  Tutorials for Developers</title> <!--#if expr="$HTTP_USER_AGENT = /^Mozilla/" --><!--#else --> <meta name="keywords" content="authoring web development web design java script graphic design html tutorials javascript dynamic html tutorial  authoring tools ..."> <meta name="description" content="The definitive guide to web development with tutorials on all aspects of web design ..."> <!--#endif --> ... </head>

If the server detects a Mozilla-based browser it sends no meta tags; otherwise , it does send them (presumably to a spider). You learned how to optimize meta tags for search engine relevance in Chapter 15, "Keyword Optimization."