11.2 Monitoring Tools

only for RuBoard - do not distribute or recompile

11.2 Monitoring Tools

Most caching products provide both custom interfaces and monitoring via SNMP (Simple Network Monitoring Protocol). The custom interfaces are typically web-based and often use Java. While these tools can display graphs for some of the parameters just described, they usually have only volatile storage. When the proxy cache restarts, the historical data is lost.

Although most products support SNMP, there is no standard SNMP proxy MIB (Management Information Base). Instead, each vendor or product has its own MIB with product-specific information. If enough customers complain about the lack of a standard MIB, vendors are likely to develop one within the IETF.

In theory, monitoring a network device via SNMP is relatively simple. You make the MIB file available to your network management application and tell it which variables to collect and display. In practice, however, it's often a frustrating process. There are a number of management applications available for all types of operating systems. As a relatively simple example, the remainder of this section describes how to install and configure UCD-SNMP and RRDTool to monitor a Squid cache. To monitor another SNMP-addressable caching product, you'll need to use its MIB.

11.2.1 UCD-SNMP

The University of California at Davis distributes a suite of SNMP tools for Unix. You can download the software from their home page, http://ucd-snmp.ucdavis.edu, or one of the mirror sites. Hopefully, you'll find the software easy to compile and install. It should be as easy as this:

 % gzip -dc ucd-snmp.tar.gz  tar xf - % cd ucd-snmp-4.1.1 % ./configure % make % su # make install 

During the configure phase, you'll be prompted to enter some information. This information is used only when you run the snmpd daemon. The snmpd program is not needed to monitor Squid or any other proxy cache. We are only interested in the snmpget command.

If you watched the install process closely, you noticed that SNMP MIB files are placed in /usr/local/share/snmp/mibs/ by default. We'll need to copy the Squid MIB to this location:

 # cp squid/src/mib.txt /usr/local/share/snmp/mibs/SQUID-MIB.txt # chmod 644 /usr/local/share/snmp/mibs/SQUID-MIB.txt 

Once the MIB has been installed, we can test everything with the following snmpget command:

 % snmpget -m SQUID-MIB -p 3401 -R webcache.isp.net public \   squid.cacheConfig.cacheAdmin 

The -m option tells snmpget to load the SQUID-MIB file. This is needed to translate the long identifier name into a sequence of numbers . The -p option specifies the destination port number for the SNMP query. Squid uses port 3401 by default. The -R option allows us to use short variable names . Without it, the last argument would need to be much longer. You should replace webcache.isp.net with the hostname of your own proxy cache. The next argument, public , specifies the SNMP community string. Essentially, this is a password. Many applications use public as the default but should allow you to change it if you like. Finally, squid.cacheConfig.cacheAdmin specifies the MIB variable we want to get. In this case, we're asking for the email address of the cache administrator. If successful, the result comes back like this:

 enterprises.nlanr.squid.cacheConfig.cacheAdmin = [e-mail] 

If you get an error message, snmpget may not be able to locate the Squid MIB file. If you get no response or a timeout, Squid might not be configured for SNMP. You'll also get a timeout if your network has a firewall that blocks UDP packets.

11.2.2 RRDTool

RRDTool is a successor to the highly popular MRTG (Multi Router Traffic Grapher) software. RRDTool manages data in the form of a round- robin database (RRD). The idea is that data is archived over time, at different granularities. A database stores the same measurements consolidated over various time intervals. For example, you can have 24 hours' worth of measurements taken at 5-minute intervals, a month's worth of measurements consolidated into 1- hour intervals, and so on. With this structure, RRDTool can store many years ' worth of measurements in a relatively small amount of space. Of course, some information is lost as old measurements are consolidated over time. You won't be able to go back and compare two five-minute intervals from a year ago.

You can get the RRDTool software from http://ee-staff.ethz.ch/~oetiker/webtools/rrdtool/. After downloading, it should compile and install easily:

 % gzip -dc rrdtool-1.0.13.tar.gz  tar xf - % cd rrdtool-1.0.13 % ./configure % make # make install 

The next step is to create a round-robin database to hold and manage the measurements. To do this, we need to know which values we'll store, their types, and for how long. The syntax for rrdtool create is somewhat confusing. Since I can't explain everything here, you may want to read the RRDTool manual before proceeding.

For this example, we'll create a database to hold HTTP and DNS median service times. The gauge data type is used for values that go up and down, such as temperature:

 % rrdtool create svctime.rrd \   --step 300 \   DS:http:GAUGE:600:U:U \   DS:dns:GAUGE:600:U:U \   RRA:AVERAGE:0.99:1:288 \   RRA:AVERAGE:0.99:6:336 \   RRA:AVERAGE:0.99:12:744 

The step argument specifies the default interval (in seconds) at which measurements are collected. The DS lines define the data sources. In this case we have HTTP and DNS request counters. The RRA lines specify how the data is archived and consolidated.

To collect the data, we use a shell script that runs as a cron job every 5 minutes. The script uses snmpget to get the data and stores the values with rrdtool update :

 #!/bin/sh SNMPGET="snmpget -q -R -s -m SQUID-MIB -p 3401 webcache.isp.net public" cd /where/rrd/files/live http=`$SNMPGET cacheHttpAllSvcTime.5  awk '{print }'` dns=`$SNMPGET cacheDnsSvcTime.5  awk '{print }'` rrdtool update svctime.rrd --template http:dns N:$http:$dns 

Squid calculates service times averaged over 1-, 5-, and 60-minute time intervals. This example uses the 5-minute interval, which is what the the .5 following both names refers to. The N in the last argument of the rrdtool update command stands for the current time (i.e., "now").

Finally, let's see how rrdgraph is used to make a graph of the data:

 % rrdtool graph svctime.day.png \   --start -1day \   --title "Service times" \   --vertical-label "seconds" \   --imgformat PNG \   DEF:http=/where/rrd/files/live/svctime.rrd:http:AVERAGE \   DEF:dns=/where/rrd/files/live/svctime.rrd:dns:AVERAGE \   AREA:http#0000FF:HTTP \   LINE2:dns#00FF00:DNS 

The --start -1day option instructs rrdgraph to show the previous 24 hours' worth of data. The AREA argument draws the HTTP service time as a filled area with color #0000FF (blue). The LINE2 argument draws the DNS service time as a solid line with double thickness in green. Figure 11-1 shows the resulting image.

Figure 11-1. Sample RRD graph

The RRDTool package also includes a program called rrdcgi . It parses HTML files with special embedded tags that look like rrdgraph commands. Using rrdcgi , you can write HTML pages that generate RRD graphs on the fly. Unfortunately, setting up rrdcgi is somewhat complicated. You'll need to refer to the RRDTool documentation for syntax and examples.

11.2.3 Other Tools

Many people use MRTG to collect and display their cache statistics. MRTG is perhaps easier to install and configure, but it's not nearly as flexible as RRDTool. MRTG also has a number of performance problems. You can download MRTG from http://www.mrtg.org.

In addition to real-time monitoring, you'll probably want to use some scripts or tools to analyze your access log files. These can tell you how many requests were served , the cache hit ratio, bandwidth utilization, and more. Again, I recommend keeping an archive of these reports and statistics for planning and troubleshooting.

Any program that analyzes access log files needs to understand the file format. Many caching products support the HTTP common logfile format. A number of products also support Squid's format, probably to take advantage of the preexisting analysis tools. Calamaris (http://calamaris.cord.de) and Webalyzer (http://www.mrunix.net/webalizer/;) are perhaps the most widely used programs. You can find additional tools listed at http://www.squid-cache.org/Scripts/.

only for RuBoard - do not distribute or recompile