Logging

Squid provides a number of logs that can be used when debugging problems and when measuring the effectiveness and identifying users and the sites they visit (Figure 12-5). Because Squid can be used to 'snoop' on user's browsing habits, one should carefully consider privacy laws in your region and, more importantly, be considerate to your users. That being said, logs can be very valuable tools in ensuring that your users get the best service possible from your cache.

Figure 12-5: Logging configuration

Access log file

The location of the cache access.log. The Squid access.log is the file in which Squid writes a small one-line entry for every request served by the cache. This option correlates to the cache_access_log directive and usually defaults to /usr/ local/squid/log/access.log or on some RPM-based systems /var/log/squid/access.log. The format of the standard log file looks like this:

973421337.543 11801 192.168.1.1 TCP_MISS/200 1999 GET http://www.google.com/ -- DIRECT/64.208.34.100 text/html 

In the preceding line, each field represents some piece of information that may be of interest to an administrator:

1. System time in standard UNIX format. The time in seconds since 1970. There are many tools to convert this to human readable time, including this simple Perl script:

   #! /usr/bin/perl -p  s/^\d+\.\d+/localtime $&/e; 

2. Duration or the elapsed time in milliseconds the transaction required.

3. Client address or the IP address of the requesting browser. Some configurations may lead to a masked entry here, so that this field is not specific to one IP, but instead reports a whole network IP.

4. Result codes provides two entries separated by a slash. The first position is one of several result codes [http://www.squid-cache.org/Doc/FAQ/FAQ-6.html#cache-result-codes], which provide information about how the request was resolved or wasn't resolved if there was a problem. The second field contains the status code [http://www.squid-cache.org/Doc/FAQ/FAQ-6.html#http-status-codes ], which comes from a subset of the standard HTTP status codes.

5. Bytes is the size of the data delivered to the client in bytes. Headers and object data are counted toward this total. Failed requests will deliver and error page, the size of which will also be counted.

6. Request method is the HTTP request method [http://www.squid-cache.org/Doc/FAQ/FAQ-6.html#request-methods ] used to obtain an object. The most common method is, of course, GET, which is the standard method web browsers use to fetch objects.

7. URL is the complete Uniform Resource Locator requested by the client.

8. RFC931 is the ident lookup information for the requesting client, if ident lookups are enabled in your Squid. Because of the performance impact, ident lookups are not used by default, in which case this field will always contain '-'.

9. Hierarchy code consists of three items. The first is simply a prefix of TIMEOUT_ if all ICP requests timeout. The second (first if there is not TIMEOUT_ prepended) is the code that explains how the request was handled. This portion will be one of several hierarchy codes [ http://www.squid-cache.org/Doc/FAQ/FAQ-6.html#hier-codes]. This result is informative regardless of whether your cache is part of a cache hierarchy, and will explain how the request was served. The final portion of this field contains the name or IP of the host from which the object was retrieved. This could be the origin server, a parent, or any other peer.

10. Type is simply the type of object that was requested. This will usually be a recognizable MIME type, but some objects have no type or are listed as ':'.

There are two other optional fields for cases when MIME header logging has been turned on for debugging purposes. The full HTTP request and reply headers will be included enclosed in [ and ] square brackets.

Debug log file

The location for Squids cache.log file. This file contains startup configuration information, as well as assorted error information during Squids operation. This file is a good place to look when a website is found to have problems running through the web cache. Entries here may point towards a potential solution. This option correlates to the cache_log directive and usually defaults to either /usr/local/squid/log/cache.log or /var/log/squid/cache.log on RPM-based systems.

Storage log file

Location of the cache's store log file. This file contains a transaction log of all objects that are stored in the object store, as well as the time when they get deleted. This file really doesn't have very much use on a production cache, and it is primarily recommended for use in debugging. Therefore, it can be turned off by entering none in the entry field. The default location is either /usr/local/squid/ log/store.log or /var/log/squid/store.log.

Cache metadata file

Filename used in each store directory to store the web cache meta data, which is a sort of index for the web cache object store. This is not a human-readable log, and it is strongly recommended that you leave it in its default location on each store directory, unless you really know what you're doing. This option correlates to the cache_swap_log directive.

Use HTTPD log format

Allows you to specify that Squid should write its access.log in HTTPD common log file format, such as that used by Apache and many other web servers. This allows you to parse the log and generate reports using a wider array of tools. However, this format does not provide several types of information specific to caches and is generally less useful when tracking cache usage and solving problems. Because there are several effective tools for parsing and generating reports from the Squid standard access logs, it is usually preferable to leave this at its default of being off. This option configures the emulate_httpd_log directive. The Calamaris cache access log analyzer does not work if this option is enabled.

Log MIME headers

Provides a means to log extra information about your requests in the access log. This causes Squid to also write the request and response MIME headers for every request. These will appear in brackets at the end of each access.log entry. This option correlates to the log_mime_hdrs directive.

Perform RFC931 ident lookups for ACLs

Indicates which of the Access Control Lists that are defined should have ident lookups performed for each request in the access log. Because of the performance impact of using this option, it is not on by default. This option configures the ident_lookup_access directive.

RFC931 ident timeout

The timeout, usually in seconds, for ident lookups. If this is set too high, you may be susceptible to denial of service from having too many outstanding ident requests. The default for this is 10 seconds, and it applies to the ident_timeout directive.

Log full hostnames

Configures whether Squid will attempt to resolve the host name, so the fully qualified domain name can be logged. This can, in some cases, increase latency of requests. This option correlates to the log_fqdn directive.

Logging netmask

Defines what portion of the requesting client IP is logged in the access.log. For privacy reasons it is often preferred to only log the network or subnet IP of the client. For example, a netmask of 255.255.255.0 will log the first three octets of the IP and fill the last octet with a zero. This option configures the client_netmask directive.

Debug options

Provides a means to configure all of Squid's various debug sections. Squid's debugging code has been divided into a number of sections. If there is a problem in one part of Squid, debug logging can be made more verbose for just that section. For example, to increase debugging for just the Storage Manager in Squid to its highest level of 9 while leaving the rest at the default of 1, the entry would look like Figure 12-6.

Figure 12-6: Setting Squid debug levels

There is a complete list of debug sections [http://www.swelltech.com/support/docs/squid/debug-sections.txt ] at the Swell Technology website and in the Squid source distribution in the doc directory. More information can be found in the Squid FAQ [http://www.squid-cache.org/Doc/FAQ/FAQ-11.html#ss11.20].

MIME headers table

The pathname to Squid's MIME table. This usually should remain at the default value. This option configures the mime_table directive and defaults to /usr/local /squid/etc/mime.conf or /etc/squid/mime.conf.