10.5 The Network Measurement Database

Squid's network measurement database ( netdb ) is designed to measure the proximity of origin servers. In other words, by querying this database, Squid knows how close it is to the origin server. The database includes ICMP round-trip time (RTT) measurements and hop counts. Squid normally uses only the RTT measurements but can also use the hop counts in some situations.

To enable netdb , you must configure Squid with the ”enable-icmp option. You must also install the pinger program with superuser permissions, as described in Section 3.6. When everything is working correctly, you should see a message like this in cache.log :

 2003/09/29 00:01:03 Pinger socket opened on FD 28 

When netdb is enabled, Squid sends ICMP " pings " to origin servers. The ICMP messages are actually sent and received by the pinger program, which runs as root . Squid is careful not to send pings too frequently, which may annoy web site administrators. By default, Squid waits at least five minutes before sending another ping to the same host, or to any other host on the same /24 subnet. You can adjust the interval with the netdb_ping_period directive.

The ICMP pings are generally small in size (less than 100 bytes). Squid includes the origin server hostname in the payload of the ICMP message, along with a timestamp.

To reduce memory requirements, Squid aggregates the netdb data by /24 subnets. Squid assumes that all hosts within the subnet have similar RTT and hop-count measurements. This scheme also allows Squid to estimate the proximity of a new origin server when other servers in the subnet have already been measured.

Along with the RTT and hop-count measurements, Squid also stores a list of hostnames associated with the subnet. A typical record may look something like this:

 Subnet 140.98.193.0 RTT    76.5 Hops   20.0 Hosts  services1.ieee.org        www.spectrum.ieee.org        www.ieee.org 

The netdb measurements are primarily used by ICP and HTCP. When you enable the query_icmp directive in squid.conf , Squid sets a flag in the ICP/HTCP queries that it sends to neighbors. This flag is a request to include proximity measurements in the ICP/HTCP reply. If your neighbors also enabled netdb , their replies should include RTT and hop-count measurements if available. Note that Squid always sends ICP replies immediately. It doesn't wait for an ICMP measurement before replying to the query. See Section 10.6.2.2 for details on how ICP uses netdb .

Squid remembers the RTT values it learns from ICP/HTCP replies. These values may be used later to optimize forwarding decisions. Squid also supports a "bulk transfer" of netdb measurements via what is called netdb exchange . Squid periodically makes an HTTP request to a neighbor for its netdb data. You can disable these requests with the no-netdb-exchange option on the cache_peer line.

The netdb_low and netdb_high directives control the size of the measurement database. When the number of stored subnets reaches netdb_high , Squid deletes the least recently used entries until the count is less than netdb_low .

The minimum_direct_hops and minimum_direct_rtt directives instruct Squid to connect directly to origin servers that are no more than some number of hops, or milliseconds , away. Requests that meet this criteria are logged with CLOSEST_DIRECT in access.log .

The cache manager's netdb page displays the entire network measurement database, including values from neighbor caches. For example:

 Network DB Statistics: Network          recv/sent     RTT  Hops Hostnames 63.241.84.0         1/   1    25.0   9.0 www.xyzzy.com     sd.us.ircache.net         21.5  15.0     bo1.us.ircache.net        27.0  13.0     pb.us.ircache.net         70.0  11.0 206.100.24.0        5/   5    25.0   3.0 wcarchive.cdrom.com ftp.cdrom.com     uc.us.ircache.net         23.5  11.0     bo1.us.ircache.net        27.7   7.0     pb.us.ircache.net         35.7  10.0     sd.us.ircache.net         72.9  10.0 146.6.135.0         1/   1    25.0  13.0 www.cm.utexas.edu     bo1.us.ircache.net        32.0  11.0     sd.us.ircache.net         55.0   8.0 216.234.248.0       2/   2    25.0   8.0 postfuture.com www1.123india.com     pb.us.ircache.net         44.0  14.0 216.148.242.0       1/   1    25.0   9.0 images.worldres.com     sd.us.ircache.net         25.2  15.0     bo1.us.ircache.net        27.0  13.0     pb.us.ircache.net         69.5  11.0 

Here you can see that the server www.xyzzy.com has an IP address in the 63.241.84.0/24 block. The RTT from this cache to the origin server is 25 milliseconds. The neighbor cache sd.us.ircache.net is a little closer, at 21.5 milliseconds.