Hack 44 Keep Server Clocks Synchronized

Make log analysis easier by keeping the time on your systems in sync.

Correlating events that occurred on multiple servers can be a chore if there are discrepancies between the machines' clocks. Keeping the clocks on your systems synchronized can save valuable time when analyzing router, firewall, and host logs after a compromise, or when debugging everyday networking issues. Luckily, it's not that hard to do this with a little help from NTP, the Network Time Protocol.

NTP is a peer-to-peer protocol designed to provide subsecond precision and accuracy between host clocks. To get this going, all you need is the NTP distribution (http://www.ntp.org/downloads.html), which contains a daemon for performing clock synchronization, plus other supporting tools. While NTP might not be installed on your system, it usually comes with the various Linux distributions, FreeBSD, and OpenBSD as an optional package or port, so poke around your installation media or the ports tree if it's not already installed. If it isn't available with your OS of choice, you can still download and compile it yourself.

Configuring ntpd as a client is a fairly simple process. However, first you'll need to find out whether you have a local time server, either on your network or at your ISP. If you don't, you'll have to locate an NTP server that will let you query from it. Don't worry, though a list of all the publicly accessible time servers is available at http://www.eecis.udel.edu/~mills/ntp/servers.html. One new term you will encounter when looking for a server is stratum (e.g., stratum 1 or stratum 2). This refers to the hierarchy of the server within the public NTP infrastructure. Stratum 1 servers are usually machines that have a direct time-sync source, such as a GPS or atomic clock signal that provides updates to the daemon running on that machine. Stratum 2 servers obtain their time sync from stratum 1 servers. Using stratum 2 servers helps to reduce the load on stratum 1 servers and is still accurate enough for our purposes. In addition, you'll want to find servers that are as geographically close to you as possible.

With this in mind, let's look for some NTP servers that we can use (using more than one is generally a good idea, in case one fails). I live in Colorado, so after following the link to the stratum 2 server list (http://www.eecis.udel.edu/~mills/ntp/clock2a.html), I find two entries:

# US CO ntp1.linuxmedialabs.com Location: Linux Media Labs LLC, Colorado Springs, CO Service Area: US Synchronization: NTP Secondary (stratum 2), i686/Linux Access Policy: open access Contact: [e-mail] Note: ntp1 is an alias and the IP address may change, please use DNS # US CO ntp1.tummy.com Location: tummy.com, ltd., Fort Collins, CO Service Area: US Synchronization: NTP Secondary (stratum 2), i686/Linux Access Policy: open access. Contact: [e-mail] Note: ntp1 is an alias and the IP address may change, please use DNS.

Since they're both listed as open access, I can just add them to /etc/ntp.conf:

server ntp1.linuxmedialabs.com server ntp1.tummy.com

In addition, ntpd can automatically correct for the specific clock frequency drift of your machine. It does this by learning the average drift over time as it receives sync messages. To enable this, add a line similar to the following to your ntp.conf:

driftfile /etc/ntp.drift

Of course, if you're keeping all of your ntpd configuration files in /etc/ntp, you'll want to use a directory similar to /etc/ntp/ntp.drift instead.

That's it. Simply add ntpd to your startup scripts, start it up, and you're ready to go.