8.4 Time Synchronization with NTP

Computers often don't work right when the hosts on a network have differing ideas about what time it is. For example, DNS servers become very upset when the master server's and slave servers' ideas of the current time are significantly different and will not accept zone transfers under such conditions. Also, many security protocols, such as Kerberos, have time-out values that depend on accurate clocks

The Network Time Protocol (NTP) was designed to remedy this situation by automating time synchronization across a network.^[16] The NTP home page is http://www.ntp.org. There is also a lot of useful information available at http://www.eecis.udel.edu/~mills/ntp.htm.

^[16] An older mechanism uses the timed daemon. I recommend replacing it with ntpd, which has the advantage of setting all of the clocks to the correct time. timed merely sets them all to the same time as the master server and has no mechanism for ensuring that the time is accurate.

You may wonder how computer clocks get out of synchronization in the first place. Computers contain a oscillator along with some hardware to interface it to the CPU. However, instability in the oscillator (for example, due to temperature changes) and latencies in computer hardware and software cause errors in the system clock (known as wander and jitter, respectively). Thus, over time, the clock settings of different computers that were initially set to the same time will diverge since the errors introduced by their respective hardware will be different.

NTP is designed to deal with these realities in a very sophisticated manner. It has been around since 1980 and was designed and written by Professor David L. Mills of the University of Delaware and his students. This protocol provides time synchronization for all of the computers within a network and is constructed to be both fault tolerant and scalable to very large networks. It also includes features for authentication between clients and servers and for collecting and displaying statistics about its operations. The protocol has a target precision of about 232 picoseconds.

What Is Time?

Here we look at time strictly from a standards point of view.

In 1967, a second was defined as "the duration of 9,192,631,770 periods of the radiation corresponding to the transition between the two hyperfine levels of the ground state of the cesium-133 atom." (Cesium atoms keep busy.) Before 1967, the length of a second was tied to the Earth's rotation, and the exact length of a second would get longer each year. The time standard consisting of these 1967 standard seconds is known as TAI (International Atomic Time).

Coordinated Universal Time (UTC) is the official standard for the current time used by NTP. UTC evolved from the previous standard, Greenwich Mean Time (GMT).

Unfortunately, TAI time does not exactly mesh with how long it really takes the earth to rotate on its axis. As a result, leap seconds are inserted into UTC about every 18 months to maintain synchronization with the planet's slightly irregular and slowing rotation. The leap seconds ensure that, on average, the Sun is directly overhead within 0.9 seconds of 12:00:00 UTC on the meridian of Greenwich, U.K.

8.4.1 How NTP Works

NTP operates in a hierarchical client/server fashion, with authoritative time values moving down from the top-level servers through lower-level servers and then to clients. The entire scheme is based on the availability of what it calls stratum 1 servers: servers that receive current time updates from a known-to-be-reliable source, such as an attached reference clock. Servers that receive time values from these servers are known as stratum 2 servers (and so on down the server hierarchy).

There are several options for obtainingauthoritative time:

• The system can be connected to an external atomic clock.

• You can connect to the National Institute of Standards and Technology (NIST) by modem and receive this data.

• You can use a Global Positioning System (GPS)-based device, which can receive time values as well as positioning information from satellite sources.

• You can obtain the authoritative time values for your network from external stratum 1 NTP servers on the Internet. This is in fact the most common practice for Internet-connected organizations that do not require the extreme precision needed for a few real-time applications (e.g., air traffic control).

  The web page http://www.eecis.udel.edu/~mills/ntp/servers.htm contains links to lists of Internet-accessible stratum 1 and 2 servers. For most sites, a stratum 2 server is sufficient. Note that some servers require advance permission before you may connect to them, so check the requirements carefully before setting up a connection to an Internet NTP server.

In client mode, NTP makes periodic adjustments to the system clock based on the authoritative time data that it receives from the relevant servers. If the current time on the system differs from the correct time by more than 128 milliseconds, NTP resets the system clock. In its normal mode of operation, however, NTP makes adjustments to the system clock gradually by adjusting its parameters to achieve the needed correction. Over time, the NTP daemon on the system records and analyzes successive time errors known as clock drift and continues to correct the time automatically based on this data, even when it cannot reach its time server system. This entire process is known as disciplining the system clock.

In actual practice, NTP requires multiple sources of authoritative time. This strategy is used to protect against both single points of failure and unreliability of any single server (due to hardware failure, malicious tampering, and so on). In other words, NTP views all time data with a certain amount of distrust, and its algorithms prefer at least three time sources. Each distinct server is sampled multiple times, and the NTP algorithms determine the best value to use for the current time from all of this data (naturally taking into account the network latency, the amount of time required for the time value to be transmitted from the remote server to the local system). This value is then used to adjust the time on the local system as described above. In a similar manner, client systems may also be configured to request time data from multiple NTP servers.^[17]

^[17] Readers interested in very accurate time will be interested in this comment from one of the book's reviewers: "One of NTP's few shortcomings is its inability to handle asymmetric path delays. The latest versions of NTP mitigate this using thehuff 'n' puff filter (see the tinker command and huffpuff keyword in the `Miscellaneous Options' documentation)."

All in all, NTP functions extremely well, and all the systems within a network can achieve synchronization to within a few milliseconds of one another. This level of accuracy is more than sufficient for most organizations.

8.4.2 Setting Up NTP

The first step for implementing NTP within your network is, as usual, planning. You'll need to decide several things, including how and where to obtain authoritative time values, the placement of NTP servers within the network, and which clients will connect to which servers. To get started, you might connect one or two local servers to three external stratum 2 servers; the local servers will become the top-level NTP servers within your organization. Then you can connect clients to the servers, and time synchronization will begin.

When things are working, you can move to the suggested configuration of three local servers each connected to three external servers and using a total of at least five external servers. Later, if necessary, you can add top-level servers or even another layer of servers within your organization that use the externally-connected ones as their authoritative time source.

Within an individual system, the NTP facility consists of a daemon process, a boot script, a configuration file, several log files, and a few utilities. Installing it is very easy. You can either download and build the package from source code or install it from a package provided by your operating system vendor.

Once the software is installed, the next step is to configure the facility. NTP's configuration file is conventionally /etc/ntp.conf . Here is a very simple sample file for aclient system:

server      192.168.15.33 logfile     /var/log/ntp driftfile   /etc/ntp.drift

The first line specifies a server to use when obtaining time data, and remaining lines specify locations of NTP's log file and drift file respectively (the latter stores data about local clock errors for future time corrections).

The configuration files forservers also include a server entry for their sources of time data. In addition, they may include lines like this one:

peer 192.168.15.56 key 7

This entry indicates that the indicated server is a peer, a computer with which the local system will exchange send and receive time data. Generally, the top-level servers within the organization can be configured as peers, functioning as both clients and servers with respect to one another and as servers with respect to general client systems. The key keyword is used to specify an authentication key for this connection (discussed below).

If a server has a reference clock connected to it, the server entry within its configuration file is somewhat different:

server 127.127.8.0 mode 5

Reference clocks are usually connected via a serial line, and they are specified with an IP address beginning with 127.127. The final two components of the IP address indicate the type of device (check the device's documentation) and unit number, respectively.

NTP also includes anauthentication facility, which enables clients and servers to verify that they are communicating with known and trusted computers. The facility is based upon a private key scheme; keys are conventionally stored in the file /etc/ntp.keys . This file can contain up to 65,536 32-bit keys. When in use, the facility adds several lines to the configuration file:

keys /etc/ntp.keys trustedkey 1 2 3 4 5 6 7 15 request key 15 control key 15

The first line identifies the NTP key file. The second line activates the indicated keys within the file, and the remaining two lines specify which key to use for NTP queries and configuration changes, where the indicated key functions as a password in those contexts (corresponding to the ntpdc and ntpq utilities, respectively). Once specified and enabled, these keys may be used with the key keyword in server and peer entries.

The most recent versions of NTP also include an additional authentication option referred to as the autokey mechanism. This scheme was designed for NTP's multicast mode, in which time data is broadcast rather than being explicitly exchanged between clients and servers. Using it, clients can generate session keys that can be used to confirm the authenticity of received data.

Once configured, theNTP daemon must be started at boot time. On System V-style systems, this is accomplished via a boot script within the usual /etc/rcn.d script hierarchy (included as part of the NTP package); on BSD-style systems, you will have to add the command to one of the boot scripts.

On client machines, at boot time, the system time may be explicitly synchronized to that of its server by running the ntpdate utility included with the package. This command takes a form something like the following:

ntpdate -bs 192.168.15.56

The -b option says to set the system time explicitly (rather than adjusting it in the normal manner), and the -s option says to send the command output to the syslog facility (rather than standard output). The remaining item on the simple command line is the IP address of a server from which to request the current time. Multiple servers may be specified if desired. Be aware that running ntpdate must take place before the NTP daemon is started. In addition, many application programs and their associated server processes react rather badly to substantial clock changes after they have started, so it is a good idea to perform time synchronization activities early enough in the boot process that they precede the starting of other servers that might depend on them.

Eventually, the command ntpdate will be retired, as its functionality has been merged into ntpd in the most recent versions. The command form ntpd -g -q is the equivalent form, and it queries the time and set the clock to it, exiting afterwards. The server to contact is specified, as usual, in the configurationfile.

8.4.2.1 Enabling ntpd under FreeBSD

FreeBSD systems provide nptd by default. The daemon is started by the rc.network boot script at startup whenever the following variables are set in rc.conf or rc.config.local in /etc:

xntpd_enable="YES"                      Start the ntpd daemon. ntpdate_enable="YES"                    Run the ntpdate command at startup. ntpdate_flags="-bs 10.1.5.22"           Specify options to ntpdate (e.g.,  desired host).

By default, ntpd is disabled.

8.4.3 A Simple Authentic Time Option

For many sites, the usual authentic time options have significant inconveniences associated with them. Reference clocks and GPS devices can be expensive, and using an Internet-based time server can be inconvenient if your connectivity to the Internet is intermittent. At my site, we've found a low-cost and simple solution suitable for our network. It involves using an inexpensive clock that automatically synchronizes to NIST's WWVB time code by receiving its radio transmission.^[18] In my case, the specific clock is an Atomic Time PC Desktop Clock (see http://www.arctime.com under desktop clocks for details), which retails for about $100 U.S. The device is shown in Figure 8-5.

^[18] One reviewer notes, "You can do something similar with a hand-held GPS with a communication port. These usually speak a marine control code, but it is trivial to convert it to RS-232."

Figure 8-5. Atomic Time PC Desktop Clock

Devices of this type can be used as reference clocks using the usual NTP facilities, but this model is not supported. However, for my site, this is not a problem. We use a simpleExpect script to communicate with the device (which is attached to the computer via a serial port) and to retrieve the current time:

#!/usr/bin/expect set clock /dev/ttyS0 spawn -open [open $clock r+] # set the serial line characteristics stty ispeed 300 ospeed 300 parenb -parodd \   cs7 hupcl -cstopb cread clocal -icrnl \   -opost -isig -icanon -iexten -echo -noflsh < $clock send "o" expect -re "(.)" send "\r"  expect -re "(.)" # expect 16 or more characters expect -re "(................*)" exit

The script defines a variable pointing to the appropriate serial line, sets the line characteristics using the stty command, and then communicates with the device via a series of send and expect commands. These tell the clock to transmit the current time and the script displays the resulting data on standard output:

Mon Oct 07 13:32:22 2002 -0.975372 seconds

We then use a Perl script to parse and reconstruct the data into a form required by the date command; for example:

date 100713322002.22

(Remember that date's argument format is mmddhhmmyyyy.ss.) We can then use configuration file entries like the following to set up NTP on that computer:

server 127.127.1.1                     # LCL (the local clock) fudge 127.127.1.1 stratum 12

These lines specify the local system clock as the NTP time source. The server then becomes the authoritative source of time information for all the other systems within the network. These other systems use the standard NTP facility for synchronizing to this time source. This level of ultimate time accuracy is perfectly adequate for our simple needs. However, we set the server's stratum to the highest value so that no one else will consider our time authoritative.

An even simpler alternative is to simply define cron jobs on the other servers to update the time from this master server once or twice a day (using ntpdate or ntpd -g -q). This approach would also avoid the latencies introduced by the spawned subprocesses.