Network Time Services


Accurate timing plays an important role in any multithreaded computer operating system. The system kernel schedules processes, and then it processes events based on time. The system time is also used for time-stamping log entries, application transaction records, email messages, security certificate management, and so on. However, unless your server has a built-in atomic clock, to keep proper time, the system clock must regularly be corrected for drifts. You can perform the correction manually or have the system clock updated automatically via some time protocol over the network. SLES 9 includes Network Time Protocol (NTP) software (xntp v4 package) to assist you in obtaining an accurate time from the Internet as well as a way to provide the same correct time to other servers and clients on your internal network. In this section, we discuss how to set up your NTP client and cover setting up an NTP server for your internal network. First, let's look at some background information.

NOTE

For internal networks, it is customary to set up a time server to provide times to the local systems and have that time server obtain its time setting from one of the well-known Internet time sources, such as the U.S. Naval Observatory.


CAUTION

We cannot stress enough how important it is that the correct time is maintained on your servers. Even if you have only one server that is running Apache (see "Web Services" later in this chapter), having the correct time is essential for secure HTTP connections. Security certificates used for HTTPS connections (or other applications that use a certificate) often contain both "not valid before" and "not valid after" dates. Should your server's time be inaccurate, it could prevent a certificate from being installed or used. And if your web server also offers e-commerce functions, such as shopping carts, accurate time-stamping of transaction records is essential.


You can adjust your server's time in a number of ways based on external NTP time sources. The NTP software suite includes two applications for setting the system's time via NTP: ntpdate and ntpd. At first glance, it seems ntpdate would be the preference because it is simple to set up and use. In its simplest form, the syntax is

 ntpdate timeserver_1 timeserver_2 timeserver_3 ... 

where timeserver_x are either the DNS names or IP addresses of NTP time servers. ntpdate must run as root on the local host. It queries each of the listed servers (multiple samples are taken to obtain an averaged time), determines the time differences between itself and each of the time servers, and sorts them into groups. The largest group of time servers with an identical time (within certain limits) is deemed to possess the "correct time," and ntpdate sets the local time to that of the first server's in that group. The drawback here is the way in which the new time is applied by ntpdate.

Time adjustments are made by ntpdate in one of two ways. If the time difference is ±128 ms or more, ntpdate will simply "step" the local time to match that of the NTP server's. If the time difference is less than ±128 ms, ntpdate will "slew" the time by making small incremental adjustments until the times match. As you can see, ntpdate could cause a sudden time change that can upset the proper operation of some applications, such as cron. Although it takes a little more work to set up, the better alternative is to use ntpd instead; ntpdate is due to be retired from the NTP distribution anyway because all its functions are available within ntpd.

CAUTION

You can use the -B switch to force ntpdate to always slew the time instead of stepping it for large time offsets. However, if the time difference is much larger than ±128 ms, slewing the clock to reach the correct time could take hours. Additionally, if the difference between the current date and the system date is too large, ntpdate can set the year to 1933 due to arithmetic overflow.


NOTE

If you're familiar with the NetWare TimeSync protocol, ntpdate's step operation is exactly the way a Secondary time server (which takes time from a Primary time server or a Reference time server) and a time client behave: The time is adjusted 100% instead of a fraction at a time. The slew operation mirrors the way in which a Primary time server adjusts its time: The time is adjusted by 50% of the difference every time until it is synchronized.


ntpd operates by exchanging messages with one or more configured time servers at designated poll intervals. To protect the network from traffic bursts, the initial poll interval for ntpd to each time server is delayed an interval randomized over a few seconds. The default initial poll interval is 64 seconds, and, consequently, several minutes can elapse before the local clock is set. (You can reduce the initial delay to set the clock by using the iburst keyword with the server configuration command.) By measuring the network roundtrip jitter and local clock frequency drift, ntpd will slowly increase the polling interval from 64 seconds to 1,024 seconds (just a little over 17 minutes). Also, if a target time server is unreachable for some time, the polling interval is also increased in step to 1,024 seconds. This helps to reduce network traffic.

NOTE

Much of the SUSE documentation, even the Novell training material to some extent, makes reference to xntpd. However, the current NTP distribution documentation uses ntpd instead because it reflects NTPv4, while xntpd was used for NTPv3. As a matter of fact, you will find on your SLES server that /usr/sbin/xntpd is actually a symbolic link to /usr/sbin/ntpd. Therefore, we use ntpd here.


Under normal conditions, ntpd adjusts the clock in small increments (that is, slewing it) so that the time scale is effectively continuous and without discontinuities. In case of extreme network congestion, ntpd may end up stepping the time; however, this situation is extremely rare because ntpd employs a number of algorithms to try to slew the clock first and steps it only after all recovery attempts have failed.

NOTE

ntpd has a built-in safeguard: If its local time differs by 1,000 seconds (16.67 minutes) or more with that of the time server's, ntpd will immediately exit and log a panic message to the system log indicating an "insane" time source.


Once installed, ntpd is set up to automatically run at server bootup. If, for some reason, it does not, you can use the insserv command as follows:

 Athena:/home/admin # /sbin/insserv /etc/init.d/xntpd 

NOTE

Instead of insserv, you can also use the Runlevel Editor in YaST to enable the service. To access this editor, select Control Center, System, Runlevel Editor or use yast runlevel.


Keep in mind that because both ntpd and ntpdate use port 123, they cannot be run concurrently on the same host.

Configuring the NTP Client

Ordinarily, ntpd reads /etc/ntp.conf at startup time to determine the synchronization sources and operating modes. Although this approach is not recommended because it is cumbersome, you can specify a working, although limited, configuration entirely on the command line, obviating the need for a configuration file. This approach may be particularly useful when the local host is to be configured as a broadcast/multicast client, with all peers being determined by listening to broadcasts at runtime.

CAUTION

The configuration filename for the NTP daemon (ntpd) is ntp.conf, and not ntpd.conf, as you may expect.


The configuration file format is similar to other Linux/Unix configuration files: Comments begin with a # character and extend to the end of the line; blank lines are ignored. Configuration commands consist of an initial keyword followed by a list of arguments, some of which may be optional, separated by whitespace. Commands may not be continued over multiple lines. The default /etc/ntp.conf file as included with the xntp package in SLES 9 is shown in Listing 8.2.

Listing 8.2. The Default /etc/ntp.conf File
 ########################################################### ## /etc/ntp.conf ## ## Sample NTP configuration file. ## See package 'xntp-doc' for documentation, Mini-HOWTO ## and FAQ. ## Copyright (c) 1998 S.u.S.E. GmbH Fuerth, Germany. ## ## Author: Michael Andres,  <ma@suse.de> ## ########################################################### ## ## Radio and modem clocks by convention have addresses in the ## form 127.127.t.u, where t is the clock type and u is a unit ## number in the range 0-3. ## ## Most of these clocks require support in the form of a ## serial port or special bus peripheral. The particular ## device is normally specified by adding a soft link ## /dev/device-u to the particular hardware device involved, ## where u correspond to the unit number above. ## ## Generic DCF77 clock on serial port (Conrad DCF77) ## Address:     127.127.8.u ## Serial Port: /dev/refclock-u ## ## (create soft link /dev/refclock-0 to the particular ttyS?) ## # server 127.127.8.0 mode 5 prefer ## ## Undisciplined Local Clock. This is a fake driver ## intended for backup and when no outside source of ## synchronized time is available. ## server 127.127.1.0  # local clock (LCL) fudge 127.127.1.0  stratum 10 # LCL is unsynchronized ## ## Outside source of synchronized time ## ## server xx.xx.xx.xx      # IP address of server ## ## Miscellaneous stuff ## driftfile /var/lib/ntp/drift/ntp.drift #path for drift file logfile /var/log/ntp       # alternate log file # logconfig =syncstatus + sysevents # logconfig =all # statsdir /tmp/        # directory for statistics files # filegen peerstats  file peerstats  type day enable # filegen loopstats  file loopstats  type day enable # filegen clockstats file clockstats type day enable # # Authentication stuff # # keys /etc/ntp.keys           # path for keys file # trustedkey 1 2 3 4 5 6 14 15 # define trusted keys # requestkey 15   # key (7) for accessing server variables # controlkey 15   # key (6) for accessing server variables 

You can make a simple client configuration by adding the list of NTP time servers that you want ntpd to contact:

 ## ## Outside source of synchronized time ## server 10.1.1.1 prefer        # IP of local time server server time.nrc.ca            # DNS name of Ottawa server server dense.utcc.utoronto.ca # University of Toronto #server ntp-1.cede.psu.edu     # Pennsylvania state #server timex.cs.columbia.edu  # New York state #server tick.jrc.us            # New Jersey state 

NOTE

The prefer option in the server command indicates that this server's time should be used for synchronization, if all things are equal (such as time jitter) from among the specified set of time sources.


For redundancy, it is recommended that you use three to five NTP sources, located on different networks. When using Internet NTP time servers, select ones that are located geographically close to you but not at the same location. The preceding sample entries show that, in addition to the local time server, ntpd also contacts two other servers located in nearby cities.

TIP

For a list of public NTP time servers, visit http://ntp.isc.org/bin/view/Servers/WebHome. Be sure to read the "Rules of Engagement" (http://ntp.isc.org/bin/view/Servers/RulesOfEngagement) before you select your time sources.


CAUTION

You should ensure the following two lines in your /etc/ntp.conf are not commented out or modified:

 server 127.127.1.0  # local clock (LCL) fudge 127.127.1.0  stratum 10 # LCL is unsynchronized 


They provide a fallback using the local system clock as a pseudo time source if none of the NTP sources you have listed are available, due to an ISP service interruption, for example. At the very minimum, the stratum setting should not be set to a value less than 5. (A discussion about stratum can be found in the "Configuring an NTP Server" section later in this chapter.)

If your NTP time source can be reached via broadcast or multicast, you don't need to use server names. Instead, enter either the broadcastclient or multicastclient command in the configuration file. To prevent a rogue time server in the network from changing your server's time, configure the authentication mechanisms listed under the Authentication stuff section in /etc/ntp.conf; refer to /usr/share/doc/packages/xntp-doc/html/authopt.html for details.

NOTE

No man page is available for the NTP distribution. However, you will find an xntp-doc package containing HTML documentation. When you install the package, the HTML files and related FAQs and README files are installed into /usr/share/doc/packages/xntp-doc.


ntpd uses a file (whose name is defined by the driftfile keyword in the configuration file) to record the "drift" (or frequency error) value it has computed. If the file exists on startup, it is read and the value used to initialize ntpd's internal value of the frequency error. The file is then updated once every hour by replacing the old file with a new one containing the current value of the frequency error. Note that the file is updated by first writing the current drift value into a temporary file and then using the rename function to replace the old version. This implies that ntpd must have write permission for the directory where the drift file is located, and that file system links, symbolic or otherwise, should be avoided when specifying the value for the driftfile keyword.

ntpd reads /etc/ntp.conf upon startup and then once every hour to look for changes. If ntpd is already running and you made changes to /etc/ntp.conf but do not wish to wait, you can use the command /etc/init.d/xntpd restart (or /etc/init.d/xntpd force-restart) to make ntpd reread the configuration file.

TIP

Instead of /etc/init.d/xntpd, you can use /usr/sbin/rcxntpd because it is just a symbolic link to /etc/init.d/xntpd, a shell script. As a matter of fact, you will find rcxntpd referred to often in the documentation; using it instead is probably more convenient because /usr/sbin is in root's PATH setting, while /etc/init.d is not.


If ntpd isn't running and you want to manually start it, use the command /etc/init.d/xntpd start; to stop it, use /etc/init.d/xntpd stop.

Because of the number of optional keywords possible for the configuration, you will find it is actually much easier to manually edit /etc/ntpd.conf instead of using YaST. But if you insist, there is a YaST module to configure the NTP client; you'll find a package called yast2-ntp-client installed. To access the NTP Client Configuration dialog box (see Figure 8.3) from the Control Center, select Network Services, NTP Client. Or you can open this dialog box directly by using yast2 ntp-client or yast ntp-client.

Figure 8.3. NTP client configuration screen.


The default dialog box allows you to specify a single host as the time source. To use multiple sources, click Complex Configuration and then use the Add button to create an entry for each desired time server. If you already have at least one time source defined (other than the local clock), the Complex Configuration screen is displayed automatically.

NOTE

One of the options in the Complex NTP Client Configuration dialog box is called Run NTP Daemon in Chroot Jail. chroot jail (or Change root jail) is a way to limit the portion of the file system that the daemon can see; this is similar to using MAP ROOT in NetWare. By default, ntpd runs in chroot jail mode. If, for some reason, you don't want ntpd in chroot jail, edit either /etc/init.d/rcxntpd or /etc/sysconfig/xntp and change the XNTPD_RUN_CHROOTED setting from yes to no.

Refer to Chapter 13, "System Security," for further discussions about chroot jails.


CAUTION

Even if you are running ntpd in chroot jail, the copy of ntp.conf to edit is still the copy in /etc, and not the copy in the jail's /etc directory (which, by default, is /var/lib/ntp/etc). That copy is updated automatically by ntpd's startup script.


When you use YaST to set up your NTP client configuration, it also updates the XNTPD_INITIAL_NTPDATE variable (used by ntpd upon system startup to obtain the current date and time information) setting in /etc/sysconfig/xntp. However, if you manually edited /etc/ntp.conf outside YaST, the XNTPD_INITIAL_NTPDATE setting may contain obsolete host data until the next time YaST is used to update the NTP client settings. The following example illustrates the telltale sign of this happening when you manually start or restart ntpd from a terminal session:

 Athena:/home/admin # rcxntpd start Try to get initial date and time via NTP from ticker failed Starting network time protocol daemon (NTPD)           done 

Ticker is a time source that has been removed from the network, and its server command entry was manually removed from /etc/ntp.conf without using YaST. As a result, /etc/sysconfig/xntp contains the outdated information (XNTPD_INITIAL_NTPDATE = ticker) leading to the failure message.

Configuring an NTP Server

ntpd can function as an NTP client or as an NTP server or as both concurrently. This allows a single server in the network to run ntpd to obtain time from an authoritative time source (either from an NTP server on the Internet or a locally attached reference clock); then it redistributes that information to other servers (running ntpd in client-only mode) internally. This reduces traffic to the external time source and ensures all NTP clients use a common time source.

There are two ways in which an NTP client can locate an NTP server. The first method is simply to statically configure either the IP address or the DNS name of the NTP source in the client's /etc/ntp.conf file. The other method is to set up the NTP time server to broadcast or multicast its time onto the network and configure the clients to listen for the broadcasted/multicasted time information.

To run ntpd with broadcasts or multicasts, add the broadcast address command to the configuration file. In broadcast mode, the address should be the broadcast address on (one of) the local network(s), such as 192.168.1.255. To enable multicast mode, the address should be 224.0.1.1 (for IPv4) or ff05::101 (site local, for IPv6)the addresses assigned to NTP. Multiple broadcast commands may be specified to support multiple local network interfaces or multiple multicast groups.

Before you allow ntpd to act as a time source, you must ensure the server has the correct time. In most cases, you accomplish this by obtaining its time from one or more of the standard Internet NTP servers. This also can be done with reference clocks. The NTP distribution includes hardware drivers for more than 40 radio and satellite clocks and modem services that can be used as reference clocks. A list of supported drivers can be found in /usr/share/doc/packages/xntp-doc/html/refclock.html. It is also possible to use an otherwise "undisciplined" local clock as a primary or backup time source. The following discussion presents information on how to set up a reference clock, using the local clock as an example.

NOTE

Many radio reference clocks can be set to display local time as adjusted for time zone and daylight saving mode. When used with NTP, the clock must be set for Coordinated Universal Time (UTC) only. The OS kernel performs the conversion to local time, so the fact that the clock runs on UTC will be transparent to the users.


The clocks are specified in /etc/ntp.conf as though they are real-time servers on the network using the server keyword. For this purpose, they are assigned special IP addresses in the form 127.127.t.u. In the address, t specifies the clock type and u is a unit number in the range 03 that distinguishes multiple instances of clocks of the same type. For example, the server's hardware clock will have a pseudo IP address of 127.127.1.0.

NOTE

You can find a list of supported clock type values (t) in /usr/share/doc/packages/xntp-doc/html/refclock.html.


Normally, the individual drivers have special parameters that describe their configuration details. The refclock.html file mentioned in the preceding note provides links to the different driver pages describing these parameters. For instance, a TimeBrick DCF77 receiver module uses the Type 8 Generic Reference Driver (PARSE). But because this Type 8 driver supports a number of different receiver modules, you need to include additional parameters to tell the driver which specific receiver module you are using. The complete server command for a TimeBrick DCF77 receiver module would be

 server 127.127.8.0 mode 6 

With the exception of the local clock, most of the radio reference clocks require support in the form of a serial port or special bus peripheral. You need to associate the clock driver to the particular hardware device by manually creating a symbolic link between them. For example, if you have a TimeBrick DCF77 receiver connected to your server's serial terminal port 1 (/dev/ttyS1), link the TimeBrick's clock device (/dev/refclock-u, where u is 0 for the first TimeBrick device) to the serial port as follows:

 ln -s /dev/ttyS1 /dev/refclock-0 

NOTE

The clock device names are listed on their respective driver pages, under the Synopsis heading at the top of each page.


The local clock driver is also a special case. A server configured with this driver can operate as a primary server to synchronize other clients when no other external synchronization sources are available. If the server is connected directly or indirectly to the Internet, there is some danger that it can adversely affect the operation of unrelated clients. For this, you would configure this driver at a stratum greater than any other likely sources of time (say 3 or 4) to prevent the server from taking over when legitimate sources are still available.

By default, the stratum for this driver is set at 5, but can be changed by the fudge configuration command and/or the ntpdc utility. As a matter of fact, the default /etc/ntp.conf sets its stratum to 10.

WARNING

You should never configure an NTP time server that might "devolve" to use its local clock to use multicast mode or broadcast mode.


The following summarizes the procedure necessary for setting up an NTP time source:

1.

If using a hardware reference clock, configure and connect the device to your server per manufacturer instructions.

2.

Update /etc/ntp.conf with the appropriate server 127.127.t.u command to support the installed clock.

3.

Create a symbolic link between the clock device name and the hardware device driver.

4.

Configure the startup scripts to run ntpd upon server boot using either insserv or the Runlevel Editor in YaST.

5.

Start up or restart ntpd for the updated ntp.conf to take effect.

Troubleshooting Tips

Most of the time, ntpd is a "configure-and-forget" utility, especially if you are running it in the time consumer (a.k.a. client) mode. The most common oversight when it comes to using public NTP time sources is due to your firewall setting: NTP requires UDP port 123 to be opened in both directions. Unlike other client/server-type protocols such as Telnet, where the client uses a high-numbered port (1024 or higher), both the NTP client and server use port 123.

Sometimes the problem may be a misconfigured ntp.conf entry causing ntpd to obtain its time from a wrong server, or something has gone astray with the time source itself. The NTP distribution includes a couple of utilities that are helpful in verifying the correct operations of ntpd running in time server mode: ntpq (NTP query) and ntpdc (a special NTP query program utility). The ntpq program implements the Mode 6 control message formats defined in the NTP specification (http://www.ietf.org/rfc/rfc1305.txt). The ntpdc program, on the other hand, implements additional functions not provided for in the NTP standard. Both programs can be used to inspect the state variables defined in the specification and, in the case of ntpdc, additional ones intended for serious debugging. In addition, ntpdc can be used to dynamically reconfigure and enable or disable some functions while the ntpd is running.

Both ntpq and ntpdc can be run either in interactive mode or controlled using command-line arguments. For quick troubleshooting, it is easiest to run them using the -p switch; the following shows sample output from ntpq:

 Athena:/home/admin # ntpq -p      remote           refid st t when poll reach delay offset  jitter =====================================================================  LOCAL(0)        LOCAL(0)   10 l    7   64    1  0.000  0.000   0.002  dense.utcc.utor .INIT.     16 u    -   64    0  0.000  0.000 4000.00 

NOTE

You can run either ntpq or ntpdc on the server itself or from another machine elsewhere in the network (by specifying the host address on the command line; the default is localhost).


The first column of the output (immediately before the remote host name) contains the "tally code" character, which indicates the status of the associated host that is a result of the clock selection process. A * means the server is the current time source with which system time is compared; a + shows the host is a candidate for the clock-combining algorithm; a - indicates the host is discarded by the clustering algorithm. You can find further information about the different tally codes in the ntpq documentation (/usr/share/doc/packages/xntp-doc/html/ntpq.html).

NOTE

To provide fault tolerance, a time server usually operates with a number of remote servers. NTP uses a set of algorithms to refine the data from each peer separately and to select and combine the data from a number of sources. First, a "short list" of time sources is created using a clock-selection process that checks for their sanity and computes a confidence level for each source. Ordinarily, the members of this set could in principle provide correct time; however, due to various error contributions, not all can provide the most accurate and stable time. The job of the clustering algorithm, which is invoked at this point, is to select the best subset of the survivors providing the least variance in the combined ensemble average, compared to the variance in each member of the subset separately. You can find detailed discussions of the various algorithms used by NTP, such as for clock selection, clustering determination, and clock-combining procedures, in RFC 1305 (http://www.ietf.org/rfc/rfc1305.txt).


The following summarizes the meaning of the rest of the columns:

  • remote shows the hostname or IP of the remote machine. (If you run ntpq with the -n switch, IP addresses will be displayed instead of DNS names.)

  • refid identifies the time source to which the remote machine is synced. If it shows an IP address or a hostname, that server is being synchronized with another NTP server. In the preceding example, it shows tick.nrc.ca is being synchronized with a GPS clock.

    NOTE

    Until the stratum of the remote host is identified (that is, not 16), the refid column displays a four-character string called the kiss code as a way to help debug the situation. Some of the common kiss codes are INIT (has not yet synchronized for the first time), AUTH (server authentication failed and is waiting for a retry), and DENY (access denied by remote host). You can find a detailed list of kiss codes in /usr/share/doc/packages/xntp-doc/html/debug.html.


  • st shows the stratum of the remote machine. A stratum of 16 means "unsynchronized," or the status cannot be determined. A stratum value of 0 is best, but you will never see one displayed by ntpq.

NOTE

In the world of NTP, stratum levels define the distance from the reference clock. A stratum-0 time source has a reference clock, such as a radio clock or its own cesium clock. Stratum-0 servers cannot be used on the network. Instead, they are directly connected to computers that then operate as stratum-1 servers. The basic definition of a stratum-1 time server is that it is directly linked (not over a network path) to a reliable source of UTC time (a stratum-0 server). In turn, a stratum-1 time server acts as a primary network time standard.

In essence, the farther away a time server is from a stratum-1 server, the higher stratum level is assigned to that time server. As you progress through different strata, there are network costs (transmission delays) involved that reduce the accuracy of the NTP server in relation to UTC. A stratum-1 time server will typically be less than a millisecond off from UTC. Over the Internet, because of network delays, a stratum-2 time server could be 10100 ms off from UTC, and each subsequent time server will add an additional 10100 ms of inaccuracy. For most applications, obtaining your time from a stratum-2 server is more than sufficient.


  • t shows the type of communication method used: u = unicast, m = multicast, l = local, - = unknown.

  • when displays the number of seconds since the last poll of the remote machine.

  • poll gives the polling interval in seconds.

  • reach shows the "reachability register" in octal, with bits entering from the least significant (rightmost) end. A peer is considered reachable if at least one bit in this register is set to one.

    This is a "rotating" register, meaning it tracks the last eight poll requests. For instance, a decimal value of 276 (binary pattern 10111110) means that out of the last eight polls, two (including the very latest poll attempt) received no response from the remote machine.

  • delay shows the time delay (in milliseconds) to communicate with the remote.

  • offset shows the offset (in milliseconds) between local time and that of the remote.

  • jitter gives the observed time error (in milliseconds) of the system clock with the remote as an average of RMS (root mean square) time differences. (In pre-NTPv4, this column was called dispersion.)

Another handy troubleshooting tool included in the NTP distribution is ntptrace. It determines where a given NTP server gets its time from and follows the chain of NTP servers all the way back to their master time source. If ntptrace is run with no arguments, it starts with localhost. Here is an example of the output from ntptrace:

 Athena:/home/admin # ntptrace localhost: stratum 16, offset 0.000000, synch distance 0.000000 time.nrc.ca: stratum 2, offset 0.002815, synch distance 0.03061 toc.nrc.ca: stratum 1, offset 0.002153, synch distance 0.00156, refid 'PPS' 

On each output line, the fields are (left to right):

  • Host name

  • Host stratum

  • Time offset between that host and the local host as measured by ntptracethis is why it is not always zero for localhost

  • Host synchronization distance and (only for stratum-1 servers) the reference clock ID

All times are given in seconds.

For additional troubleshooting and debugging tips, such as enabling debug and verbose output for ntpd, consult the NTP distribution's HTML documentation, especially /usr/share/doc/packages/xntp-doc/html/debug.html.



    SUSE LINUX Enterprise Server 9 Administrator's Handbook
    SUSE LINUX Enterprise Server 9 Administrators Handbook
    ISBN: 067232735X
    EAN: 2147483647
    Year: 2003
    Pages: 134

    flylib.com © 2008-2017.
    If you may any questions please contact us: flylib@qtcs.net