Project42.Manage the System Log Daemon

Project 42. Manage the System Log Daemon

"How do I control what system messages are logged, and to which file they are logged?"

This project looks at the system log server (or system log daemon) to which log messages are sent. It shows how the default configuration for this daemon might be changed, how to send log messages manually, and how to view log messages. Project 41 looks at viewing log files by using more traditional Unix tools, such as tail.

Understand Logging

As we saw in Project 41, your Mac runs many background processes, such as the system components of OS X itself, the Apache Web server, and the firewall. When these components have something to report, they write to a log file. Major servers such as the Apache Web server control their own logging. Others, such as mail and FTP (File Transfer Protocol), and system components such as crontab and the kernel log messages through the system log daemon (syslogd).

The System Log Daemon

The syslogd daemon is the central point to which log messages may be sent. A process using syslogd need not take responsibility for contention (more than one process writing to the same log file at the same time) or configuration issues such as what messages should be logged and to where.

A message sent to syslogd generally includes a facility, which is an indication of the originating system, and a level, which is an indication of the urgency. The syslogd daemon filters and channels messages based on these two parameters.

Eight levels are defined. In decreasing order of severity, they are

• Error levels: emergency, alert, critical, error

• Warning and information levels: warning, notice, informational, and debugging

Facilities include

• Daemons for system components: authorization, cron, FTP, kernel, and mail

• General-purpose facilities: user and locally defined local0 to local7

The syslogd daemon directs messages by considering their associated level and facility. For example:

• All messages with facility ftp, whatever their level, go to /var/log/ftp.log.

• All messages with a level of notice and above, whatever their facility, go to /var/log/system.log.

• Messages from netinfo with a level of error and above go to /var/log/netinfo.log.

The three examples above can be expressed in the form facility.level log-file. A star (*) in place of a facility or level name denotes "all."

• ftp.* /var/log/ftp.log

• *.notice /var/log/system.log

• netinfo.err /var/log/netinfo.log

The configuration file for syslogd is in /etc/syslog.conf. Display it, and you'll see how the examples above fit in. A semicolon-separated list of facility.level pairs, or selectors, is permissible to channel several categories to the same log file.

Here's an extract of the file:

Tip

Type

$ man 5 syslog.conf

for more information about the syslog configuration file.

$ cat /etc/syslog.conf ... *.notice;authpriv,remoteauth,ftp,install.none;   kern.debug;mail.crit  /var/log/system.log ... mail.*                  /var/log/mail.log ftp.*                   /var/log/ftp.log netinfo.err             /var/log/netinfo.log install.*               /var/log/install.log install.*               @127.0.0.1:32376 local0.*                /var/log/ipfw.log *.emerg                 *

Note

Other shells, such as Tcsh, use the syntax `command` instead of $(command).

Configure syslogd

We change configuration settings by editing the file /etc/syslog.conf and restarting the daemon. Suppose that we wish to log errors (or, more precisely, all messages with a level of error or more severe) to the file /var/log/my.log. First, we edit the configuration file /etc/syslog.conf, giving an administrator password when prompted.

Note

It's necessary to create the log file before restarting syslogd. This is not necessary when syslogd is stopped and started from scratch.

$ sudo nano /etc/syslog.conf $ Password:

Add this line to the end of the file (making sure to terminate it by pressing Return).

*.err                 /var/log/my.log

Learn More

Refer to "How to Become the Root UserHow to Become the Root User" in Project 2 for more information on the sudo command.

Next, create an empty log file, and restart syslogd by typing

$ sudo touch /var/log/my.log $ sudo kill -HUP $(head -1 /var/run/syslog.pid)

Naturally, you'll want to test this new logging configuration. You can use the wait-and-see approach, waiting for an error message to be logged and then viewing my.log. Alternatively, send an appropriate message to syslogd directly (see the next section, "Send Log Messages").

Learn More

Project 30 covers the nano text editor, and Project 40 explains how to restart daemons by using kill -HUP.

Send Log Messages

Suppose that we wish to test our new syslogd configuration. To this end, we have two utilities at our disposal:

• Unix's logger utility

• Apple's own syslog utility

Tip

Send an urgent message to all users and all terminal sessions by typing

$ logger -p user.emerg ¬     Urgent message to ¬     all users ...

Let's use logger to send a log message. We specify option -p followed by the selector facility.level. To test our new selector *.err, we send a log message with a level of error (or above) and with any facility. Let's send an error message purporting to come from the mail system. Type

$ logger -p mail.err This is an error message

Now view the log file.

$ tail /var/log/my.log Jul 10 20:14:22 saruman saruman: This is an error message

Alternatively, we might send a critical message by using the local1 facility.

$ logger -p local1.crit This is a critical message $ tail /var/log/my.log Jul 10 20:14:22 saruman saruman: This is an error message Jul 10 20:15:37 saruman saruman: This is a critical message

Apple's syslog

The syslog command is an Apple utility written to perform a variety of tasks relating to the system logging facility. It lets you send messages (just like logger, described in the preceding section), view messages, and prune (remove) unwanted messages.

syslogd in Tiger

In OS X 10.4 (Tiger), Apple replaced the BSD syslogd with its own compatible, but extended, daemon of the same name. Apple's variant holds messages in an internal database as well as sending them to the traditional log files. It's this database that syslog queries, watches, and prunes.

The syslog utility is also new in Tiger. In earlier versions, you have to use logger.

Let's use syslog to send a log message to syslogd. We specify option -s to send and option -k followed by any number of keywordvalue pairs to describe the selector and the message. We might send a mail system (Facility mail) error (Level error) message (Message "message text...") by using the following command.

$ syslog -s -k Facility mail Level error Message ¬     "Mail system error."

The default syslogd configuration has this message sent to both the system and mail log files, as we can see by tailing both files.

Note

A log message that is sent to two or more log files will nevertheless appear just once in the syslog message database.

$ tail -n1 /var/log/system.log Jul 11 13:50:39 localhost syslog[4450]: Mail system error. $ tail -n1 /var/log/mail.log Jul 11 13:50:39 localhost syslog[4450]: Mail system error.

The syslog command can read and filter the message database. Let's search for the message we just sent and all others from facility mail at level error. (Note that -k is required before every keywordvalue pair, unlike the send-message example above, which allows a list of pairs to follow a single -k switch.)

Tip

The syslog command has a tail -f mode of operation enabled by the option -w.

$ syslog -k Facility mail -k Level error ... Jul 11 13:50:39 syslog[4450] <Error>: Mail system error.

As a final example, let's display all critical error messages from the secure shell server (sshd). We specify that Sender must be sshd and Level must be critical.

$ syslog -k Sender sshd -k Level critical Jul  9 03:31:58 Sauron sshd[27172] <Critical>: fatal: Timeout before authentication for 216.138.221.246

The syslog command is capable of quite complex message sending and retrieval. We display all mail logs for which the level is error or less severe (has a greater numerical value than) by typing

$ syslog -k Facility mail -k Level ge error

Read the syslog man page for full details.

It can also prune the message database to remove unwanted messages and stop the database from getting too big. Check out the Unix man page for more information.

Send Messages to Another Host

You'll need admin access to two Macs to follow this example.

Sometimes, you may wish to employ a logging policy whereby all machines log to a central server. Let's do this, logging all error messages to the host 217.155.168.146. (In "Configure syslogd," earlier in this project, we sent the same set of log messages to the file /var/log/my.log.)

First, we must configure the server to accept log messages from the network. To do this, we relaunch syslogd, specifying option -u. The syslogd daemon is one of the services launched by Apple's launch daemon launchd (but not in versions before Mac OS X 10.4). Edit launchd's configuration settings for syslogd by using a text editor such as nano, adding the line "<string>-u</string>".

$ sudo nano ¬     /System/Library/LaunchDaemons/com.apple.syslogd.plist $ Password:

After this

<string>/usr/sbin/syslogd</string>

Add this

<string>-u</string>

Then tell launchd to reload (stop and restart) syslogd.

$ sudo launchctl unload ¬     /System/Library/LaunchDaemons/com.apple.syslogd.plist $ sudo launchctl load ¬     /System/Library/LaunchDaemons/com.apple.syslogd.plist

Finally, check that syslogd has indeed been launched with the -u option specified by typing

$ ps ax | grep syslogd 29287 ?? Ss 0:03.73 /usr/sbin/syslogd -u

Make sure UDP traffic isn't blocked by your firewall (System Preferences, Sharing, Firewall tab, Advanced... button).

On the client machine(s), we edit /etc/syslog.conf so that all error message are logged to the server. Follow the instructions given in "Configure syslogd" earlier in this project, but add the following line instead (specifying the IP address of one of your servers, not mine, please!).

Tip

To send all log messages to a central server and have none accepted locally, configure the clients with a syslog.conf file containing just

*.* @Server-IP-address

*.err @217.155.168.146:514

Now restart syslogd.

$ sudo kill -HUP $(head -n1 /var/run/syslog.pid)

We'll use logger to send an error-level log message, which should appear in system.log on the local machine and on the server if our changes have worked correctly.

$ logger -p local0.err This is a cross-host test message

Warning

Your machine is vulnerable to attack when UDP port 514 is open. A malicious remote server could flood you with bogus log messages. Although my server has this port open, the firewall facing the Internet does notso don't even think about it.