System Logs

Out of the box a Mac OS X computer logs much of the activity that happens on the system. Most events that involve sharing in any way over the network such as Web or FTP server, and most events that involve changes happening on the system are logged by the system. Logs are often overlooked and are only consulted in the event of a system failure, security breach or other catastrophic event. Familiarize yourself with the types of items your system logs, and the content of those logs when things are normal. If you have a good idea of what goes on under normal circumstances, you will know when things are happening that are out of the ordinary. The log files are simply text files, and there are a variety of ways of interacting with these logfiles. We will explain how the system decides what to log and where, and make some changes to the defaults to highlight important security related events. We’ll discuss both traditional Unix tools to view and parse the logs, as well as the included Console Application and some third party GUI tools for reading logfiles.

Introducing Syslog

Syslog is the System Event Logger originally written by Eric Allman. Syslog is a standard component of Unix and other *nix systems. Rather than making each application author worry about where how and when to log, and burdening users with the concern about finding each individual program’s log files, syslog offers developers simple logging routines to access from their programs in the form of openlog and other libraries. Syslogd is the logging daemon which is launched at system startup by the startup item /System/Library/StartupItems/SystemLog. The file /etc/syslog.conf is read by the syslog daemon as it starts up, and configures how the daemon processes incoming messages. There is also a user command called logger that can be used interactively or by shell scripts to create entries in the system log.

Programs that take advantage of syslog for logging, and this includes the Mach Kernel, write their messages to a special file, which only the syslog daemon reads. Syslog parses the incoming messages according to its configuration file and takes one or more of these four actions:

• Forward the message to a syslog daemon running on a different host

• Append the message to a specified file

• Output the message to /dev/console

• Output it to the screen of defined users if they are logged in

The syslog.conf file defines how messages are handled based on two specific criteria called levels and facilities. The levels refer to the severity of an event, and the facility refers to the specific program running. Both the levels and facilities supported by Mac OS X are described in the man page for syslogd.conf.

Configuring Syslog to separate interesting messages

The directory /var/log is where most logfiles are kept on a Mac OS X system. Syslog is configured by default to log most messages to the file /var/log/system.log. Separate logfiles are created by default for authorization actions (generally logins), lpr printer messages, mail, ftp and netinfo errors. This is a reasonable set of defaults. However, in the interest of securing your system, and as a way of example we’ll make modifications to the configuration to isolate messages from both sudo and ssh.

All lines in the /etc/syslog.conf take the form of : selector <TAB> action. It is critical that in the examples below and in any changes you make to the file that you use a tab between items on a line, or unpredictable results may occur. Before making any changes to the syslog.conf file, make a backup of the original file, just in case. To backup the file open the Terminal application and enter the following command : sudo cp /etc/syslog.conf /etc/syslog.orig.conf.

Separating sudo messages

Sudo is configured to log messages to a facility called local2. It just so happens that sudo is the only program installed in a default system that uses the facility local2. This makes it a trivial matter to isolate messages generated by sudo. Open /etc/syslog.conf in your favorite editor and make the following addition to the file:

local2.*               /var/log/sudo.log

In this example, we’ve told syslog to select all messages from the facility local2 at all severity levels and write those messages to the file /var/log/sudo.log.

Syslog will not create a file by itself, so we’ll have to create the logfile. Additionally we’ll set the proper permissions on the file, and finally we’ll restart syslog to force it to reread its configuration file. Open the Terminal application and type the following commands:

sudo touch /var/log/sudo.log  sudo chmod 640 /var/log/sudo.log  sudo kill -HUP `/bin/cat /var/run/syslog.pid`

Separating ssh messages

If you are allowing connections to your system via ssh, you’ll want to log those connections so you can monitor them. In order to do this we’ll need to modify both the sshd configuration file and the syslog configuration file.

First open the ssh server configuration file located at /etc/sshd_config with sudo and your favorite editor and change the value of SyslogFacility from AUTH to LOCAL7. Ensure that you’ve removed the # character from the beginning of the line. Then remove the # character from the LogLevel line immediately below. Save the file and close it.

Next, open /etc/syslog.conf in your favorite editor and make the following addition to the file:

local7.*               /var/log/sshd.log

In this example we’ve told syslog to select all messages from the facility local7 at all severity levels and write those messages to the file /var/log/sshd.log.

Again, syslog can not create files in the /var/log directory for us, so we’ll go create the logfile with proper permissions, then we’ll restart syslog to force it to reread the configuration file. Open the Terminal application and type the following commands:

sudo touch /var/log/sshd.log sudo chmod 640 /var/log/sudo.log sudo kill -HUP `/bin/cat /var/run/syslog.pid`

Viewing system logs

Now that we’re familiar with syslog and the configuration of system messages written to log files, we’ll need to examine these files to see what kind of events are occurring on our computer and ensure that everything is going the way we expect it to be. Here we have several options. We can browse the log files in the Terminal application or use the Console application to view log messages. For debugging new services, and other system occurrences, looking at the log file in real time can be extremely helpful. A new feature in Panther is that Apple System Profiler also includes a log file viewer , as shown in Figure 26-12. This is useful if you need to provide copies of log files for technical support types.

Figure 26-12: Using Apple System Profiler to view system logfiles.

Using Terminal to view logfiles

Tail, less, cat and grep are all very useful when it comes to viewing and searching logfiles.

Use the tail command to view the end of a logfile by typing tail /var/log/system.log. Tail can be used to continuously monitor a file by using the -f switch. When tail is invoked with the -f switch, it will not stop when the EOF (end of file) marker is reached, but will wait for additional input. To open and watch a logfile use the command tail -f /var/log/system.log. Replace /var/log/system.log with the logfile you are watching.

Use the less command to scroll through a logfile page by page. You can search within the file you are reading by typing the / character followed by your search string ie: /craigz. The first occurrence of your search term will be highlighted. You can jump to the next occurrence by typing the n key on your keyboard, and continue typing n to move through found items. You can also move line by line down the file by using either the return key or the down arrow. The up arrow can also be used to scroll upwards line by line. To move through the document page by page use the space bar.

If you want to check on a specific text string in your logfile, you can use the cat command in conjunction with grep. For example to search for the string craigz in the system log file, type the following command cat /var/log/system.log | grep craigz.

Using the Console application to view logfiles

The Console application is located in the folder Utilities inside the Applications folder. The left side of the application displays two stand-alone files: console.log and system.log as well as three folders where logs are located, /Library/Logs ~Library/Logs and /var/log. Each of these folders has disclosure triangles next them that can be expanded to show individual log files within. This window is shown in Figure 26-13.Once a file is selected on the left, its contents will populate the right side of the window. The search box on the top right is labeled filter. You can type text in that window, and only lines containing that text string will be left in the content window. Clicking the Clear icon will clear the window. Using the Reload button will refresh the window content, and the Mark button will insert a line with a timestamp. The Mark button is quite useful if you are watching a file for a specific event as you can insert timestamps to mark your place, and note between similar looking lines.

Figure 26-13: Using the Console application to view system logfiles.