The Logging System

I l @ ve RuBoard

Information logging is key to a properly secured system. Most systems have a great deal of system logging capabilities. However, they often have to be configured or enabled. UNIX systems have a number of standard logging facilities which are common to all UNIX systems. However, logs by themselves offer little security if they are not being reviewed. Logs are a tool and must be use effectively by scanning through them regularly. Unfortunately, logs are often used only in a reactionary way because of a lack of time and because of the sheer size of the logs. Consolidation of logs and automated scanning can reduce this task to a reasonable task.

Table 14-1. HP-UX Audit Events and Categories

Event	Categories
create	Log all creations of objects
delete	Log all deletions of objects
readdac	Log reads of discretionary access control (DAC) information
moddac	Log all modifications of object's Discretionary Access Control (DAC) information
modaccess	Log all modifications other than DAC
open	Log all openings of objects
close	Log all closings of objects
process	Log all operations on processes
removable	Log all removable media events (mounting and unmounting events)
login	Log all logins and logouts
admin	Log all administrative and privileged events
ipccreat	Log all IPC create events
ipcopen	Log all IPC open events
ipcclose	Log all IPC close events
uevent1	Log user -defined event
uevent2	Log user-defined event
uevent3	Log user-defined event

On UNIX systems, the primary logging service is syslog for system messages and klog for kernel messages. These logs are usually consolidated with the syslog facility, along with any system utilities or applications which utilize logging.

The syslog service parses and separates log messages based on a system of "Facility" and "Level." The "Facility" is the software or subsystem which is generating the log message, and the "Level" is a setting indicating criticalness. Table 14-2 lists the "Levels" recognized by syslog.

Table 14-2. Syslog Levels

Level	Meaning
emerg	A panic condition. This is normally broadcast to all users.
alert	A condition that should be corrected immediately, such as a corrupted system database.
crit	Critical conditions, such as hard device errors.
err	Errors.
warning	Warning messages.
notice	Conditions that are not error conditions, but should possibly be handled specially.
Info	Informational messages.
debug	Messages that contain information normally of use only when debugging a program.

Application Logging

Most applications have very little, if any, logging. When an application has logging, it is either because of legal requirements or to aid in the development or debugging of the software. Daemons and servers are somewhat more likely to provide some level of logging.

Logging from a software standpoint is very simple. An application API is provided to the syslog facility. It provides the ability for applications to put logging messages into the system's standard logging facility. Using this facility allows the system administrator to set the level of logging needed and how the logs are to be stored. Syslog APIs are widely available for C. Linux provides a Perl interface to syslog and HP-UX provides a command interface, logger. The following is a description of the C language API for the syslog service.

void openlog(const char *ident, int logopt, int facility)

openlog is called to initialize communication to the log file.

ident is the identification string that precedes every message.

logopt is a mask of bits, indicating logging options.

The values for logopt are:

LOG_PID ” Log the process ID with each message; useful for identifying instances of daemons.

LOG_CONS ” Force writing messages to the console if unable to send it to syslogd. This option is safe to use in daemon processes that have no controlling terminal because syslog() forks before opening the console.

LOG_NDELAY ” Open the connection to syslogd immediately. Normally, the open is delayed until the first message is logged. This is useful for programs that need to manage the order in which file descriptors are allocated.

LOG_NOWAIT ” Do not wait for children forked to log messages on the console. This option should be used by processes that enable notification of child termination via SIGCLD, because syslog() might otherwise block, waiting for a child whose exit status has already been collected.

void syslog(int priority, const char *message, ...)

syslog writes the message onto the system log maintained by syslogd. It does not log a message that does not have a level set. If it cannot pass the message to syslogd, it attempts to write the message on /dev/console if the LOG_CONS option is set.

priority is encoded as the logical OR of a level and a facility. The level signifies the urgency of the message, and facility signifies the subsystem generating the message. Facility can be encoded explicitly in priority, or a default facility can be set with openlog.

message is tagged with priority.A trailing newline is added if needed. This message is read by syslogd and is written to the system console, log files, selected users' terminals, or forwarded to syslogd on another host as appropriate.

void closelog(void)

closelog closes the log file.

The logger command provides a program interface to the syslog system log module. With this, a message can be given on the command line, which is logged immediately, or a file is read and each line is logged. If no file or message is specified, the contents of the standard input are logged. The logger command recognizes the following command-line options and arguments:

-t	Defines the tag.
-p	Defines the priority. The priority can be specified numerically or as a facility.level pair. The default is user.notice.
-i	Logs the process ID of the logger process with each line.
-f	Logs the contents of the specified file.

To send the message "System going down immediately!!!" to the syslog daemon, at the emerg level and user facility, the logger command would be:

 logger -p user.emerg "System going down immediately!!!" 

Kernel Logging

On HP-UX, kernel logging is a high-availability feature that gives system administrators the ability to collect the information necessary to diagnose problems with the HP-UX kernel while the system is running. /usr/sbin/kl is used to specify the levels of events to be logged and the kernel subsystems that will write messages to memory or disk. It also provides for managing the contents of the logfile in memory and on disk.

At start-up, kernel logging determines its default configuration by reading the file /etc/nettlgen.conf . The kl command permits only temporary changes to the default kernel logging configuration without having to stop and restart the kernel logging facility. To make permanent changes to the values in the /etc/nettlgen.conf file, run nettlconf .

Mission critical systems should always enable kernel logging. Failing to enable kernel logging causes diagnostic information about any suspicious events that might occur on the system to be lost. The recommended classes to capture are Disaster, Error, and Warning. Use the kl -l w all command to do so. To minimize kernel logging's impact on a running system, use the kl -l e all command to set all kernel subsystems to capture error-level log messages only. Only users with appropriate privileges (root) can invoke the kl command to control the kernel logging facility.

Enabling Logging

The main starting point to enable logging on a UNIX system is to ensure that the inet daemon is logging. This is done by starting the daemon with the "-l" option. The internet daemon will then log every time a service under its control is used and from where the connection came. Many of the services which the internet daemon starts have their own logging which can provide more information. For example, ftpd , the file transfer protocol daemon, will log connections with the (-l) option. The newest release will also log the commands sent to the ftp server, (-L) and the files received (-i) and transmitted (-o). One should consult the man page for each service which is running on the system to be sure its logging is enabled.

Any system utility which is not started by inetd should provide its own logging. Some of these utilities log by default, others require enabling the logging features. Most of the utilities utilize the syslog service. The HP-UX environmental daemon (envd), the password and group daemon (pwgrd) and ups_mond, and the HP PowerTrust Uninterruptible Power System monitor daemon are examples of utilities on HP-UX which are not started by inetd and are running on most systems.

Configuring syslog for Local Logging

The syslog daemon can redirect the logged information into separate log files based on the combination of "Facility" and "Level." Only the "Facility" and "Level" combinations listed, or indicated by wildcards, will be logged. The following is a example default /etc/syslog.conf file:

 mail.debug             /var/adm/maillog  mail.none              /var/adm/maillog  auth.notice            /var/adm/authlog  lpr.debug              /var/adm/lpd-errs  kern.debug             /var/adm/messages  *.emerg;*.alert;  *.crit;*.warning;*.err;  *.notice;*.info        /var/adm/messages 

When logging locally, a compromised system's logs have to be considered suspect. Any local files on a compromised system could have been altered. It is not possible to tell if the logs have been altered with the standard syslog system. There are no integrity checks within the syslog facility.

Configuring syslog for Remote Logging

Remote logging reduces the risk that a compromised system's logs will be altered to hide the attack. Since the logs are on a remote logging server, that server too would have to have been compromised to change the logs. Configuring a system to use a remote logging server is a simple task. All the logs can be redirected to that logging server by placing the following entry into /etc/syslog.conf:

 *.*    @logging-host 

I l @ ve RuBoard