The biggest problem with logging is information overload. How can one administrator be expected to monitor thousands and thousands of lines each day? Many people will quickly admit that they do check their logs, but only when they want to diagnose a problem that already exists. Logs, properly monitored , can provide an effective form of intrusion detection, enabling one to react to potential problems before they become disasters.
In the previous chapter, we looked at a portscan monitor, PortSentry, by Psionic software. In this chapter, we introduce another Psionic product, LogSentry. LogSentry is an extremely simple utility that performs a very important task: It watches your logfiles for you.
As you read through the available logs on your system, you'll notice that problems are typically denoted with words such as denied or refused . LogSentry contains a long list of "danger" words that can indicate an attack or unusual system activity. On a timed schedule, it checks your logfiles against the keyword list and generates a report of log information that you may want to investigate further. LogSentry doesn't have any problem with keeping track of its place when reading the file, nor does it mind logfile rotations . Best of all, it's easy to set up and adds next to no overhead to your system.
To install LogSentry, first make sure that you have your logs configured appropriately and have a list of the files that you want to monitor. If you have installed additional software that you want to monitor, you may want to check through the logs it generates to look for keywords that indicate warnings you want reported .
Next, download LogSentry from Psionic at http://www.macosxunleashed.com/downloads. Unarchive and enter the distribution directory:
% curl -O http://www.macosxunleashed.com/downloads/logsentry-1.1.1.tar.gz % tar zxf logsentry-1.1.1.tar.gz % cd logcheck-1.1.1/
Now, a somewhat odd second step: Move the file INSTALL to a different filename. The compilation process attempts to use a file name install , but fails because of the lack of casesensitivity of the Mac OS X HFS+ filesystem.
% mv INSTALL INSTALL.readme
Create the directories where LogSentry will be installed: /usr/local/etc/ , /usr/local/bin , and /usr/local/etc/tmp :
# mkdir -p /usr/local/etc/tmp # mkdir /usr/local/bin # chmod 600 /usr/local/etc/tmp
Finally, make and install the software with make bsdos .
# make bsdos make install SYSTYPE=bsdos Making generic cc -O -o ./src/logtail ./src/logtail.c ./src/logtail.c: In function `main': ./src/logtail.c:51: warning: return type of `main' is not `int' Creating temp directory /usr/local/etc/tmp Setting temp directory permissions chmod 700 /usr/local/etc/tmp Copying files cp ./systems/generic/logcheck.hacking /usr/local/etc cp ./systems/generic/logcheck.violations /usr/local/etc cp ./systems/generic/logcheck.violations.ignore /usr/local/etc cp ./systems/generic/logcheck.ignore /usr/local/etc cp ./systems/generic/logcheck.sh /usr/local/etc cp ./src/logtail /usr/local/bin Setting permissions chmod 700 /usr/local/etc/logcheck.sh chmod 700 /usr/local/bin/logtail chmod 600 /usr/local/etc/logcheck.violations.ignore chmod 600 /usr/local/etc/logcheck.violations chmod 600 /usr/local/etc/logcheck.hacking chmod 600 /usr/local/etc/logcheck.ignore Done. Don't forget to set your crontab.
LogSentry is now installed and can be accessed via /usr/local/etc/logcheck.sh , but first you should customize the location where your logfiles are stored and where the resulting reports should be sent.
To do this, open the file /usr/local/etc/logcheck.sh in your favorite editor. First, configure where the outgoing report should be sent. The default value, root , can be changed to a remote email address. Search for the SYSADMIN line:
# Person to send log activity to. SYSADMIN=root
and change it to reflect your email address:
# Person to send log activity to. SYSADMINemail@example.com
You must have sendmail configured properly for this to work. It does not need to run as a daemon, but must still have the appropriate directory permissions (or DontBlameSendmail flags) set to process outgoing mail.
Next, search for the BSDI logfile configuration section. In the out-of-the-box logcheck.sh file, it should look like this:
# BSDI 2.x $LOGTAIL /var/log/messages > $TMPDIR/check.$$ $LOGTAIL /var/log/secure >> $TMPDIR/check.$$ $LOGTAIL /var/log/maillog >> $TMPDIR/check.$$ $LOGTAIL /var/log/ftp.log >> $TMPDIR/check.$$ # Un-comment out the line below if you are using BSDI 2.1 #$LOGTAIL /var/log/daemon.log >> $TMPDIR/check.$$
Unfortunately, these aren't the appropriate names for your Mac OS X files. Modify the lines to read as follows :
# BSDI 2.x $LOGTAIL /var/log/system.log > $TMPDIR/check.$$ $LOGTAIL /var/log/secure.log >> $TMPDIR/check.$$ $LOGTAIL /var/log/mail.log >> $TMPDIR/check.$$ $LOGTAIL /var/log/ftp.log >> $TMPDIR/check.$$
This example assumes that you haven't changed the Mac OS X syslog configuration. If you have , you should change the file to match your particular setup, including adding new lines to match additional logfiles.
For most users, setup is now complete. If, however, you'd like to customize the keywords to which LogSentry responds (or doesn't respond), four files in /usr/local/etc/ can be changed:
logcheck.violations . A list of "negative" keywords/strings that will be searched for and included in the LogSentry report.
logcheck.hacking . A list of keywords/strings that could indicate a potential attack taking place. Matches against the hacking list will be flagged with the heading ACTIVE SYSTEM ATTACK in the output report.
logcheck.violations.ignore . Strings that will be ignored, even if they match the logcheck.violations file.
logcheck.ignore . Strings that will be ignored, regardless of whether they appear in either the logcheck.violations and logcheck.hacking files.
As soon as you are satisfied with your setup, run logcheck.sh for the first time by typing /usr/local/etc/logcheck.sh . If you have a reasonable amount of system activity, you should receive a log alert in your email. For example:
Security Violations =-=-=-=-=-=-=-=-=-= Nov 23 11:45:16 despair sshd: Failed password for jray from 22.214.171.124 port 49279 ssh2 Nov 23 17:54:10 despair su: jray to root on /dev/ttyp7 Nov 23 18:24:37 despair sendmail: gANNObux001633: SYSERR(root): hash map "access": unsafe map file /etc/mail/access.db: Permission denied Nov 23 18:26:04 despair sendmail: gANNQ4ux001647: SYSERR(root): hash map "access": unsafe map file /etc/mail/access.db: Permission denied Nov 23 18:26:45 despair sendmail: gANNQjux001672: SYSERR(root): hash map "access": unsafe map file /etc/mail/access.db: Permission denied Unusual System Events =-=-=-=-=-=-=-=-=-=-= Nov 23 11:45:16 despair sshd: Failed password for jray from 126.96.36.199 port 49279 ssh2 Nov 23 11:45:18 despair sshd: Accepted password for jray from 188.8.131.52 port 49279 ssh2 Nov 23 11:46:11 despair sudo: jray : TTY=ttyp7 ; PWD=/Users/jray ; USER=root ; COMMAND=/bin/tcsh Nov 23 17:25:50 despair sudo: jray : TTY=ttyp7 ; PWD=/Users/jray/logsentry/logcheck-1.1.1 ; USER=root; COMMAND=/bin/tcsh Nov 23 17:54:10 despair su: jray to root on /dev/ttyp7 Nov 23 18:01:09 despair sudo: root : TTY=ttyp7 ; PWD=/Users/jray/logsentry/logcheck-1.1.1 ; USER=root; COMMAND=/bin/tcsh Nov 23 18:24:37 despair sendmail: gANNObux001633: SYSERR(root): hash map "access": unsafe map file /etc/mail/access.db: Permission denied relay=localhost [127.0.0.1], reject=451 4.3.0 Temporary system failure. Please try again later. Nov 23 18:26:04 despair sendmail: starting daemon (8.12.2): queueing@01:00:00 Nov 23 18:26:04 despair sendmail: gANNQ4ux001647: SYSERR(root): hash map "access": unsafe map file /etc/mail/access.db: Permission denied Nov 23 18:26:04 despair sendmail: ruleset=check_relay, arg1=localhost, arg2=127.0.0.1, relay=localhost [127.0.0.1], reject=451 4.3.0 Temporary system failure. Please try again later.
As you can see, LogSentry does a great job of finding the entries that are worth looking at ”and delivering them to your mailbox ”without requiring any interaction with you (after you start the program).
The author of LogSentry recommends running the logcheck.sh script every hour to keep LogSentry up to date on system activities. You can automate this by adding the following line to the /etc/crontab file:
00 * * * * root /bin/sh /usr/local/etc/logcheck.sh
Alternatively, you can create a root user cron job file with the contents
00 * * * * /bin/sh /usr/local/etc/logcheck.sh
and submit the file to the cron process with crontab <filename> .
Log-watching is a very monotonous and time-consuming process of being a system administrator. It's an often-overlooked task that everyone puts off. With LogSentry, it can be performed effortlessly.
An alternative logfile analysis program is swatch. Like LogSentry, swatch watches for potentially "bad" signs in your system logs. If you'd like to try something different, swatch is available at http://www.oit.ucsb.edu/~eta/swatch/.