< Day Day Up > |
Monitoring the logs you collect is vital. A variety of programs exist that will monitor log messages as they come in and notify administrators when suspicious or unusual activity is detected. We will examine two such programs available for both FreeBSD and OpenBSD: logcheck and swatch. Both these programs look for certain patterns in logfiles and then perform one or more given actions. To know what to look for, you will have to spend considerable time looking at system logs and determining what log entries you need to be notified about. It might be helpful to conduct some attacks against your systems to generate key events. Using tools like nessus and nmap and picking up a few exploits from the Security Focus web site (http://www.securityfocus.com/) will help you generate some of the logs you will want to capture. As new vulnerabilities are discovered, run them against your systems. Did they cause your log monitoring scripts to alert you in the way you desired? If not, adjust your configuration.
10.5.1. Automated Auditing Using logchecklogcheck is based upon a component of the Gauntlet firewall system by Trusted Information Systems, Inc. but later completely written by Craig Rowland of Psionic at the time it was called logsentry. This is all ancient history. logcheck has not seen much development in more than seven years. Nevertheless, it requires very little effort to set up and its functionality continues to be useful. The way logcheck works is best described by its README:
With this design, logcheck notifies administrators by email when key warning events occur, and when new, unexpected, log entries turn up. logcheck runs periodically through cron and reviews all messages since last execution. If there is anything to report, logcheck sends an email. 10.5.1.1 Installationlogcheck can be installed from ports/security; the name of the port is logcheck on FreeBSD and logsentry on OpenBSD. Both of these ports download the same compressed source tarball. As with other ports, logcheck is installed by running make install from the logcheck or logsentry ports directory. This will install the logtail binary and several sample configuration files and documents. Run pkg_info -L on the logcheck or logsentry package to determine exactly where the files have been installed on your system. Although the logcheck program has not been modified in years, the installation defaults do change from time to time. 10.5.1.2 ConfigurationThe heart of logcheck is logcheck.sh. This file contains key configuration elements like the address to which notifications should be sent, the location of additional configuration files, and the location of the logfiles to monitor. Each of these topics deserves some discussion as follows:
For the sake of convenience and for a little added security, you may want to move the logcheck.sh script and configuration files into a directory under /usr/local/etc accessible only as root. Whether you choose to take this approach or not, the next step is to tailor the additional configuration files for your environment or at the very least review them.
Notice that being too general in either the hacking or violations configuration files will result in too many false positives whereas being too general in the ignore files will result in interesting log messages being missed. While logcheck is easy to configure, it's even easier to configure poorly. This phase of logcheck configuration is usually iterative, involving the following five steps.
Your goal in configuring logcheck this way is to be able to run logcheck very frequently but receive very little email. The more false positives you receive, the less likely you are to pay attention to the results produced by logcheck. Configure carefully, and configure well. As systems and applications are deployed in your network, revisit and revise your logging and logcheck configuration. 10.5.1.3 Drawbackslogcheck is not perfect. It suffers from lack of active development, which means that you will need to move on to another piece of log monitoring software if you need additional functionality. While logcheck does a very good job monitoring logfiles, it can be inconvenient to deal with an additional logfile containing all messages. Moreover, if you choose to monitor logfiles from applications that cannot log via Syslog, you will need to make duplicates of these logs or inject them into Syslog using the -f flag of the logger command. This can quickly become an administrative headache. Fortunately, the log monitoring world is full of options. Next, we explore another popular log monitoring utility, swatch. 10.5.2. Automated Auditing Using swatchswatch , the simple watcher, was originally designed to monitor logfiles written by syslogd but can now monitor arbitrary logfiles and is quite flexible. swatch expects a single configuration file that contains actions to perform when specified patterns have been found. swatch is almost as easy to install and configure as logcheck, but is more flexible and can perform a variety of actions besides sending email. 10.5.2.1 InstallationIn OpenBSD and FreeBSD, the installation procedure for swatch on a loghost system is identical: % cd /usr/ports/security/swatch % make && sudo make install On both platforms, this will install two binaries (swatch and swatch_oldrc2newrc) and the manpages for both. While example configuration files are included in the compressed swatch tarball, they are not installed during the install process. In OpenBSD the two example files are located in the w-swatch-version/swatch-version/examples/ subdirectory of the swatch ports directory; in FreeBSD, check in the work/swatch-version/examples/ subdirectory. Additional sample configuration files for swatch may be found at http://www.loganalysis.org. 10.5.2.2 ConfigurationYou can start using swatch with very little configuration, and getting things operational is trivial. As with logcheck however, fine-tuning swatch to work well in your environment will take some time. Before we get carried away, let's explore swatch's basic functionality. swatch reads configuration from ${HOME}/.swatchrc by default. You can, and usually should, change this behavior, by providing the -c argument and pointing to a configuration file you create, perhaps in /usr/local/etc/. A contrived configuration file is provided in Example 10-7. As mentioned previously, swatch expects a series of patterns and actions in its configuration file. Patterns are provided as arguments to the watchfor and ignore statements, which do exactly what you think they do. Example 10-7. Simple swatch configurationignore /refused connection from 10.1/ ignore /BAD SU john to root/ watchfor /inetd.* refused connection/ echo normal throttle 01:00:00 mail addresses=root,subject=Refused\ Connection,when=2-6,9-5 watchfor /file system full/ echo bold throttle 00:05:00 mail addresses=root,subject=FILE\ SYSTEM\ FULL,when=1-7,1-24 watchfor /BAD SU.*to root/ echo bold throttle 00:01:00 ignore statements are best placed early in the configuration file to weed out logs that don't need to be parsed. This statement is followed by a regular expression, enclosed in slashes, which will cause log messages to be ignored if the pattern expressed matches the log message. watchfor statements are also followed by a regular expression. When the expression matches a message, various actions are performed.
As the actions are fully documented in the manpage for swatch(1), we will not go into great detail here. However, the actions shown in Example 10-7 are the most popular and deserve some treatment as follows:
At this point, you should have enough information to configure swatch. All that remains is that you know what to look for. For that, you'll need to study your logs, not this chapter. 10.5.2.3 Running swatchAfter you have created a configuration file for swatch, you need to start it. swatch may be run on the command line or as a daemon by using the --daemon command-line argument. It might also be helpful to write a startup script that starts swatch as a daemon and redirects standard output to the console or a separate logfile. Be aware that swatch will need to be restarted whenever the logfile it reads is rotated. To do this, specify a restart time using the --restart-time command-line argument, which ensures the daemon restarts as soon after the log is rotated as possible. 10.5.2.4 Catching new messagesswatch is not designed to alert you when new messages show up in your logs. Whereas logcheck sends you alerts unless you tell it not to, swatch will only send you alerts when you tell it to. Fortunately, due to swatch's flexibility, it's possible to configure it very much like logcheck using a series of ignore lines and watchfor /.*/. This expression will match all entries that have not been already ignored or caught by other watchfor statements. 10.5.3. Ongoing MonitoringOnce you have configured logcheck or swatch for your environment, you might be tempted to never evaluate your log monitoring capability again. If you used logcheck or have configured swatch to alert you of new messages, you will still have to return to log monitoring to cut down on unimportant messages that did not exist when you initially configured the log monitoring system. Despite the fact that your logs are automatically monitored and you have little motivation to return to manually scanning them, you should nevertheless make a point to do so. A periodic manual scan or tail of logs in the background can help you catch activity that log scanners will not. You may notice certain entries appear that you would actually like to know about, yet somehow you have managed to configure your log scanner to ignore them. Watching logs scroll by periodically may also alert you to changes in the frequency with which certain log entries appear a minor warning may not be a problem now and again, but several thousand warnings an hour might be worth examining. The point is, don't underestimate the value of periodic human scans of your logfiles. Finally, make yourself revisit your log monitoring configuration. Continue running vulnerability test kits or sample exploits to help validate the configuration of your log monitoring solution. |
< Day Day Up > |