13.1 Barnyard (and Sguil) | Managing Security with Snort and IDS Tools

One of the costliest activities Snort performs is its alert logging. Data needs to be gathered, formatted, and written. In the case of database writes, Snort must send the alert to the database and wait for confirmation of a successful write. The situation is made even worse when the database server is running on another system on network.

Snort has the ability to dump the information that it has gathered on a particular alert into a binary file. This is very quick, since no processing needs to be performed on the data. The Barnyard application reads this file, formats the alert data, and writes it to the chosen output mechanism. The output mechanism can be the conventional Snort logfile, syslogs, comma-separated-value formatted (CSV) file, or a database server. Barnyard can be configured to run on the same platforms as Snort, and their installation and configuration are very similar. Figure 13-1 illustrates the way Snort and Barnyard work together. Barnyard does a very good job of logging to the ACID database allowing Administrators to continue using familiar tools.

Figure 13-1. Barnyard working with Snort

13.1.1 Configuring Snort's Unified Binary Output

When using unified binary output, Snort is configured normally; the only difference is the output plug-in selected in the snort.conf file. The log_unified output plug-in is the only output plug-in that should be configured. The format of the directive is:

output log_unified: filename <filename>, limit <log file size>

filename: The base filename Snort uses when logging alerts. A timestamp is appended to the filename. The file is created in the location you chose for Snort logfiles to be written (/var/log/snort by default, but configurable using -l.
logfile size: Designates the maximum size that a logfile can attain. Consider 128 MB a minimum size. When this file is full, Snort stops writing to that one and starts another with the same base name but a new timestamp.

A sample snort.conf entry would be:

output log_unified: filename unified.log, limit 512

When Snort starts writing to a new file as a result of the size limit being reached, Barnyard continues processing with new file automatically if it is running in continual mode (see Section 13.1.4 below).

13.1.2 Installing Barnyard

Download Barnyard from http://www.snort.org/dl/barnyard and extract it to a standard location (I prefer /usr/local/src/barnyard). To enable database support, you need to use a directive when running configure. To enable support for MySQL support, use (--enable-mysql); to enable PostgreSQL, use (--enable-postgres). The command line is below:

# cd /usr    /local/src/barnyard/barnyard-0.x.0/ # ./configure --enable-mysql # make # make install

After install, you can find the Barnyard executable in /usr/local/bin and the barnyard.conf file in the /etc directory in your source directory. You can copy this to a location like /usr/local/etc or keep it where it is.

13.1.3 The barnyard.conf File

Most Barnyard options are managed using the barnyard.conf file. There are two sections to the file: the declarations and the output plug-ins.

Here are the configuration declarations:

config daemon

Designates that Barnyard runs in the background (as a daemon). The same as the -D command-line option.

config localtime

Sets Barnyard to use local time instead of UTC. This is generally not recommended due to the confusion it can cause with multiple sensors.

config hostname : <hostname>

Sets the hostname of the sensor. Currently only used by the ACID database plug-in.

config interface : <interface name>

Sets the interface name for use with the ACID database plug-in.

config filter: < filter regular expression>

Normally, this is set to stop an infinite loop if you are sniffing traffic on the same interface you are running a console using SSH. Here's a sample setting:

Config filter: not port 22

The output plug-ins consist of:

output alert_fast: < filename>

Outputs a file that is similar to Snort's "fast" format; very stripped down, with no packet headers.

output log_dump: < filename>

Outputs a file that is similar to Snort's ASCII packet dump mode.

output alert_csv: < long list of options>

(Experimental plug-in) Creates a file with user-designated fields. It might be useful for some custom scripting, but for the vast majority of administrators who are sophisticated enough to use Barnyard, a database is a better choice. If you want to do custom scripting, refer to the database schema in Appendix A and create database-aware scripts. You'll be happier.

output alert_syslog: < facility>, < priority>

Outputs using the syslog mechanism with the same options as Snort (see Chapter 5).

output alert_syslog2: < facility>, < priority>

Uses a mechanism similar to the alert_syslog output line, but has the expanded formats of syslog2 available.

output log_pcap: <filename>

Outputs data in the standard pcap format.

output alert_acid_db : <database type>, sensor_id <sensor id>, database <database name>, server <server address>, user <username for database>, password <database password>

Sends alert information (less detailed than log information) to an ACID database. Designate the database type (mysql, postrgres, and so on), and supply the location of the server (localhost or IP address), database name, and the authentication information for the database user.

output log_acid_db : <database type>, sensor_id <sensor id>, database <database name>, server <server address>, user <username for database>, password <database password>, detail <full or fast>

Sends detailed log information to an ACID database. Designate the database type (mysql, postrgres, and so on) and supply the location of the server (localhost or IP address), database name, and the authentication information for the database user. One last bit of data to include is the detail option (fast or full). I always use full. Here's a sample configuration line for the ACID output plugin:

output log_acid_db: mysql, sensor_id 1, database snort, server 10.10.10.25, user  snort_user, password pa$$w0rd, detail full

output sguil : <db type>, sensor_id <sensor ID>, database <db name>, server <IP address or hostname>, user <db username>,password <db password>, sguild_host <IP address or hostname>, sguild_port <port number>

Sends detailed log information to a sguil server database. Designate the database type (mysql, postgres, and so on), supply the location of the database server, the database user and password, the sguil server location, and the port that sguil is listening on (defaults to 7736). Here's sample configuration for the sguil output plug-in:

output sguil: mysql, sensor_id 1, database sguil, server 10.10.10.25, user sguil_ user, password pa$$w0rd, sguild_host localhost, sguild_port 7736

13.1.4 Barnyard Command-Line Options

Barnyard runs in one of three modes: one-shot, continual, or continual with checkpoint. One-shot mode (or batch mode) is used to run Barnyard against a single unified logfile and then exit. Continual watches a unified logfile as it is written to. The continual with checkpoint mode is similar to the continual mode except it keeps track of where it is in the file by keeping a pointer in a file (sometimes called a waldo file). If Barnyard crashes, processing will continue at this point in the unified logfile.

The mode that Barnyard is running in (as well as other potentially ephemeral configuration settings) is designated at the command line. The command line options for Barnyard are:

-R: Similar to the -T test mode of Snort. Processes the configuration and tells you if you have any problems. This has gotten very useful in recent versions of Barnyard.
-c < path to barnyard.conf>: The path to the configuration file.
-d < dir>: The directory where Snort will be writing the unified binary format logs.
-L < dir>: If Barnyard is configured to output to files, this path designates where the files should be written.
-v: Increases verbosity of console output.
-s < file>: The path to Snort's sid-msg.map file. Included with the Snort rules download and specifies the sid for an alert message.
-g < file>: The path to Snort's gen-msg.map file. Included with the Snort rules download and specifies the detection generator that is associated with a generator ID.
-p < file>: The path to Snort's classification.config file. This is included with the Snort rules download and specifies the alert classifications used by Snort.
-a < dir>: If a unified logfile is processed, it can be archived to another location as specified.
-f < base>: The base name for the Snort unified binary format logs. This is the filename without the appended timestamp.
-n: Only process new events.
-w < file>: The checkpoint (waldo) file used for continual with checkpoint mode.
-D: Run in daemon mode.
-X < file>: Specify a file to store the PID of the Barnyard process.
-o: Enable one-shot mode on the specified file.
-t < timestamp>: You can specify the timestamp for the first file to be processed. The timestamp is in Unix time (seconds since the epoch).

Here are some sample command lines (they can get very long). First, there's batch (one-shot) mode:

barnyard -c /usr/local/etc/barnyard.conf -d /var/log/snort \ -c /usr/local/share/snort_rules/classification.config \ -s /usr/local/share/snort_rules/sid-msg.map \ -g /usr/local/share/snort_rules/gen-msg.map -o unified.log.1083726235

Then we have continual mode with checkpoint:

barnyard -c /usr/local/etc/barnyard.conf -d /var/log/snort \ -c /usr/local/share/snort_rules/classification.config \ -s /usr/local/share/snort_rules/sid-msg.map \ -g /usr/local/share/snort_rules/gen-msg.map -w /usr/local/etc/waldo.chk \ -f unified.log

13.1.5 Sguil: An Alternative Management Console

Developer:	Bamm Visscher
Link:	http://squil.sourceforge.net
Download Link:	http://sourceforge.net/project/showfiles.php?group_id=71220
Platform:	Tcl/Tk client/server architecture
Prerequisites:	MySQL, Snort, Barnyard, Tcl/Tk 8.3 or newer (http://www.tcl.tk/software/tcltk/), Tclx libraries (http://tclx.sourceforge.net), Mysqltcl (http://www.xdobry.de/mysqltcl/), incr tcl (itcl) (http://incrtcl.sourceforge.net/itcl/), tcllib extension (http://tcllib.sf.net), Tcpflow (http://www.circlemud.org/~jelson/software/tcpflow/), p0f (http://lcamtuf.coredump.cx/p0f.shtml)

As the prerequisite list indicates, installing Sguil can be hairy. Getting it running involves installing the standard Snort components, Tcl (tool control language) and tk (a graphical user interface toolkit), several add-on Tcl libraries, and the Tcpflow and p0f applications. Then set up a database for Sguil to use, install the GUI server, and the GUI client, patch Snort's source code and recompile, configure Barnyard's Sguil output plugin, and configure a script to get the data from Snort, Tcpflow, and p0f into the database. Detailed installation instructions are available on the Sguil web page.

Sguil is a near real-time interface to Snort alerts that relies on Barnyard, Tcpflow, and p0f to gather alert data. Tcpflow and p0f are used to create a transcript of network traffic that can be useful in discrimating false positives and post-incident forensic analysis. The interface is actually very nice to use and presents the alert information in a useful format.

An extract from the Sguil home page:

Events can be validated by placing them into one of seven incident categories or marking the event as having no further action required (NA). These actions remove the events from the RealTime tab of all the connected clients but are not deleted from the database. Archived events can easily be retrieved from the database through preformatted queries, or the analyst can create a custom query using SQL.

Sguil isn't that different from the ACID interface: both allow you to monitor alerts and search by event, sensor, alert classification, alert priority, or timestamp. Sguil is more up-to-date with the latest preprocessors particularly the new flow-portscan system. Both ACID and Sguil lack sensor-management tools.

Where connecting to ACID is easy since it is a web-based interface, the only way to get a remote client to connect to a central server is by using an exported X-session (a security no-no). Looking back at the last few paragraphs, I see that I'm drawing a lot of comparisons between ACID and sguil. Functionally, they are very similar. Sguil's transcript feature differentiates it.

A daunting installation, poor client model, and lack of many new features make it difficult to recommend Sguil. I advise sticking with ACID.

Figures Figure 13-2 and Figure 13-3 show screenshots of Sguil in action.