Hack 92 Use Snort in High-Performance Environments with Barnyard

Decouple Snort's output stage so it can keep pace with the packets.

Snort by itself is fine for monitoring small networks or networks with low amounts of traffic, but it does not scale very well without some additional help. The problem is not with Snort's detection engine itself, but stems from the fact that Snort is a single-threaded application. Because of this, whenever an alert or log event is triggered, Snort must first send the alert or log entry to its final destination before it can go back to looking at the incoming data stream. This isn't such a big deal if you're just having Snort write to a file, but it can become a problem if you are logging to a database, which can cause Snort to wait a relatively long time for the database insert to complete. This of course is exacerbated when you're having Snort log to a remote database server.

To solve this, another application called Barnyard (http://www.snort.org/dl/barnyard/) was written. Functionally, Barnyard is the equivalent of Snort's output plug-ins all rolled into one program, with a frontend for reading in files that Snort generates and then sending them to the same database or other destination that you would normally have Snort log to. The only draw back to Barnyard is its limited database support: Barnyard supports only MySQL, whereas Snort supports MySQL, PostgreSQL, Oracle, and ODBC outputs (Barnyard claims to support PostgreSQL, but unfortunately its current support is shaky at best).

After downloading Barnyard and unpacking it, change to the directory it created and run its configure script:

$ ./configure --enable-mysql

This will enable MySQL support when Barnyard is compiled. If you've installed your MySQL libraries and include files in a nonstandard place (i.e., underneath the /usr or /usr/local hierarchies), you'll probably need to add the --with-mysql-includes and --with-mysql-libraries command-line options.

After you're done with the configure script, you can compile Barnyard by running make. When it finishes compiling, install it by becoming root and running make install.

Before you use Barnyard, you'll need to configure Snort to use its unified output format. This is a binary format that includes both the alert information and the data for the packet that triggered the alert, and it is the only type of input that Barnyard will understand.

To configure Snort to use the unified output format for both alert and log events, add lines similar to these to your Snort configuration (e.g., /etc/snort/snort.conf or /usr/local/etc/snort/snort.conf):

output alert_unified: filename snort.alert, limit 128 output log_unified: filnemae snort.log, limit 128

The filenames specified here are the basenames for the files that Snort will write its alert and log event information to. When it writes a file, it will append the current Unix timestamp to the end of the basename. In addition, the size of these files will be limited to 128MB.

Now you'll need to create a configuration file for use with Barnyard. To run Barnyard in daemon mode and have it automatically fork itself into the background, add this line to your configuration file:

config daemon

If you're going to be logging to a database for use with ACID [Hack #83], you'll also want to add two lines similar to these:

config hostname: colossus config interface: eth0

These two lines should be set to the name of the machine that you're running Barnyard on and the interface that Snort is reading packets from. Next, you'll need to actually tell Barnyard to read your unified log files.

You can do this for alert events by using this line:

processor dp_alert

Or, if you want to process log events, use this line:

processor dp_log

Note that Barnyard can process only one type of unified log at a time. So, if you want it to process both alert and log events, you'll need to run an instance of Barnyard for each type.

Now all that's left to configure is where Barnyard will send the data. If you want to use Snort's fast alert mode to generate single-line abbreviated alerts, you can use the alert_fast output plug-in:

output alert_fast: fast_alerts.log

Or, if you want Barnyard to generate ASCII packet dumps of the data contained in the unified logs, you can use a line similar to this:

output log_dump: ascii_dump.log

To have Barnyard output to your syslog daemon, you can use the alert_syslog plug-in just like you would in your snort.conf. For instance, if you wanted to send data to the local syslogd and use the auth facility and the alert log level, you could use a line like this:

output alert_syslog: LOG_AUTH LOG_ALERT

Or, if you want to send to a remote syslog daemon, you can use a line similar to this:

output alert_syslog: hostname=loghost, LOG_AUTH LOG_ALERT

You can also have Barnyard create Pcap-formatted files from the data in the unified logs. This is useful for analyzing the data later in tools such as Ethereal. To do this, use the log_pcap plug-in:

output log_pcap: alerts.pcap

Finally, you can also have Barnyard output to a database by using the alert_acid_db plug-in for logging alert events and the log_acid_db for capturing log events.

For instance, this line would send alerts to the SNORT MySQL database running on dbserver using the username snort:

output alert_acid_db: mysql, sensor_id 0, database SNORT, server dbserver, user snort

The sensor_id is the one assigned by ACID to the particular instance of Snort that is gathering the data. You can find what sensor ID to use by clicking on the Sensors link on ACID's front page [Hack #83], which will show you a list of the sensors that are currently logging to ACID.

The log_acid_db plug-in is similar, except it does not use the sensor_id option:

output log_acid_db: mysql, database SNORT, server dbserver, user snort, detail full

You can start Barnyard by simply using a command similar to the following if Snort's configuration files are stored in /etc/snort and Snort is set to keep its logs in /var/log/snort:

# barnyard -f snort.alert

Of course, this assumes that you used snort.alert when configuring Snort's alert_unified plug-in. If your Snort configuration files aren't stored in /etc/snort, you can specify the locations of all the files that Barnyard needs to access by running a command similar to this one:

# barnyard -c /usr/local/etc/snort/barnyard.conf \  -g /usr/local/etc/snort/gen-msg.map \ -s /usr/local/etc/snort/sid-msg.map -f snort.alert

This would tell Barnyard where to find all the files it needs if they are in /usr/local/etc/snort (and are too stubborn to create a symlink to /etc/snort). If you're using a directory other than /var/log/snort to store Snort's logs, you can specify it with the -d option.

Congratulations. With Barnyard running, you should be able to handle much larger volumes of traffic without dropping log entries or missing a single packet.