Optimizing Logging

Problem

You want to optimize your logging.

Solution

To log in binary format, you must use the -b command-line option in conjunction with the -l option:

C:Snortin>snort -l c:snortlog -b

You can also specify a name for the logfile by using the -L option:

C:Snortin>snort -l c:snortlog -b -L test

If you are running Snort in NIDS mode, you can also perform binary logging by configuring the log_tcpdump output plug-in in the /etc/snort.conf file:

output log_tcpdump: tcpdump.log

You can also use the unified output plug-in in the /etc/snort.conf file:

output alert_unified: filename snort.alert, limit 128

output log_unified: filename snort.log, limit 128

 

Discussion

Two methods can be used to optimize Snort logging: binary logging and unified logging.

To log in binary format you must use the -b command-line option in conjunction with the -l option. The -b option specifies to log the packets in binary format. You won't see any data output on the screen when you are logging in binary format, unless you also use the -v command-line option. You don't need to specify the -d or -e command-line options, because by default, the binary option logs the entire packet.

C:Snortin>snort -l c:snortlog -b

This command creates a file called snort.log.1084553605 in the C:Snortlog directory. You can also specify a name for the logfile by using the -L option.

C:Snortin>snort -l c:snortlog -b -L test

The numbers assigned to snort.log.xxx or filename.yyy are known as Epoch time; this is the number of seconds since 01-01-1970.

This command creates a file called test.1084554709 in the C:Snortlog directory. If you are running Snort in NIDS mode, you can also perform binary logging by uncommenting the following line from the /etc/snort.conf file:

# log_tcpdump: log packets in binary tcpdump format

# -------------------------------------------------

# The only argument is the output file name.

#

output log_tcpdump: tcpdump.log

The following command runs Snort in NIDS mode and creates a binary file with the name tcpdump.log.number, such as tcpdump.log.1086466896, in the C:Snortlog directory:

C:Snortin> snort -l c:snortlog -c c:snortetcsnort.conf

Unified logging also logs in binary format; however, it uses a Snort output plug-in to reduce the load on the Snort processing engine. To enable unified logging, you must make a change to the /etc/snort.conf file by uncommenting and configuring the following output plug-ins:

# unified: Snort unified binary format alerting and logging

# -------------------------------------------------------------

# The unified output plugin provides two new formats for logging and

# generating alerts from Snort, the "unified" format. The unified

# format is a straight binary format for logging data out of Snort

# that is designed to be fast and efficient. Used with barnyard (the

# new alert/log processor), most of the overhead for logging and

# alerting to various slow storage mechanisms such as databases or the

# network can now be avoided.

#

# Check out the spo_unified.h file for the data formats.

#

# Two arguments are supported.

# filename - base filename to write to (current time_t is appended)

# limit - maximum size of spool file in MB (default: 128)

#

output alert_unified: filename snort.alert, limit 128

output log_unified: filename snort.log, limit 128

Both the filename and file size limit parameters are configurable. Unified logs have a varying number assigned to them after the .alert and .log filename extensions. In our example, running Snort with the following command-line options created the unified logging files snort.alert.1086463191 and snort.log.1086463191 in the C:Snortlog directory:

C:Snortin> snort -l c:snortlog -c c:snortetcsnort.conf

The best way to optimize Snort logging is to use unified logging with a separate log-processing tool such as Barnyard. Binary logging makes logging more efficient because the Snort engine doesn't have to translate the data into human-readable format. Logging traffic in binary mode is great for high-speed networks and compact storage. Binary files can then be reviewed later using Snort, TCPDump, Ethereal, or other binary log-compatible programs. If you are using Snort on a very high-speed network, such as 1 Gbps or greater, you benefit more from unified logging. This allows the Snort engine to write logs and alerts quickly to a binary file, while offloading the spooling and processing to another program. Offloading the processing of logs and alerts means Snort can spend more time capturing data, thus decreasing the likelihood that packets are dropped. You must use a unified log reader, such as Barnyard, to process the logfiles.

See Also

Recipe 5.9

Reading Unified Logged Data