Hack 97 Verify File Integrity and Find Compromised Files

Use Tripwire to alert you to compromised files or verify file integrity in the event of a compromise.

One tool that can help you detect intrusions on a host and also ascertain what happened after the fact is Tripwire (http://sourceforge.net/projects/tripwire). Tripwire is part of a class of tools known as file integrity checkers, which can detect the presence of important changed files on your systems. This is desirable because intruders who have gained access to a system will often install what's known as a root kit, in an attempt to both cover their tracks and maintain access to the system. A root kit usually accomplishes this by modifying key operating system utilities such as ps, ls, and other programs that could give away the presence of a backdoor program. This usually means that these programs will be patched to not report that a certain process is active or that certain files exist on the system. Attackers could also modify the system's MD5 checksum program (e.g., md5 or md5sum) to report correct checksums for all the binaries that they have replaced. Since using MD5 checksums is usually one of the primary ways to verify whether a file has been modified, it should be clear that something else is sorely needed.

This is where Tripwire comes in handy. It stores a snapshot of your files in a known state, so you can periodically compare the files against the snapshot to discover discrepancies. With this snapshot, Tripwire can track changes in a file's size, inode number, permissions, or other attributes, such as the file's contents. To top all of this off, Tripwire encrypts and signs its own files, to detect if it has been compromised itself.

Tripwire is driven by two main components: a policy and a database. The policy lists all files and directories that Tripwire should snapshot, along with rules for identifying violations (i.e., unexpected changes). For example, a simple policy might treat any changes in /root, /sbin, /bin, and /lib as violations. The Tripwire database contains the snapshot itself, created by evaluating the policy against your filesystems. Once setup is complete, you can compare filesystems against the snapshot at any time, and Tripwire will report any discrepancies.

Along with the policy and database, Tripwire also has configuration settings, stored in a file that controls global aspects of its behavior. For example, the configuration specifies the locations of the database, policy file, and tripwire executable.

Tripwire uses two cryptographic keys to protect its files. The site key protects the policy file and the configuration file, and the local key protects the database and generated reports. Multiple machines with the same policy and configuration may share a site key, but each machine must have its own local key for its database and reports.

One caveat with Tripwire is that its batch-oriented method of integrity checking gives intruders a window of opportunity to modify a file after it has been legitimately modified and before the next integrity check has been run. The modified file will be flagged, but it will be expected (because you know that the file is modified) and probably dismissed as a legitimate change to the file. For this reason, it is best to update your Tripwire snapshot as often as possible. Failing that, you should note the exact time that you modified a file, so you can compare it with the modification time that Tripwire reports.

Tripwire is available with the latest versions of Red Hat and as a port on FreeBSD. However, if you're not running either of those, you'll need to compile it from source. To compile Tripwire, download the source package and unpack it. Next, check whether you have a symbolic link from /usr/bin/gmake to /usr/bin/make. (Operating systems outside the world of Linux don't always come with GNU make, so Tripwire explicitly looks for gmake, but this is simply called make on most Linux systems.) If you don't have such a link, create one.

Another thing to check for is a full set of subdirectories in /usr/share/man. Tripwire will need to place manpages in man4, man5, and man8. On systems where these are missing, the installer will create files named after those directories, rather than creating directories and placing the files within the appropriate ones. For instance, a file called /usr/man/man4 would be created instead of a directory of the same name containing the appropriate manual pages.

Now change your working directory to Tripwire source's root directory (e.g., ./tripwire-2.3.1-2) and read the README and INSTALL files. Both are brief but important.

Finally, change to the source tree's src directory (e.g., ./tripwire-2.3.1-2/src) and make any necessary changes to the variable definitions in src/Makefile. Be sure to verify that the appropriate SYSPRE definition is uncommented (SYSPRE = i686-pc-linux or SYSPRE = sparc-linux, etc.).

Now you're ready to compile. While still in Tripwire's src directory, enter this command:

$ make release

Then, after compilation has finished, run these commands:

$ cd .. $ cp ./install/install.cfg . $ cp ./intall/install.sh

Now open install.cfg with your favorite text editor to fine-tune the configuration variables. While the default paths are probably fine, you should at the very least examine the Mail Options section, which is where we initially tell Tripwire how to route its logs. Note that these settings can be changed later.

If you set TWMAILMETHOD=SENDMAIL and specify a value for TWMAILPROGRAM, Tripwire will use the specified local mailer (sendmail by default) to deliver its reports to a local user or group. If instead you set TWMAILMETHOD=SMTP and specify values for TWSMTPHOST and TWSMTPPORT, Tripwire will mail its reports to an external email address via the specified SMTP server and port.

Once you are done editing install.cfg, it's time to install Tripwire. While still in the root directory of the Tripwire source distribution, enter the following:

# sh ./install.sh

You will be prompted for site and local passwords: the site password protects Tripwire's configuration and policy files, whereas the local password protects Tripwire's databases and reports. This allows the use of a single policy across multiple hosts, to centralize control of Tripwire policies but distribute responsibility for database management and report generation.

If you do not plan to use Tripwire across multiple hosts with shared policies, there's nothing wrong with setting the site and local Tripwire passwords on a given system to the same string. In either case, choose a strong passphrase that contains some combination of upper- and lowercase letters, punctuation (which can include whitespace), and numerals.

When you install Tripwire (whether via binary package or source build), a default configuration file is created, /etc/tripwire/tw.cfg. You can't edit this file, because it's an encrypted binary, but for your convenience, a clear-text version of it, twcfg.txt, should also reside in /etc/tripwire. If it does not, you can create the text version with this command:

# twadmin --print-cfgfile > /etc/tripwire/twcfg.txt

By editing this file, you can make changes to the settings you used when installing Tripwire, and you can change the location where Tripwire will look for its database. This can be done by setting the DBFILE variable. One interesting use of this is to set the variable to a directory within the /mnt directory hierarchy. Then, after the database has been created you can copy it to a CD-ROM and remount it there whenever you need to perform integrity checks.

After you are done editing the configuration file, you can re-encrypt it by running this command:

# twadmin --create-cfgfile --site-keyfile ./site.key twcfg.txt

You should also remove the twcfg.txt file.

You can then initialize Tripwire's database by running this command:

# tripwire --init

Since this uses the default policy file that Tripwire installed, you will probably see errors related to files and directories not being found. These errors are nonfatal, and the database will finish initializing. If you want to get rid of these errors, you can edit the policy and remove the files that were reported as missing.

First you'll need to decrypt the policy file into an editable plain text format. You can do this by running the following command:

# twadmin --print-polfile > twpol.txt

Then comment out any files that were reported as missing. You will probably want to look through the file and determine whether any files that you would like to catalog aren't already in there. For instance, you will probably want to monitor all SUID files on your system [Hack #2]. Tripwire's policy-file language can allow for far more complex constructs than simply listing one file per line; read the twpolicy(4) manpage for more information if you'd like to use some of these features.

After you've updated your policy, you'll also need to update Tripwire's database. You can do this by running the following command:

# tripwire --update-policy twpol.txt

To perform checks against your database, run this command:

# tripwire --check

This will print a report to the screen and leave a copy of it in /var/lib/tripwire/report. If you want Tripwire to automatically email the report to the configured recipients, you can add --email-report to the end of the command. You can view the reports by running twprint.

For example:

# twprint --print-report --twrfile \  /var/lib/tripwire/report/colossus-20040102-205528.twr

Finally, to reconcile changes that Tripwire reports with its database, you can run a command similar to this one:

# tripwire --update --twrfile \ /var/lib/tripwire/report/colossus-20040102-205528.twr

You can and should schedule Tripwire to run its checks as regularly as possible. In addition to keeping your database in a safe place, such as on a CD-ROM, you'll also want to make backup copies of your configuration, policy, and keys. Otherwise you will not perform an integrity check in the event that someone (malicious or not) deletes them.

See Also