Hack68.Check for Rootkits and Other Attacks

Hack 68. Check for Rootkits and Other Attacks

Let chkrootkit automatically check your externally facing machines for rootkits and other attacks.

A rootkit is a software package that enables an unauthorized user to obtain root or administrative privileges on a machine. Rootkits are usually installed by exploiting a known security problem. Once installed, they can capture passwords, monitor system status, send system authentication information to other hosts, and even execute programs at scheduled intervals.

While rootkits are conceptually quite interesting, being "rooted" (the term for being compromised such that unauthorized people have root access to your system) is not. Luckily, just as there are plenty of scripts that automate installing rootkits, there are also some great software packages that detect rootkits and identify compromised systems and applications. Some packages, such as Tripwire [Hack #66] and Afick [Hack #67], generally monitor file sizes and signatures and let you know if something has changed that shouldn't have. This hack explores chkrootkit, one of the most powerful and popular software packages for actually detecting rootkits themselves and discusses how to install and use it to detect and close down invasions.

7.7.1. Types of Rootkits

Linux rootkits work in various ways, usually as kernel modules, user-space software packages that replace system binaries, or a combination of both. Kernel rootkits insert loadable kernel modules that replace system calls with hacked versions that capture information and often hide information about specific processes from the user, whereas user-space rootkits generally replace system binaries such as ps, login, passwd, and so on with hacked versions that also capture information and hide information about specific processes and directories. For example, the t0rn rootkit mentioned in the "True Confessions" sidebar replaces system binaries such as ps, top, and ls with versions that won't list anything that is running from its /usr/src/.puta directory. Pretty clever, actually.

chkrootkit runs on Linux systems using any 2.x kernel and has also been used and tested on FreeBSD 2.2.x, 3.x, 4.x and 5.x systems; OpenBSD 2.x and 3.x systems; NetBSD 1.6.x systems; Solaris 2.5.1, 2.6, 8.0, and 9.0 systems; and various HP-UX, Tru64, and BSDI system releases. At the time that this book was written, chkrootkit could detect rootkits such as 55808.A Worm, Adore LKM, Adore Worm, AjaKit, Anonoying, Aquatica, ARK, Bobkit, dsc-rootkit, duarawkz, Ducoci, ESRK, Fu, George, Gold2, Hidrootkit, Illogic, Kenga3, kenny-rk, knark LKM, Lion Worm, LOC, LPD Worm, lrk, Madalin, Maniac-RK, MithRa's Rootkit, Monkit, Omega Worm, OpenBSD rk v1, Optickit, Pizdakit, Ramen Worm, rh-shaper, RK17, Romanian, RSHA, RST.b trojan, Scalper, Sebek LKM, ShitC Worm, Shkit, Showtee, shv4, SK, Slapper A-D, SucKIT, TC2 Worm, t0rn, TRK, Volc, Wormkit Worm, x.c Worm, zaRwT, and ZK.

A basic problem in rootkit detection is that any system on which a rootkit has been installed can't be trusted to detect rootkits. This can be resolved by doing regular system maintenance by running chkrootkit from a bootable CD. We'll come back to that later. For now, let's install chkrootkit and put it through its paces.

True Confessions

Hi, my name is Bill, and one of my systems was rooted once. Where better to confess my sysadmin indiscretions than in a book that will hopefully be read by zillions of people?

Years ago, long before home gateways and Network Address Translation (NAT) boxes were sub-$100 consumer electronics devices, I built my own home gateway, like most Linux geeks. By putting an extra Ethernet card in an ancient Pentium box and writing a few ipfwadm rules (the ancestor of yesterday's ipchains and today's iptables), I could do NAT and masquerading of my internal systems through my external network interface. This worked fine 24x7 for quite a while, modulo the occasional power failure, and I used it to run my home name server and route to external DNS servers. I never updated any of the software on the box, based on the "don't fix it if it isn't broken" rule (which is a very bad rule for sysadmins to follow when it comes to security updates). One day, I logged in on the box to check something and noticed that the output from my favorite invocation of the ps command didn't display output in the same way that it usually did. So I checked /var/log/messages and found a few messages that indicated that someone had been probing my DNS server, attempting to induce a buffer overflow. I poked around a bit andno big surprisefound that the machine had been hacked and the t0rn rootkit (http://www.sans.org/y2k/t0rn.htm) was installed.

My reaction to this was different than most. Since none of my home machines themselves were hacked (I checked) and I was curious, I changed all of my passwords on systems that I might have contacted since the rootkit had been installed (from work, of course, not from home), and stopped doing anything at home that required a remote password for a few days. I then put a README.txt file in /usr/src/.puta, which is where t0rn puts most of its files, saying something along the lines of "Hi there, congrats, and how'd you get in?" I got mail within a day or so from the guy who'd hacked my box, we exchanged a few mail messages through the anonymous remailer he was using, and he turned me on to some of the rootkits that he had access to. I would have been completely anal about this if this was a work machine, but as it was, he seemed like a pretty smart guy and I learned a few things. I rebuilt the machine (with updated software) within a week or two, anywayI'm friendly, but not suicidal.

The point here is not that my system was hacked, but rather that the potential is always out there. Crackers can often exploit newly discovered or unpatched problems in system software to install rootkits on your system, some of which are both fast and clever. Adding chkrootkit to your system's toolbox can help you detect this sort of invasion and shut it down as quickly as possible.

7.7.2. Obtaining, Building, and Installing chkrootkit

chkrootkit is open source and is freely available from http://www.chkrootkit.org/download. The current version at the time this book was written was 0.45. Newer versions are better, since each version of chkrootkit adds software and support for detecting more and more rootkits. The chkrootkit executable is a shell script that runs the binaries and other scripts that are included as part of the chkrootkit package.

After downloading the source tarball, you can build chkrootkit as shown in the following example:

 $ tar zxf chkrootkit.tar.gz  $ cd chkrootkit-0.45  $ make  *** stopping make sense ***  make[1]: Entering directory `/home/wvh/src/chkrootkit-0.45'  gcc -DHAVE_LASTLOG_H -o chklastlog chklastlog.c  gcc -DHAVE_LASTLOG_H -o chkwtmp chkwtmp.c  gcc -DHAVE_LASTLOG_H -D_FILE_OFFSET_BITS=64 -o ifpromisc ifpromisc.c gcc -o chkproc chkproc.c gcc -o chkdirs chkdirs.c gcc -o check_wtmpx check_wtmpx.c gcc -static -o strings-static strings.c gcc -o chkutmp chkutmp.c make[1]: Leaving directory `/home/wvh/src/chkrootkit-0.45' 

chkrootkit's Makefile doesn't provide an install target, so you must either manually copy its binaries somewhere or run it from the directory in which you built it. If you do the latter, I'd suggest removing all the source code files to make it harder for anyone who has cracked your system to hack your chkrootkit installationnot impossible, just harder.

7.7.3. Running chkrootkit

Once you've built chkrootkit, you simply run it from wherever you've put the binaries by executing ./chkrootkit or by invoking the full pathname to the chkrootkit shell script. You must execute chkrootkit as the root user or via sudo. The output from a run of chkrootkit looks like the following:

 # ./chkrootkit ROOTDIR is '/' Checking 'amd'… not found Checking 'basename'… not infected Checking 'biff'… not found Checking 'chfn'… not infected  Checking 'chsh'… not infected  Checking 'cron'… not infected  Checking 'date'… not infected  Checking 'du'… not infected  Checking 'dirname'… not infected  Checking 'echo'… not infected  Checking 'egrep'… not infected  Checking 'env'… not infected  Checking 'find'… not infected  Checking 'fingerd'… not found  Checking 'gpm'… not infected  Checking 'grep'… not infected  Checking 'hdparm'… not infected  Checking 'su'… not infected  Checking 'ifconfig'… not infected  Checking 'inetd'… not tested  Checking 'inetdconf'… not found  Checking 'identd'… not found  Checking 'init'… not infected  Checking 'killall'… not infected  Checking 'ldsopreload'… not infected  Checking 'login'… not infected  Checking 'ls'… not infected  Checking 'lsof'… not infected  Checking 'mail'… not infected  Checking 'mingetty'… not infected  Checking 'netstat'… not infected  Checking 'named'… not infected  Checking 'passwd'… not infected  Checking 'pidof'… not infected  Checking 'pop2'… not found  Checking 'pop3'… not found  Checking 'ps'… not infected  Checking 'pstree'… not infected  Checking 'rpcinfo'… not infected  Checking 'rlogind'… not found  Checking 'rshd'… not found  Checking 'slogin'… not infected  Checking 'sendmail'… not infected  Checking 'sshd'… not infected  Checking 'syslogd'… not infected  Checking 'tar'… not infected  Checking 'tcpd'… not infected  Checking 'tcpdump'… not infected  Checking 'top'… not infected  Checking 'telnetd'… not found  Checking 'timed'… not found  Checking 'traceroute'… not infected  Checking 'vdir'… not infected  Checking 'w'… not infected  Checking 'write'… not infected  Checking 'aliens'… no suspect files Searching for sniffer's logs, it may take a while… nothing found  Searching for HiDrootkit's default dir… nothing found  Searching for t0rn's default files and dirs… nothing found  Searching for t0rn's v8 defaults… nothing found  Searching for Lion Worm default files and dirs… nothing found  Searching for RSHA's default files and dir… nothing found  Searching for RH-Sharpe's default files… nothing found  Searching for Ambient's rootkit (ark) default files and dirs…nothing found  Searching for suspicious files and dirs, it may take a while…   /usr/lib/jvm/java-1.4.2-sun-1.4.2.08/jre/.systemPrefs   /usr/lib/perl5/5.8.6/x86_64-linux-thread-multi/.packlist  Searching for LPD Worm files and dirs… nothing found  Searching for Ramen Worm files and dirs… nothing found  Searching for Maniac files and dirs… nothing found  Searching for RK17 files and dirs… nothing found  Searching for Ducoci rootkit… nothing found  Searching for Adore Worm… nothing found  Searching for ShitC Worm… nothing found  Searching for Omega Worm… nothing found  Searching for Sadmind/IIS Worm… nothing found  Searching for MonKit… nothing found  Searching for Showtee… nothing found  Searching for OpticKit… nothing found  Searching for T.R.K… nothing found  Searching for Mithra… nothing found  Searching for OBSD rk v1… nothing found  Searching for LOC rootkit… nothing found  Searching for Romanian rootkit… nothing found  Searching for Suckit rootkit… nothing found  Searching for Volc rootkit… nothing found  Searching for Gold2 rootkit… nothing found  Searching for TC2 Worm default files and dirs… nothing found  Searching for Anonoying rootkit default files and dirs… nothing found  Searching for ZK rootkit default files and dirs… nothing found  Searching for ShKit rootkit default files and dirs… nothing found  Searching for AjaKit rootkit default files and dirs… nothing found  Searching for zaRwT rootkit default files and dirs… nothing found  Searching for Madalin rootkit default files… nothing found  Searching for Fu rootkit default files… nothing found  Searching for ESRK rootkit default files… nothing found  Searching for anomalies in shell history files… nothing found  Checking 'asp'… not infected Checking 'bindshell'… not infected  Checking 'lkm'… chkproc: nothing detected  Checking 'rexedcs'… not found  Checking 'sniffer'…   eth0: not promisc and no PF_PACKET sockets   vmnet8: not promisc and no PF_PACKET sockets   vmnet1: not promisc and no PF_PACKET sockets Checking 'w55808'… not infected  Checking 'wted'… chkwtmp: nothing deleted  Checking 'scalper'… not infected  Checking 'slapper'… not infected Checking 'z2'… chklastlog: nothing deleted Checking 'chkutmp'… chkutmp: nothing deleted 

It seems like I'm clean, and that's a lot of tests! As you can see, chkrootkit first checks a variety of system binaries for strings that would indicate that they've been hacked, then checks for the indicators of known rootkits, checks network ports for spurious processes, and so on. I feel better already.

If you are running additional security software such as PortSentry (http://sourceforge.net/projects/sentrytools/), you may get false positives (i.e., reports of problems that aren't actually problems) from the bindshell test, which looks for processes that are monitoring specific ports.

If you want to be even more paranoid than chkrootkit's normal behavior, you can run chkrootkit with its -x (expert) option. This option causes chkrootkit to display detailed test output in order to give you the opportunity to detect potential problems that may be evidence of rootkits that the version of chkrootkit that you're using may not (yet) be able to identify.

7.7.4. Automating chkrootkit

Running chkrootkit "every so often" is a good idea, but running it regularly via cron is a better one. To run chkrootkit automatically, log in as root, su to root, or use sudo to run crontab -e and add chkrootkit to root's list of processes that are run automatically by cron. For example, the following entry would run chkrootkit every night at 1 A.M. and would mail its output to [e-mail]:

 03***(cd /path/to/chkrootkit; ./chkrootkit 2>&1 | mail -s "chkrootkit \  output" [e-mail]) 

7.7.5. Summary

A basic problem in rootkit detection is that any system on which a rootkit has been installed can't be trusted to detect rootkits. Even if you follow the instructions in this hack and run chkrootkit via cron, you only have a small window of opportunity before the clever cracker checks root's crontab entry and either disables or hacks chkrootkit itself. The combination of chkrootkit and software such as Tripwire or Afick can help make this window as small as possible, but regular system security checks of externally facing machines from a bootable CD that includes chkrootkit, such as Inside Security's Insert Security Rescue CD (http://sourceforge.net/projects/insert/), is your best solution for identifying rootkits so that you can restore compromised systems.

7.7.6. See Also

• http://www.chkrootkit.org

• "Tame Tripwire" [Hack #66]

• "Verify Fileystem Integrity with Afick" [Hack #67]

• Insert Security Rescue CD: http://www.inside-security.de/insert_en.html

• Rootkit Hunter: http://www.rootkit.nl

• Windows users: http://research.microsoft.com/rootkit/

• Windows users: http://www.sysinternals.com/utilities/rootkitrevealer.html