| Often, a successful attacker will try to hide their tracks with greater success, and therefore simple service monitoring won't be of assistance. The attacker might be far more skillful at hiding his tracks than you are at tracking down anomalous system states. Linux systems are too diverse, customizable, and complicated to define an iron-clad, fully comprehensive list of definitive symptoms proving that the system is compromised. As with any kind of detective or diagnostic work, you must look for clues where you canas systematically as you can. RFC 2196, "Site Security Handbook," provides a list of signs to check for. The "Steps for Recovering from a UNIX or NT System Compromise," available from CERT at http://www.cert.org/tech_tips/root_compromise.html, provides another list of anomalies to check for. The following sections incorporate both lists, including all or most of their points in one form or another. The anomalies have been roughly categorized into the following: indications related to the system logs; changes to the system configuration; changes related to the filesystem, file contents, file access permissions, and file size; changes to user accounts, passwords, and user access; problems indicated in the security audit reports; and unexpected performance degradation. The anomalous indications often cross category boundaries. System Log Indications System log indications include unusual error and status messages in the logs, truncated log files, deleted log files, and emailed status reports: System log files Unexplained entries in the system log files, shrinking log files, and missing log files all suggest that something is wrong. For example, /var/log/messages contains the majority of the system log information on most Linux systems. If that log file is zero-sized or is missing large portions, additional investigation is warranted. System daemon status reports Instead of (or in addition to) writing to the log files, some daemons such as crond send status reports in email. Having unusual or missing reports suggests that something is not right. Anomalous console and terminal messages Unexplained messages, possibly meant to announce the hacker's presence, during a login session are obviously suspicious. Repeated access attempts Ongoing login attempts or illegal file access attempts through FTP or a web server, particularly attempts to subvert CGI scripts, are suspicious when the attempts are persistent, even if the attempts appear to end in repeated failure.
Chapter 10, "Intrusion Detection Tools," details some automatic log-monitoring programs that can be helpful when putting up an alert or taking some other action in real time. System Configuration Indications System configuration indications include modified configuration files and system scripts, unintended processes running inexplicably, unexpected service port usage and assignments, and changes in network device operational status: cron jobs Check the cron configuration scripts and executables for modification. Altered system configuration files A filesystem integrity check, manual or using a tool as described in Chapter 12, "Filesystem Integrity," would indicate changed configuration files in /etc. These files are critical to proper system functioning. Any change to a file (such as in /etc/, like /etc/passwd, /etc/group, /etc/hosts.equiv, and similar files) is important to check. Unexplained services and processes, as shown by ps Unexpectedly running programs are a bad sign. Be aware that as part of the attack, the ps command itself may have been replaced. More on this later. Unexpected connection and unexpected port usage, as shown by netstat or tcpdump Unexpected network traffic is a very bad sign. System crashes and missing processes System crashes, as well as unexpected server crashes, might be suspect. A system crash can also suggest an attacker-initiated system reboot, which could be necessary to restart certain critical system processes after replacement with a trojan version. Changes in device configuration Reconfiguring a network interface to be in promiscuous or debug mode is a sign that a packet sniffer is installed.
Filesystem Indications Filesystem indications include new files and directories, missing files and directories, altered file contents, MD5sum mismatches, new setuid programs, and rapidly growing or overflowing filesystems: New files and directories Besides files with suddenly bad digital signatures, you might discover new files and directories. Especially suspicious are filenames starting with one or more dots and legitimate-sounding filenames appearing in unlikely places. setuid and setgid programs New setuid files, and newly set setuid files, are a good place to start looking under the hood for problems. Missing files Missing files, particularly log files, indicate a problem of some kind. Rapidly changing filesystem sizes, as shown by df If the machine is compromised, rapidly growing filesystems might be a sign of a hacker's monitoring program producing large log files. Modified public file archives Check the contents of your web and FTP areas for new or modified files. New files or directories in /dev CERT warns especially to check for the presence of new ASCII files or directories in /dev; these are typically Trojan programs' configuration files.
User Account Indications User account indications include new user accounts, changes to the passwd file, unusual activity in the user process accounting reports or missing process accounting reports, changes to user filesespecially environmental filesand loss of account access: New and modified user accounts New accounts in /etc/passwd and processes running under new or unexpected user IDs as shown by ps are indications of new accounts. Accounts with suddenly missing passwords indicate an open account. User accounting records Unusual user accounting reports, inexplicable logins, missing or edited log files (such as /var/log/lastlog, /var/log/pacct, or /var/log/usracct), and irregular user activity are signs of trouble. Changes to root or user accounts A serious sign is if a user's login environment is modified or damaged to the point that the account is inaccessible. Of particular concern are changes to users' .rhost and .forward files, and changes to their PATH environment variable. Loss of account access Similar to changes to a user's login environment is intentional access denial, whether by changing the account password, by removing the account, or, for regular users, by changing the runlevel to single-user mode.
Security Audit Tool Indications Security audit tool indications include filesystem integrity mismatches, file-size changes, changes to file-permission mode bits, new setuid and setgid programs, alerts from Intrusion detection tools such as Snort, and service monitoring data. Files with mismatched hash signatures can be files that are new, files whose lengths or creation or modification dates have changed, and files whose access modes are altered. Of particular concern are newly installed trojan horse programs. Frequent targets for replacement are programs managed by inetd or xinetd, inetd or xinetd itself, ls, ps, netstat, ifconfig, telnet, login, su, ftp, syslogd, du, df, sync, and the libc library. System Performance Indications System performance indications include unusually high load averages and heavy disk access. Unexplained, poor system performance could be caused by unusual process activity, unusually high load averages, excessive network traffic, or heavy filesystem access. If your system shows signs of a successful compromise, don't panic. Don't reboot the systemimportant information could be lost. Simply physically disconnect the system from the Internet. |
