Section 18.1 Finding the Cracker s Running Processes | Real World Linux Security Prentice Hall Ptr Open Source Technology Series

18.1 Finding the Cracker's Running Processes

Once you have detected that your system has been broken into, it would be very helpful to try to find any running processes that the cracker has left behind. Remember that any program on the system might have been compromised.

It is preferable to operate as an unprivileged user account that does not have access to anything important. This is because you do not know what programs have been compromised. For example, perhaps the cracker discovered that /usr/local/bin was mode 777 (world-writable) and he placed a compromised version of date in it. He might be waiting for something as innocent as root invoking date to get full control of the system that is, "own it" in cracker parlance. By operating as that unprivileged user, clearly you limit the consequences of this or other actions.

Keep a "stealth" version of ps under an unassuming name. If you run an application, call it by that name. Do a ps of your system and note root programs such as lpd and sendmail as possible names to call your "stealth ps" executable. Some crackers might notice that sendmail should not have an argument of axlww so if you are feeling ambitious, grab the source of ps and tweak it to create a custom version that defaults to these flags. (The "a" flag requests all processes, not just yours; the "x" flag also includes daemons not associated with a terminal; the "l" flag requests long format to give more details; the "w" flag allows longer lines; and the second "w" allows unlimited lines.)

Back during my gray hat student days at Berkeley, one time I was operating covertly as root, repeatedly doing a ps^[a] to watch for system administrators trying to detect us. I was "riding shotgun" while Doug Merritt installed a Trojan.

I noticed someone logged in on an administrator account do a ps and knew that we were detected. Twenty seconds later the administrator was using the write command to contact me, asking who I was.

Fortunately we were in a little-used terminal room and we were logged off and out of Evans Hall in 60 seconds flat! The lesson is that each side in this war should have been using stealth versions of ps. The better crackers now do this; you should too!

^[a] Most people do not know that the "command name" that ps (process status) displays is specified by the command's parent process and is arbitrary. In other words, when you ask the shell to start ps, the shell chooses to claim that the child process name is ps.

A common cracker technique is to specify a different name such as cc or assignment7. The name (or inode number in pre-2.2 kernels) of the executable that Linux's /proc/PID/exe symbolic link points to cannot be spoofed, however, except with a kernel hack. A useful enhancement to ps would to be to warn if exe does not match the command name (recognizing that the shell may show the program name as ps instead of /bin/ps).

18.1.1 Handling Deleted Executables

One cracker trick is to remove the executable of a running program from the file system. Recall that this will cause the reference to the name of the file in its directory to be removed but the file still will exist until all programs that have the file open (as open file descriptors) close it.

A running program "in execution" is treated as an open file. Crackers know that the first thing most SysAdmins do when they detect an intrusion is to shut down the system, either to copy the disk for evidence or analysis or in the hope that the problem will go away. Of course, on a clean shutdown the program's execution will be stopped and the program's data blocks and inode freed. If the system is shut down abruptly, the cleanup will be done upon reboot by fsck.

The method for detecting these executing programs and making copies of them automatically (for analysis) is discussed in "Detecting Deleted Executables" on page 655. Any of these executables that are found are almost certainly Trojans, unless one of them is a program under development by a programmer whose building of a new version caused the deleted version to be removed from disk.

A very useful feature in the kernel is that the symbolic link in /proc to the executable is good even though the original file has been removed from the file system! This allows you to make a copy of the file for analysis as simply as

 cp /proc/479/exe /home/samspade/del_cracker

In other words, if the cracker did

 cd /tmp .genie& rm .genie

this technique still will recover a copy for you to analyze and present as evidence.

Top