As detailed in Chapter 4, the first step when investigating an incident is to determine if there actually was one - there must be a
In one incident, an organization
Once the nature and severity of an incident has been determined, it is advisable to
Investigating computer intrusions usually involves a large amount of digital evidence. Investigators must search large log files for relevant entries, examine programs to determine their purpose closely, and explore the network for additional clues. In addition to being technically challenging, there is often pressure on an
Under such conditions,
A common challenge that arises during intrusion investigations is the need to protect the target systems against further attack. Investigators may even be asked to remove a backdoor and repair the target system before they have collected evidence from the system. Whenever possible, evidence should be preserved prior to repairing the target system or altering its state in any other way. It is usually
A routine vulnerability scan of a network detected Back Orifice running on a Windows 2000 server. Because of the critical role that this server
D:\>regdmp \Registry <cut for brevity> (HKLM\System\CurrentControlSent\Services) WinLogin Type = REG_DWORD 0x00000110 Start = REG_DWORD 0x00000004 ErrorControl = REG_DWORD 0x00000000 ImagePath = REG_EXPAND_SZ ' "C:\WINNT\System32\wlogin.exe"' DisplayName = WinLogin
Furthermore, NT Application Event logs showed that Norton AntiVirus had detected Back Orifice but had not been able to remove it:
D:\>dumpel -c -l application <cut for brevity> 1/19/2002,12:32:48 AM,4,0,20,Norton AntiVirus,N/A,CONTROL, Unable to restore C:\WINNT\system32\wlogin.exe from backup file after clean failed, 1/19/2002,1:09:11 AM,1,0,5,Norton AntiVirus,N/A, CONTROL, Virus Found!Virus name: B02K.Trojan Variant in File: C:\WINNT\Java\w.exe by: Scheduled scan. Action: Clean failed : Quarantine succeeded : Virus Found!Virus
name: BO2K.Trojan Variant in File: C:\WINNT\system32\wlogin.exe by: Scheduled scan. Action: Clean failed : Quarantine failed : 1/19/2002,1:09:11 AM,4,0,2,Norton AntiVirus,N/A, CONTROL, Scan Complete: Viruses:2 Infected:2 Scanned:62093 Files/Folders/Drives Omitted:89
The intruder had also installed an IRC bot in C:\WINNT\Java that contained several possible leads including IP addresses, nicknames, and IRC channel passwords. However, because the priority was to recover the system, this evidence was collected hastily and the Trojan horse program was removed. After removing the rogue service from the Registry, the server was rebooted to ensure that all remnants of the process were eliminated. Unfortunately, the domain controller did not reboot successfully. Attempting to fix the problem had effectively done more damage than the intruder, interrupting business operations while attempting to restore the server. After some pandemonium, the system was restored from backup, a lengthy process resulting in a prolonged interruption in business that the organization had hoped to avoid.
By the time the domain controller had been recovered, the organization was more interested in apprehending the culprit. Their concerns were exacerbated when they realized that the intruder could have obtained passwords from the server and used them to compromise other system on the network. Unfortunately, much of the evidence had been
One of the more difficult decisions is whether to shut down a compromised system or collect some data from it beforehand. When investigating a computer intrusion, it is often desirable to capture and record system information that is not collected by a bitstream copy of the hard disk. For instance, it is useful to document current network connections, which
Notably, shutting a system down does not
COMPUTERNAME=WWW ComSpec=C:\WINNT\system32\cmd.exe CONTENT_LENGTH=0 .GA TEWAY_INTERFACE=CGI/1.1 .HTTP_ACCEPT=image/gif, image/x-xbitmap, ima ge/jpeg, image/pjpeg, */* HTTP_HOST=192.168.16.133 HTTP_USER_AGENT=Microsoft URL Control - 6.00.8862 . HTTP_CACHE_CONTROL=no-cache
HTTPS=off .INCLUDE=C: \Program Files\Mts\Indude INSTANCE_ID=1 LIB=C:\Prog ram Files\Mts\Lib .LOCAL_ADDR= 192.168.16.133 .NUMBER_OF_PROCES SORS=1 .Os2LibPath=C:\WINNT\System32\os2\dll; OS=Windows_NT Path=C:\Perl\bin;C:\WINNT\system32;C:\WINNT;C:\Program Files\Mts .PATH_TRANSLATED=c:\lnetpub\ wwwrootPATHEXT=.COM;.EXE;.BAT;.CMD;.VBS;.JS;.VBE;.JSE;.WSF;.WSH PROCESSOR_ARCHITECTURE=x86 PROCESSOR_IDENTIFIER=x86 F amily 6 Model 5 Stepping 2, Genuinelntel PROCESSOR_LE VEL=6 PROCESSOR_REVISION=0502 QUER Y_STRING=/c+ping+172.16.81.74+-n+56000+-w + 0+-l+56000 REMOTE_ADDR= 18.104.22.168 REMOTE_HOST522.214.171.124 REQUEST_METHOD=G ET SCRIPT_NAME==/msadc/ / / / / / /winnt/system3 2/cmd.exe SERVER_NAME=192.168.16.133 SERVER_PORT=80 SERVE R_PORT_SECURE=0 SERVER_PROTOCOL=HTTP/1.1/ S ERVER_SOFTWARE=Microsoft-IIS/4.0 SystemDrive=C: <cut for brevity> "c:\lnetpub\wwwroot\msadc\ \ \ \ \ \winnt\system32\cmd.exe" /c ping 172.16.81.74 -n 56000 -w 0 -l 56000.
When dealing with a computer intrusion, do not assume that the incident is isolated - there may be other systems on the network that are involved.
A system administrator found unusual files on a Windows NT server that he was responsible for. The host had been compromised via the IIS Web server and was running Serv-U FTP Server v3.0 ("c:\winnt\system32\setup\x2x\rundll16.exe") on ports 666 and 9669. The FTP server was being used to share pornography, feature length
#0dayvcd with password psA4C70E33CF55B74D5F1C21B8EE46DD8F Vcd with password pt0BED4C47C1826BE160D6FA8E4F85A28F admin with password qgEDA3C477AF1702713437C873A460F230
Another file named "msgtoadmin.txt" contained the following text:
Note To admin
Well what can I say. I broke in yes. But I'm not here to attack. I'm sorry for any inconvience this may have caused you. No viruses or worms have been installed. That is not my intension. I just love you bandwidth:) If you have read this you must have cought me. And once you have don't worry, I'm gone and won't bother you again. again sorry for any inconvience this may have caused. Have a good day,
Not taking the intruder's assurances to heart, the system administrator port scanned all of the systems on the network looking for ports 666 or 9669 and found several other similarly compromised systems. The administrator found other compromised systems by monitoring network traffic for
When responding to an incident on a large network, port scanning may produce too many false positives in which case simple Perl scripts can be created to scan machines on a network and inspect their responses for specific class characteristics to determine if they are compromised. For instance, in the
When a computer program is executed it is
There are a number of utilities that enable investigators to gather information about processes that are currently running on a Windows NT/2000/XP computer. Commands such as netstat and nbtstat are installed with the operating system and other specialized tools that are
The following intrusion detection system logs show an attack against a critical UNIX machine (192.168.128.14) from another important Windows 2000 server (192.168.164.163) on the network:
[**] [1:1326:1] EXPLOIT ssh CRC32 overflow NOOP [**] 04/24-03:28:43 192.168.164.163 192.168.128.14 S: 2445 D: 22 [**] [1:1326:1] EXPLOIT ssh CRC32 overflow NOOP [**] 04/24-07:18:21 3 192.168.164.163 192.168.128.14 S: 2888 D: 22
A port scan of the Windows 2000 server, named "server1," showed many open ports, including one that gave a command prompt to anyone who connected using Telnet:
% bin/probe_tcp_ports 192.168.164.163 Port 80 (possibly http) Port 135 (possibly rpc service) Port 139 (possibly rpc service) Port 443 (possibly https) Port 445 (possibly netbios) Port 1025 Port 1046 Port 1048 Port 1051 Port 1061 Port 1433 (possibly ms-sql) Port 2025 Port 3372 Port 3389 Port 3497 Port 4362 Port 7904 Port 12323 Port 43958 % telnet server1 12323 Microsoft Windows 2000 [Version 5.00.2195] (C) Copyright 1985-2000 Microsoft Corp. C:\WINNT\system32>
The network cable was disconnected from server1 immediately to prevent further unauthorized remote access. A rapid response and recovery was desired to minimize the impact on business continuity. Management wanted to determine what the intruder changed on the system and what actions were necessary to remove all backdoors.
The output of the netstat command confirmed the ports that were seen with the remote port scan, but did not show the remote addresses of machines that were connected to this system because the network cable had been unplugged. The processes listed using Alt-Ctrl-Del included two unrecognized processes named sqldiagmsrv and sqldiagncv as shown in Figure 19.1. More details about these processes, like how long they had been running, could be obtained using pslist. 
Figure 19.1: Unusual process
These unrecognized processes were examined more closely to determine what they were doing on the system. The fport command showed that c:\winnt\system32\sqldiagncv.exe was bound to port 12323.
D:\>fport FPort v1.33 - TCP/IP Process to Port Mapper Copyright 2000 by Foundstone, Inc. http://www.foundstone.com Pid Process Port Proto Path 1152 inetinfo -> 80 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 484 svchost -> 135 TCP C:\WINNT\system32\
svchost.exe 1152 inetinfo -> 443 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 8 System -> 445 TCP 556 msdtc -> 1025 TCP C:\WINNT\System32\msdtc.exe 960 MSTask -> 1027 TCP C:\WINNT\system32\MSTask.exe 1152 inetinfo -> 1028 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 892 sqlservr -> 1029 TCP C:\MSSQL7\binn\sqlservr.exe 8 System -> 1031 TCP 892 sqlservr -> 1433 TCP C:\MSSQL7\binn\ sqlservr.exe 1152 inetinfo -> 2025 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 556 msdtc -> 3372 TCP C:\WINNT\System32\msdtc.exe 368 termsrv -> 3389 TCP C:\WINNT\System32\termsrv.exe 1152 inetinfo -> 4362 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 1152 inetinfo -> 7904 TCP C:\WINNT\System32\inetsrv\inetinfo.exe 1052 sqldiagncv -> 12323 TCP c:\winnt\system32\sqldiagncv.exe 1068 wingtm -> 43958 TCP C:\WINNT\system32\wingtm.exe 484 svchost -> 135 UDP C:\WINNT\system32\svchost.exe 8 System -> 445 UDP 256 services -> 1026 UDP C:\WINNT\system32\services.exe 516 spoolsv -> 1030 UDP C:\WINNT\system32\spoolsv.exe 916 rtvscan -> 2967 UDP C:\ProgramFiles\NavNT\rtvscan.exe 1152 inetinfo -> 3456 UDP C:\WINNT\System32\inetsrv\inetinfo.exe
The handle command, which lists which system resources each process is using, showed that the sqldiagncv executable was running with SYSTEM level authority, allowing significant access to the system:
sqldiagncv.exe pid: 1052 NT AUTHORITY\SYSTEM 18: File C:\WINNT\system32 e0: Section \BaseNamedObjects\__R_0000000000f2_SMem__
The listdlls command showed the command line parameters that sqldiagncv was executed with as well as its associated dynamic link libraries:
sqldiagncv.exe pid: 1052 Command line: c:\winnt\system32\sqldiagncv.exe -l -d -p 12323 -t -e cmd.exe Base
SizeVersion Path 0x00400000 0x13000 c:\winnt\system32\sqldiagncv.exe 0x77f80000 0x7b000 5.00.2195.2779 C:\WINNT\System32\ntdll.dll 0x77e80000 0xb5000 5.00.2195.4272 C:\WINNT\system32\KERNEL32.dll 0x75050000 0x8000 5.00.2195.2871 c:\winnt\system32\WSOCK32.dll 0x75030000 0x13000 5.00.2195.2780 c:\winnt\system32\WS2_32.DLL 0x78000000 0x46000 6.01.9359.0000 C:\WINNT\system32\MSVCRT.DLL 0x77db0000 0x5c000 5.00.2195.4453 C:\WINNT\system32\ADVAPI32.DLL 0x77d40000 0x70000 5.00.2195.4266 C:\WINNT\system32\RPCRT4.DLL 0x75020000 0x8000 5.00.2134.0001 c:\winnt\system32\WS2HELP.DLL 0x785c0000 0xc000 5.00.2195.2871 C:\WINNT\System32\rnr20.dll 0x77e10000 0x64000 5.00.2195.4314 C:\WINNT\system32\USER32.DLL 0x77f40000 0x3c000 5.00.2195.3914 C:\WINNT\system32\GDI32.DLL 0x77980000 0x24000 5.00.2195.4141 c:\winnt\system32\DNSAPI.DLL 0x77340000 0x13000 5.00.2173.0002 c:\winnt\system32\iphlpapi.dll
Searching the Registry revealed that the sqldiagncv process was being started as a service named sqldiagmsrv:
sqldiagmsrv Type = REG_DWORD 0x00000010 Start = REG_DWORD 0x00000002 ErrorControl = REG_DWORD 0x00000001 ImagePath = REG_EXPAND_SZ c:\winnt\system32\sqldiagmsrv.exe DisplayName = sqldiagmsrv ObjectName = LocalSystem Parameters Application = c:\winnt\system32\sqldiagncv.exe -I -d -p 12323 -t -e cmd.exe
The last write time of this Registry key was consistent with the intruder's other activities on the system:
D:\> keytime  system/currentcontrolset/services/sqldiagmsrv HKEY_LOCAL_MACHINE\system/currentcontrolset/services/sqldiagmsrv, Wed 4/3/2002 14:21:09:971
A copy of the sqldiagncv executable was placed on an analysis system for further inspection and it quickly became apparent that it was netcat:
C:\WINNT\system32>sqldiagncv -h [v1.10 NT] connect to somewhere: nc [-options] hostname port[s] [ports] listen for inbound: nc -l -p port [options] [hostname] [port] options: -d detach from console, stealth mode -e prog inbound program to
exec[dangerous!!] -g gateway source-routing hop point[s], up to 8 -G num source-routing pointer: 4, 8, 12, ... -h this cruft -i secs delay interval for lines sent, ports scanned -I listen mode, for inbound connects -L listen harder, re-listen on socket close -n numeric-only IP addresses, no DNS -o file hex dump of traffic -p port local port number -r randomize local and remote ports -s addr local source address -t answer TELNET negotiation -u UDP mode -v verbose [use twice to be more verbose] -w secs timeout for connects and final net reads -z zero-I/O mode [used for scanning] port numberscan be individual or ranges: m-n [inclusive]
In summary, the intruder used the Windows 2000 system to launch an attack against the SSH server on an internal UNIX machine, thus bypassing the firewall which did not allow connections to the SSH server from the Internet.
In some instances, it may be desirable to capture data in memory relating to a particular process using the pmdump  utility. For instance, the following commands show pmdump being used to copy the contents of memory relating to a Pretty Good Privacy (PGP) process:
D:\>pslist pgptray Name Pid Pri Thd Hnd Mem Elapsed Time PGPtray 1332 8 7 150 1264 2:20:33.466 D:\>pmdump 1332 d:\evidence\pgptray.mem
The resulting memory dump file may contain a PGP passphrase or data in unencrypted form.
Although the full contents memory can be dumped into a file using dd (e.g. dd </dev/mem > host.mainmemory), this
% ps -aux I more USER PID %CPU %MEM SZ RSS TT S START TIME COMMAND root 3 0.4 0.0 0 0 ? S Apr 25 64:39 fsflush root 199 0.3 0.2 4800 1488 ? S Apr 25 2:14 /usr/sbin/syslogd root 3085 0.2 0.2 2592 1544 ? S 14:07:12 0:00 /usr/lib/sendmail root 1 0.1 0.1 1328 288 ? S Apr 25 4:03 /etc/init - root 3168 0.1 0.1 1208 816 pts/5 O 14:07:27 0:00 ps -aux root 2704 0.1 0.2 2096 1464 ? S 14:05:37 0:00 /usr/local/etc/sniffer root 163 0.0 0.1 1776 824 ? S Apr 25 0:19 /usr/sbin/inetd -s root 132 0.0 0.1 2008 584 ? S Apr 25 0:00 /usr/sbin/keyserv root 213 0.0 0.1 1624 776 ? S Apr 25 0:16 /usr/sbin/cron root 239 0.0 0.1 904 384 ? S Apr 25 0:07 /usr/lib/utmpd
Additional information about each process, including a list of files and sockets that they are using, can be obtained using the lsof utility. Much of the detail provided by lsof may not be useful in most cases, such as which libraries are being accessed by each process. However, lsof can be useful for finding programs and files created by an intruder and can be compared with the output from ps to find discrepancies caused by rootkits. If a particularly interesting process appears in this list like "sniffer" or "destroyer," an investigator might want to take a closer look. Some types of UNIX allow one to save and view the contents of RAM that is associated with a particular program using the "gcore" command.
Another approach to examining processes on a UNIX system is through the /proc virtual file system. For instance, the following files on a Linux system are linked with the command line parameters, memory contents, and other details associated with a running netcat process:
$ ls -l /proc/1104 total 0 -r--r--r-- 1 eco eco 0 May 17 12:36 cmdline lrwxrwxrwx 1 eco eco 0 May 17 12:36 cwd -> /usr/local/bin -r-------- 1 eco eco 0 May 17 12:36 environ lrwxrwxrwx 1 eco eco 0 May 17 12:36 exe -> /usr/sbin/nc dr-x----- 2 eco eco 0 May 17 12:36 fd -r--r--r-- 1 eco eco 0 May 17 12:36 maps -rw------ 1 eco eco 0 May 17 12:36 mem -r--r--r-- 1 eco eco 0 May 17 12:36 mounts lrwxrwxrwx 1 eco eco 0 May 17 12:36 root -> / -r--r--r-- 1 eco eco 0 May 17 12:36 stat -r--r--r-- 1 eco eco 0 May 17 12:36 statm -r--r--r -- 1 eco eco 0 May 17 12:36 status $ more /proc/1104/cmdline /usr/sbin/nc-l-p31337-t
The grave-robber program in The
-rw-r--r-- 1 root root 1558129 May 30 18:50 coroner.log -rw-r--r-- 1 root root 154596 May 30 18:50 MD5_all -rw-r--r-- 1 root root 5618 May 30 18:50 error.log drwx------ 2 root root 4096 May 30 18:50 trust drwx------ 2 root root 4096 May 30 18:50 user vault drwx------ 10 root root 4096 May 30 18:49 conf_vault -rw-r--r-- 1 root root 2939919 May30 18:48 body drwx------- 2 root root 4096 May 30 18:48 command out drwx------ 2 root root 8192 May 30 18:48 icat drwx------ 2 root root 8192 May 30 18:47 proc drwx------ 2 root root 4096 May 30 18:47 removed_but_running drwx------ 2 root root 16384 May 30 18:47 pcat -rw-r--r-- 1 root root 10470 May 30 18:45 body.S
The "coroner.log" documents each action taken by grave-robber along with the date and time. Extracted data, such as recovered files, and process memory obtained using pcat and from the /proc virtual file system are organized into directories. The output of certain commands like lsof and ps are saved in the "command_out" directory and a mactime database (a.k.a. body file) of all files on the system is created. System configuration files and other files of interest are also preserved. Additionally, grave-robber calculates the MD5 values of all files, including the file containing the MD5 values. Even though a log file is created when grave-robber is run, it is advisable to document the process by taking notes and using the script command as discussed in Chapter 15.
As demonstrated in previous case examples, intruders use the Registry to ensure that programs they have installed stay running, even after the system is rebooted. For instance, Trojan horse programs often have associated entries in the Registry. The most common locations in the Registry for Trojans are:
HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\Run HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\RunOnce HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\Current Version\RunOnceEx HKEY_LOCAL_MACHINE\Software\Microsoft\Windows\CurrentVersion\RunServices HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\Run HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnce HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunOnceEx HKEY_CURRENT_USER\Software\Microsoft\Windows\CurrentVersion\RunServices
This list is not exhaustive since intruders regularly think of new ways to utilize the Registry such as making entries in the following keys:
HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Session Manager\KnownDLLs HKEY_CLASSES_ROOT\exefile\shell\open\command HKEY_LOCAL_MACHINE\Software\Classes\exefile\shell\open\command
The default Registry value for version 7.2.1 of SubSeven is "WinLoader" and for Back Orifice 2000 it is "UMGR32," but these can be modified to make them harder to detect. Also, some Trojan horse programs do not just use the Registry. For instance, Subseven can be configured to start using entries in WIN.INI (e.g. run = subseven.exe) and SYSTEM.INI (e.g. shell = explorer.exe subseven.exe).
When it is necessary to make a bitstream copy of the hard drive on a compromised system, it is possible to do so over the network. For example, the following shows one partition on "host13" being
host 13# df -k Filesystem kbytes used avail capacity Mounted on /proc 0 0 0 0% /proc /dev/dsk/c0t3d0s0 134335 30698 90204 26% / /dev/dsk/c0t3d0s6 737894 461662 217201 69% /usr fd 0 0 0 0% /dev/fd swap 139944 29044 110900 21% /tmp host13# dd bs = 4096 if = /dev/dsk/c0t3d0s0 nc examiner 3416 examiner# nc -l -p 3416 > host13-c0t3d0s0.dd examiner# mount -o ro,loop,ufstype = sun -t ufs host13-c0t3d0s0.dd /mnt/host13
There are other
When investigating computer intrusions, it is often necessary to inspect files closely to determine what they are and how to interpret them. One approach to classifying files placed on a system by an intruder is to search the Internet for files with similar characteristics. For instance, the denial of service attack tools that were used to attack Yahoo and other large Internet sites contain information that can be useful for locating the source of the attacks. For instance, the following lines can be extracted from a denial of service tool called "trin00." The IP addresses at the end
socket bind recvfrom %s %s %s alf3YWfOhw.V. PONG *HELLO* 10.154.101.4 126.96.36.199
In addition to classifying a certain piece of digital evidence, it is often desirable to find unique characteristics that differentiate a given piece of digital evidence from other, similar pieces of digital data. In particular, it is very desirable to be able to determine the source of a piece of digital evidence. For instance, being able to show that a given sample of digital evidence originated on a suspect's computer could be enough to connect the suspect with the crime.
CASE EXAMPLE (LONDON 2002):
21-year-old Samir Rana, nicknamed "t0rner," was arrested following a year-long investigation into the creation of the Linux rootkit called "tOrnkit" and on suspicion of being a leading member of the infamous hacker
Executable version of keytime.pl from