15.4 Filtering and Data Reduction

Investigations involving computers often result in a large amount of data, much of it unrelated to the crime under investigation. Also, when dealing with files containing captured network traffic, there may be privileged or confidential information that forensic examiners are required to ignore or remove. Therefore, data filtering and reduction are an essential part of any investigation involving networks, enabling a more efficient and thorough forensic analysis of the digital evidence.

Filtering out irrelevant data from log files may be as simple as extracting entries that match certain criteria such as a certain time period, an IP address, or failed logon events. For instance, the following output shows only failed logon events relating to the user "eco" extracted from a Windows NT Event Log using ntlast utility.^[5]

    C:\>ntlast -f -u eco -file e:\case1\dc2\sec.evt    eco        WORKSTN13              MY-DOMAIN           Sun Jan 19 11:00:11 am 2003    eco        WORKSTN10              MY-DOMAIN           Wed Jan 15 05:39:39 pm 2003

When examining established connections through a Cisco PIX firewall, it may be desirable to focus on one host rather than review every connection;

    pix01# show conn foreign 192.168.0.232 255.255.255.255    7354 in use, 24529 most used    TCP out 192.168.0.232:3129 in 172.16.1.23:80 idle 0:12:04 Bytes 45235 flags UIO    TCP out 192.168.0.232:3130 in 172.16.1.23:22 idle 0:00:01 Bytes 4395 flags UIO    TCP out 192.168.0.232:3131 in 172.16.1.23:443 idle 0:00:54 Bytes 9935 flags UIO

However, this approach to collecting evidence from a firewall violates the recommendation provided in the previous sections - collect first and filter and analyze later. Therefore, it is advisable to display all connections, logging the results into a file, and then searching these results for the entries of interest. As another example of data reduction, the following output shows windump being used to extract data relating to one IP address from a file containing network traffic relating to many computers.

    E:\case1\networktraffic>windump -r monitor1-01192003.dmp host 64.4.45.7    00:08:07.534671 64.4.45.7.80 > 192.168.1.102.1037: S 1378721726:1378721726(0)    ack 250897286 win 17316 <mss 1322,nop,nop,sackOK>    00:08:07.688663 64.4.45.7.80 > 192.168.1.102.1037: P 1:155(154) ack 338 win 16979    00:08:07.689768 64.4.45.7.80 > 192.168.1.102.1037: F 155:155(0) ack 338 win 16979    00:08:07.839232 64.4.45.7.80 > 192.168.1.102.1037: . ack 339 win 16979    00:08:07.942829 204.60.0.2.53 > 192.168.1.102.1038: 6 1/4/4 A 64.4.45.7 (208) (DF)    00:08:08.067639 64.4.45.7.80 > 192.168.1.102.1039: S 2707800119:2707800119(0)    ack 251070441 win 17316 <mss 1322,nop,nop,sackOK>    00:08:08.240567 64.4.45.7.80 > 192.168.1.102.1039: P 1:435(434) ack 410 win 16907    00:08:08.244832 64,4.45.7.80 >192.168.1.102.1039: .435:971(536) ack 410 win 16907    00:08:08.245727 64.4.45.7.80 >192.168.1.102.1039:.971:1073(102) ack 410 win 16907    00:08:08.371354 64.4.45.7.80 > 192.168.1.102.1039: . 1073:1609(536) ack 410 win    <cut for brevity>

Most commercial sniffers have the ability to create filters, only displaying packets that match certain criteria. Alternatively, ranking hosts based on the amount of data that they are sending and receiving can reveal one host that is involved in a suspiciously large amount of data transfer as shown in Table 15.2.

Table 15.2: Connections between hosts, ordered by total number of application bytes transferred. Data extracted from tcpdump file (available on book Web site) using Argus *"ramon -c -A -M Matrix"*. The same summary can be obtained using the NetIntercept "Traffic Load" report (available on the Web site).
SOURCE IP	DESTINATION IP	SOURCE BYTES	DESTINATION BYTES
192.168.0.5	207.68.162.250	49900	230869
192.168.0.5	207.68.162.24	47819	146996
192.168.0.5	65.54.228.250	12212	158032
192.168.0.5	207.68.172.245	12963	48012
192.168.0.5	65.54.208.222	11217	40002
192.168.0.5	208.185.54.22	2304	42975

Similarly, viewing the number of connections between hosts may be useful for traffic analysis as shown in Table 15.3.

Table 15.3: Communication between hosts, ordered by number of connections. Data extracted from tcpdump file using the NetIntercept "Top N" report (available on book Web site).
CONNECTIONS	SOURCE IP ADDRESS	DESTINATION IP ADDRESS
81	192.168.0.5	207.68.162.24
31	192.168.0.5	207.68.162.250
9	192.168.0.5	65.54.228.250
8	192.168.0.5	207.68.177.125
7	192.168.0.5	65.54.208.222

^[5]http://www.foundstone.com