Forensic Analysis in Action

		Chapter 11 - Honeypot Data Analysis Honeypots for Windows by Roger A. Grimes Apress 2005

This section uses two real-life honeypot exploits to demonstrate the structured forensic analysis of honeypots. The first is an example of a low-interaction honeypot, and the second is from aWindows 2000 real honeypot.

A KFSensor Honeypot

I frequently recommend honeypots as an EWS within a network—a canary in the coal mine sort of thing. Since collecting complete hacker or malware information is necessary, I can use alow- to medium-emulation honeypot. My usual choice is Honeyd (covered in Chapters 5 to 7) or KFSensor (covered in Chapter 8). If the client has the money, I’ll always suggest using KFSensor. It’s the best Windows honeypot offering, full of features, and easy to set up. This forensic example follows three days of honeypot activity on a KFSensor honeypot on a DMZ.

Because there were legitimate public services on the DMZ, the following ports were allowed through the external firewall: 21, 22, 25, 53, 80, 81, 443, 1433, 1434, 8080, and a few others. Ethereal was used as the network sniffer. I was using RDP (which is encrypted and authenticated by default) over nonstandard ports to administer the remote honeypot. For that reason, some traffic from my banneretcs.com domain at random intervals was detected by the honeypot.

Initial Review

The honeypot went live at 9:16 A.M. The first probe came 70 minutes after the honeynet was placed into production. Over the next three days, the honeypot recorded 1,022 different events over almost all the allowed ports, as shown in Figure 11-3. Ethereal recorded 120MB of data in almost 900,000 packets. The honeynet averaged about four packets per second.

Figure 11-3: Main KFSensor screen showing some of the 1,022 events

Note 

In this described session, Ethereal had to be stopped shortly after it was started to put in an Ethereal filter to filter out irrelevant bridge-related broadcasts. Filtering out legitimate traffic and fine-tuning the sniffer is a normal part of any honeypot setup.

My cursory review of the KFSensor alert messages showed most traffic was to the IIS service, followed by traffic to Microsoft SQL Server ports. I could readily see buffer overflow and directory transversal attacks against IIS in rapid succession. I was surprised that I didn’t have more port 25 traffic, as spammers and spam worms are rampant these days.

Analysis

I fed the Ethereal capture files into Snort. Snort alerted on 32 different types of exploits, again most related to HTTP.

The Ethereal capture files were three separate capture files, one for each day. I used Ethereal’s Mergecap.exe command-line program to merge all three files into one larger file for easier analysis. I used the following commands:

 Mergecap.exe -v -w c:\logs\ethereal.cap - c:\ethereal_day1.cap  - c:\ethereal_day2.cap - c:\ethereal_day3.cap 

This process took about five minutes on a mid-range Pentium computer.

I opened the larger Ethereal.cap capture file in Ethereal (the GUI product), and it took a little over one minute to load. My Ethereal summary distribution reports showed traffic came from 47 separate source IP addresses (including two that were related to my remote monitoring). Aprotocol distribution analysis report took several minutes to run, as shown in Figure 11-4.

Figure 11-4: Ethereal generating a protocol distribution report

The protocol distribution report revealed that HTTP requests accounted for nearly 65% of all traffic, as shown in Figure 11-5, followed by small amounts of SMTP and FTP traffic.

Figure 11-5: Portion of Ethereal protocol distribution report

Looking at the KFSensor logs, I saw that the first sustained attack was an IIS buffer overflow and directory transversal attack. As Figure 11-6 shows, the honeypot got 15 different exploit attempts in two seconds. This told me that the attack was either automated by malware or by a scanning script. The attacks might have been directed because all exploit attempts were Windows and IIS-related, but the fact that they were automated decreased the chances of a directed attack. There were no Apache exploits in the bunch. Because of the C:\Winnt directory reference in most commands, I saw that the hackers were attempting to attack either Windows NT 4.0 and IIS 4 or Windows 2000 and IIS 5. Windows XP Professional and Server 2003 use C:\Windows as the default OS directory name, unless the computer was upgraded.

Figure 11-6: KFSensor logs showing the first IIS attack

I opened one of the logged events generated by an attack and looked at the content detail, as shown in Figure 11-7. I then copied the content from the top line of the Received box (showing what was sent by the attacker) and used Google to search on the string. Google revealed that the attack traffic was from a Nimba-style worm.

Figure 11-7: KFSensor log detail for one of the attacks

Windows Media Services Buffer Overflow Attack

Another single IIS probe was looking for the Nsiislog.dll file. This is a Windows Media Services buffer overflow vulnerability in IIS 5 and Windows 2000. It was reported and patched in 2003. The attacker tried only once and did not come back.

Using Ethereal’s View Filter feature, I looked at just the packets associated with the single source IP address, as shown in Figure 11-8. There are a total of five packets, but only one with any real payload data. The other packets are TCP handshake and Window size negotiations. Interestingly, there are other related packets, such as the ACK,SYN packet sent as part of the three-way TCP handshake. It isn’t listed because the filter is one way, so it does not show traffic headed back to the attacker. In order to capture all traffic, I would need to modify the filter to look for all traffic headed to or from the source IP address.

Figure 11-8: Ethereal capture showing Windows Media Services buffer overflow attack

SQL Server SA Password-Guessing Attack

There were dozens of SQL Slammer worm attempts and FTP password guessing attempts. KFSensor captured all of the FTP password attempts, the vast majority of which used administrator as the logon name and password as the password. There was also a SQL Server password-guessing attack. The attacker sent 48 different password guesses in 25 seconds. Each used the logon name SA and tried various passwords, including sa, blank, password, sa12, sa123, 123, 12345, 1, 1234567, and super. Each password was attempted three times. This is another automated attack, of course. A manual hacker would have tried once, and then moved on.

Open-Relay Attack

On day three at 9:26 A.M., the first probe from a spammer (actually a spam bot) arrived. Up until this point, the honeypot had generated about a 100 separate alerts. Traffic was bursty and random, with long periods of nothing happening. That was about to change, as shown in Figure 11-9. Two packets were sent from source IP address 220.242.53.140 to port 80 using the CONNECT command. The HTTP CONNECT command was originally introduced to allow access to HTTPS servers, but it can also be used to relay any type of traffic to any destination and port. If a server is found that supports HTTP CONNECTs to external IP addresses, the spammers have found an open relay from which to send spam.

Figure 11-9: KFSensor’s logs of the spam open relay

The first two packets, I later learned, were test packets to see if an open relay existed. KFSensor allowed enough response to happen to fool the spammer. This was a spam bot designed specifically to find open proxies. It sent two test packets, and then reported the found open relay back to its web site and database. The spam bot web site acts as a coordinator for all open proxies on the Internet that spammers and hackers can use. Sixteen minutes later, traffic from the first spammer arrived; one minute later, the second arrived; and so on. Once my open relay was found, all the spammers paying the spam bot service were notified of the new victim. One by one, spammers began to appear. Within minutes, the network bandwidth of the DMZ had been fully utilized (this was when no e-mail was really leaving the DMZ, which would have doubled the traffic).

I was able to see the spam being sent (more of the mortgage variety than the porn or Viagra type) and who it was being sent to (random yahoo.com addresses). There was so much spam occurring that the different spammers were literally fighting it out to make more connections to the honeypot. If wanted to, I could have researched the spam-sending machines, but they would have no doubt led to other innocent compromised machines. Still, if I were tasked with prosecuting spammers, this would be an intriguing method to use.

Lessons Learned

This medium-emulation honeypot suffered a fair amount of attacks in the three days it was up. Most attacks were automated, and all could be defeated with simple security measures.

The customer learned that attacks were happening on the DMZ, and password guesses and old exploits were frequent. Using longer and complex passwords would have defeated many of these attacks, as well as, up-to-date patching. The biggest current threat appeared to be spammers and their spam worm creations. Once they detected the availability of an open-proxy server, almost every other attack was pushed out as the spammers took control.

The WhiteDoe Real Honeypot

I was hired to find out how hackers were successfully breaking in to a computer belonging to a large public school system (I’ll call it WhiteDoe Public School System). The school system administrator was very good at her job, but not trained in computer security or antihacking techniques. She had found a hacked server a few months before my arrival. She hired an outside firm to find out how the hackers were breaking in and to remove the malicious code. The outside firm couldn’t find the hackers, so they called in an “expert.” My client calmly recalled the 15-year-old teenager dressed in dark, gothic clothes arriving the next day. He was “one of the best hackers in the world,” she was told. She was made to leave the computer room as he practiced his craft. Of course, months later, the system was still hacked. She knew it was still hacked because of the lack of free disk space.

After the hacker wunderkind had visited, the server quickly ran out of available free disk space. The client added two new 9GB hard drives. Two days later, both drives were out of room. This was despite the fact that the server hosted only a small web site and ten Exchange Server users. Clearly, the hackers loved the new disk space, and they were greedy. A cursory review of the server revealed nothing suspicious, other than the low drive space. The client and I could not find the files that were taking up all the space. An up-to-date antivirus scan was executed, and it found nothing.

I created a production honeypot that mimicked the hacked Windows server. The real honeypot was running Windows Server 2003 Standard Edition, without hotfixes or service packs, and connected to the same domain as the other server. Identical local user accounts were created, along with identical passwords. I installed Exchange Server and IIS on the server. I used atransparent bridge and a managed switch (as discussed in Chapter 2) to isolate the honeypot and the hacked server into the same virtual LAN. The monitoring PC ran Snort and Ethereal. I took a baseline on the honeypot and on the honeypot network. Then the client and I went to lunch and waited. I wasn’t even through with my hamburger when my pager went off.

Exploit Code Happens

Ethereal sent my pager a SMS message indicating activity on the honeypot. We raced back to the honeypot. We could not see anything moving on the console screen, but disk activity was high and Ethereal was busy capturing packets. Snort did not alert us at any time. In a few hours, we had hundreds of thousands of captured packets. All traffic headed to the honeypot was coming directly from the previously exploited server. After a few hours, we took the honeypot and the exploited server offline.

New snapshots were taken and compared against the original. In summary, dozens of new folders had been created, 8GB of data transferred, and two new services installed. The hackers had come in through using the administrator account and password. This proved that somehow the hackers had cracked the administrator password. It would need to be changed. After the hackers got their initial access, they immediately created two new local accounts on the server, for later access (in case we changed the administrator password). They then installed a remote-access trojan called Ghost RAdmin (http://www3.ca.com/securityadvisor/pest/pest.aspx?id=453084944). Ghost RAdmin is based on a legitimate remote-control program called Remote Administrator (http://securityresponse.symantec.com/avcenter/venc/data/remacc.radmin.html), but it has been co-opted by malicious hackers.

The hackers then set up an IRC server running on port 6667 and an FTP server. Ghost RAdmin was used to copy gigabytes of pirated DVDs, DVD-Rs, games, appz (rogue applications), and MP3s. The directory structure was hidden under C:\Windows\Fonts. The hackers created subfolders under that folder called \.Fonts1\.system\.1, as shown in Figure 11-10. Beginning the folders with a period made them appear invisible in Windows Explorer, although when I typed in their absolute names, the folders appeared.

Figure 11-10: Hacker’s malicious folder structure

Actually, only the .system folder was invisible, and consequently, anything below it. For whatever reason, .Fonts1 and the other subfolders were visible. When I first saw the .Fonts1 folder under the regular Fonts folder, I wasn’t sure if the folder was malicious or if it was just some sort of Windows temporary folder (or something like that). Fortunately, my baseline comparisons told me it was the former.

When I explored the new directories, I found German-language versions of Microsoft Encarta Professional 2004 and Blue Crush DVD, among other files. The hackers were clearly intending to set up the hacked server as an IRC-accessible FTP site. Ethereal confirmed my suspicion when I saw text headed to an IRC server network advertising the new FTP server and DVDs. I found trojan and configuration files in the bogus .system directory, as shown in Figure 11-11.

Figure 11-11: Bogus .system directory

Here, I found a trojan version of Svchost.exe, named after a legitimate Windows file normally found in C:\Windows\System32. The legitimate version is used to launch the different Windows RPC services. The trojan version, at over 560KB, is significantly bigger than the normal Svchost.exe, which is 7KB or 8KB. The trojan could not have overwritten the legitimate version of Svchost.exe in the System32 directory because of Windows File Protection (WFP), so it wrote it to a bogus new directory instead. It then appeared in Task Manager as a process called Svchost.exe, along with all the other legitimate processes of the same name.

Ethereal picked up a significant amount of IRC and FTP traffic. The IRC traffic was encrypted or encoded. In the bogus directory, I found an IRC-related file affiliated with the trojan called R_bot.ini, as shown in Figure 11-12. It contained information about the involved IRC network. It listed the involved servers, port addresses, and more important, the IRC channel name being used.

Figure 11-12: R_bot.ini IRC configuration file

As you can see in Figure 11-12, the IRC channel was called FunPink and the password was alfred645. I immediately used that information to join the chat and to sniff more traffic. I was glad to find (and to document) that all traffic was headed to and from the bogus directories. The hackers were using the school’s computers as an FTP server, but they didn’t appear to be taking confidential school files or e-mail messages and passing them to remote locations. Sniffing more, Ifound out the location of the other files on the original server. I had missed them before in my initial analysis, but now I knew why. The original server had directories created using unprintable German Unicode characters. Once we knew what we were looking for, removing the hackers’ files and the trojans was a piece of cake.

Lessons Learned

There may have been other ways to solve this particular problem, but the honeypot proved invaluable to our investigation. The encrypted IRC channel would not have been readily apparent if I did not learn about the channel name and password.

We learned that hackers are readily using other people’s computers to set up FTP and IRC servers, and they don’t care if they run out of free space. At first, I thought the latter fact to be a hacker mistake. If the hackers watched the free disk space, there is a good chance the client would not have noticed and would not have called me to investigate. But when you are robbing and using other people’s resources, if you get kicked off one computer, you find another.

We also learned that Windows Explorer doesn’t handle German Unicode characters well or even directories beginning with periods (such as .system). The client also learned that the uber-hacker teenager probably wasn’t the best choice for the job. A structured analytical approach will usually prevail in the end.