Monitoring

		Chapter 10 - Honeypot Monitoring Honeypots for Windows by Roger A. Grimes Apress 2005

Once a baseline has been documented, the next step is to activate monitoring systems. Monitoring is the process of collecting all information headed to and from the honeypot system.

In-Band vs. Out-of-Band Monitoring

Monitoring systems can be in-band or out-of-band. In-band monitoring is any monitoring system that functions as part of the honeypot system or requires a fully functional honeypot in order to collect data. Examples of in-band monitoring systems include Windows event logs, object access auditing, and keystroke logging.

The biggest benefit of in-band monitoring is the ability to record data modifications and communications close to the source. For instance, if the hacker is using an encrypted network communications channel to send commands, only an in-band mechanism has a chance of recording the manipulations. However, in-band monitoring is risky because the hacker can alter or delete the data collected within the system. Hackers frequently erase their exploit tracks and clear event log files. Malicious malware can utilize stealth routines modifying OS APIs to hide their activities.

Whenever possible, in-band monitoring should report its data to external computer management systems. For instance, you can run the Windows Performance Monitoring console on the local machine, but you would be better off running it externally and connecting to the remote computer. There is less of a chance that the hacker will notice, and the operational effects of the monitoring software will be reduced. In-band monitoring can sometimes cause differences in the very statistic that you are trying to monitor.

Out-of-band monitoring systems operate outside the confines of the honeypot system and can capture information even if the honeypot system itself is down. Out-of-band monitoring can be done by an IDS, packet-capturing utility, or any of your network devices. Out-of-band monitoring isn’t as susceptible to hacker compromise. External log files, if properly protected, are difficult for the hacker to even notice, much less modify. If planned correctly, it can be nearly impossible for hackers to even know their activities are being recorded. As covered in Chapter 2, if your honeypot system contains a managed Ethernet switch, port mirroring can allow out-of-band monitoring that cannot be detected.

Some monitoring utilities attempt to be both in-band and out-of-band at once. Sebek (covered next) runs in-band on the honeypot system, but operates covertly to cover its tracks. But even covert in-band systems can be detected if the hacker is looking for them, and thus are riskier than true out-of-band systems.

The key to a good monitoring strategy is to use a layered, complementary mix of in-band and out-of-band mechanisms. What one misses, the other gets. In-band solutions may capture everything typed on a system and defeat encrypted network channels, but they fail to capture all network traffic or attacks against other hosts. A vulnerability scan against your honeypot can be used to illustrate the proper approach. In-band systems will probably catch only successful exploits against the system, whereas out-of-band systems would capture all traffic. An in-band system would detail the resulting exploit as it modified the host system, but might miss similar tries against ports and services that were not vulnerable to the attack code.

The first step in any honeypot system monitoring plan is to take in-band and out-of-band baseline measurements.

Monitoring Programs

There are hundreds of monitoring programs and utilities that you can use to monitor your Windows honeypot. Here, I’ll provide just a sampling. Also, many of the utilities listed in the previous section about taking baselines provide monitoring capabilities, too.

Sebek

There have been many console keystroke loggers in the Unix/Linux world for years, but only recently has the same functionality been ported to Windows.

Sebek (http://www.honeynet.org/tools/sebek) is a tool built specifically for honeypots to solve the basic monitoring problem described in the previous section: in-band monitoring captures everything the hacker does to a particular honeypot, but an in-band tool is at risk of hacker detection and manipulation. Sebek is a stealth logger, pulling tricks from Unix trojan rootkits. Michael Davis, who ported the Windows version of Honeyd, also ported Sebek from its Unix parent. Unfortunately, like the Windows version of Honeyd, the Windows version of Sebek doesn’t have all the functionality of the Unix version. Still, it is valuable for what it does. When installed on a real Windows honeypot, it will monitor and transmit any commands initiated using the Cmd.exe console. Mr. Davis eventually hopes to add Registry and file system monitoring to Sebek.

Sebek will hide itself (the specific file is called Sebek.sys) on the machine and specifically conceal the traffic it creates from prying hacker eyes. When in memory, Sebek looks for system calls to enumerate loaded drivers and processes. If another process tries to list it, Sebek will deny the enumeration. It will also block any requests to the file system or Registry where Sebek is stored and loaded. (In order to locate or unload Sebek, you must boot into Safe Mode or use the Recovery Console.) Although Sebek will fool most hackers, it can be found if the hackers are looking for the right clues.

During the setup of Sebek, you customize the Sebek installation to place a random “magic” number in every packet that it sends from the remote honeypot to the monitoring host. Sebek will monitor outgoing network traffic for packets with the magic number and deny any requests to external processes.

The Sebek installation process involves running a server setup on a monitoring workstation and a separate client setup on the honeypot system. The Sebek server is the central logging system that collects all the Sebek packets from all participating honeypots. The server is made up of three tools:

• Sbk_extract collects Sebek packets for analysis.

• Sbk_ks_log.pl is a Perl script that takes the Sebek packets and displays the attacker’s keystrokes to the screen.

• Sbk_upload.pl is a Perl script that takes the Sebek packets and uploads them to a local or remote database for more advanced analysis.

Note 

Sebek requires Cygwin or Active Perl to be installed to support the Perl scripts.

Commercial Keylogger

There are dozens of commercial keylogger programs available that can be used on a honeypot to collect keystrokes, mouse clicks, emails, chats, and almost any PC activity. Many commercial keylogging programs were made as programs that parents or spouses could install and run to track the online activities of children or dubious spouses. iOpus Software’s STARR (http://www.spy-software-directory.com/starr.asp) and Spector (http://www.spectorsoft.com) are among the most popular choices. There are many web pages listing various commercial spyware programs including http://www.spy-software-review.com.

Sunbelt Software has a nice list of the different products available (http://research.sunbelt-software.com/threat_library_list.cfm?category=Commercial%20Key%20Logger). Some of those products are legitimate software products made by legitimate companies. Others on the list, however, are spyware programs made by less reputable companies. Beware and research before you buy. At Keylogger.org (http://www.keylogger.org), you can download a dozen different keylogger programs to try or buy a $19.95 CD-ROM to save the downloading time. Although commercial keyloggers aren’t specifically made for honeypots, most make at least some attempt to hide themselves against easy discovery.

Sysinternal Utilities

Sysinternal’s (http://www.sysinternals.com) Mark Russinovich and Bryce Cogswell deserve an honorary Internet Oscar for all the free and cool utilities they’ve given the system administrator world. Their utilities are unmatched for their usefulness and detail. Most Sysinternal tools come in Windows 9x and later versions (including 64-bit versions). Leading the way are these utilities:

• Filemon: This utility monitors and displays file system activity on a system in real time. Its timestamping feature will show you precisely when every open, read, write, or deletion happens, and its status column tells you the outcome. The output window can be saved to a file for off-line viewing. It has full search and filtering capabilities.

• Regmon: This monitors and displays Registry accesses (see Figure 10-5). If you’ve never used a utility like Regmon before, you’ll be surprised by how much activity is really occurring on aWindows system supposedly not doing anything. Most first-time users are quickly overwhelmed by the amount of data collected. Fortunately, Regmon, like Filemon, contains filtering to limit your investigation to just what interests you.

  
  Figure 10-5: Sysinternal’s Regmon utility

• Autoruns: This is one of my new favorites. When executed on a system, it shows you nearly every Windows Registry and file location (certainly the most popular ones used by hackers and malware) where programs can be automatically started from when Windows starts. It will show you the Registry autorun areas, browser helper objects, Windows login programs, and even services and DLL files. You can disable and enable any of the found programs on the fly, and a great feature allows you to filter out all Microsoft signed executable (leaving potential malware behind).

• Process Explorer: This will show you what programs and processes are running and the computer system resources devoted to each process. Have you ever wondered what file or directory was opened by a program? Have you ever wanted to track which DLL files are associated with a single application and track the filename on the disk to the memory image? Process Explorer can do that and more.

• PsTools: This is a set of Windows administration tools that you can download as a package or individually. It includes the utilities listed in Table 10-2.

Table 10-2: Sysinternal PsTools Utilities

Name	Description
PsExec	Allows you to execute a process or program remotely
PsFile	Shows files opened remotely
PsGetSid	Shows the SID of a computer or user account
PsKill	Can kill a local or remote process even when Task Manager says it cannot
PsInfo	Lists detailed information about a system
PsList	Lists detailed information about a process
PsLoggedOn	Shows who is logged in locally and via file and printer sharing
PsLogList	Collects event log messages
PsPasswd	Changes account passwords
PsService	Shows and manages local and remote services
PsShutdown	Shuts down or reboots local or remote computers
PsSuspend	Lets you suspend processes
PsUptime	Shows how long a system has been running since its last reboot

And there are dozens of other useful, and free, Sysinternal utilities. Most of Sysinternal’s utilities were not made for honeypot use. They don’t hide their activities and contain no stealth routines. But don’t overlook their importance, because they are excellent for baselining and for later forensic analysis.

Foundstone Utilities

Although I have already mentioned the Fport and Vision port mappers in the “Network Baselines” section, Foundstone (http://www.foundstone.com) has a host of other free forensic utilities that may prove useful for documenting and analyzing honeypot systems.

• Pasco and Galeta: Pasco is a utility that will display Internet Explorer activity. If the hackers used the honeypot to surf the Web, Pasco can make documenting what they did easier. Arelated tool, called Galleta, examines Internet Explorer cookies.

• Rifiuti: This tool examines the content of the Info2 file in the Recycle Bin. Because the Recycle Bin is rarely checked and not often scanned (by older versions) of antivirus software, hackers can potentially hide their warez (illegal or malicious software) there.

• NTLast: This is aWindows security log analyzer. It tracks who has gained access to a system, and then documents the details.

• Forensic Toolkit: This is a file properties analyzer. It examines files for unauthorized activity. It will list files by their last access times, scan the disk for hidden files or alternative data streams, dump file and permission attributes, discover altered access control lists (ACLs), reveal weak NULL sessions, and report on audited files.

• ShoWin: This utility shows information about Windows and reveals found passwords.

• BinText: This tool finds ASCII and Unicode strings in files. It’s great for searching potentially dangerous files for the telltale signs of a malware code writer’s signature.

SecurIT Informatique Inc. Utilities

SecurIT (http://iquebec.ifrance.com/securit/) has a collection of open-source and commercial utilities perfect for honeypot (or IDS) data collection.

• Open-source ComLog: This is a stealth command-prompt capture utility for Windows NT and above OSs. It’s ideal for maintaining a log history of commands typed at the command prompt, or for capturing intruder activity with IIS abuse, Netcat tunnels, or shell-shoveling attacks. Most command shells are text-based. ComLog works by replacing the normal Cmd.exe command shell, which is renamed cm_.exe for normal execution. ComLog can be used with SecureIT’s LogAgent for automatic forwarding of the logs to a central location or monitoring console. Unfortunately, for ComLog to work on Windows 2000 and above, you need to disable Windows File Protection (http://www.winnetmag.com/Article/ArticleID/38777/38777.html or http://www.mvps.org/PracticallyNerded/Windows/2K/2K_Disable_fileprotect.htm). While I don’t normally recommend disabling Windows File Protection, the usefulness of ComLog is a good argument for doing so on a honeypot.

  Note 

  Netcat tunnel refers to the use of Netcat to open file-transfer sessions between a compromised machine and the remote intruder. After gaining access to the computer, the first thing the hacker will often do is send and open a command shell on the exploited computer (called shoveling a shell). Then the hacker can type in new commands, map drives, and send and receive files.

• Commercial ComLog: This version works in the same way as the popular open-source version, except that it can be uniquely configured via a configuration file. You can choose the filename for cm_.exe and specify pattern strings to be hidden from the monitored users (to hide other in-band monitoring and logging processes).

• ADSScan: This is an alternate data stream checker. NTFS files can contain one or more data streams. In normal circumstances, applications can use the alternate data streams, which are not usually readily visible to most programs and security tools, to hold multiple types of data related to one file. For instance, WordPad and Microsoft Office applications can store the document history and previous versions of the same document in the alternate data streams. Malware writers started using alternate data streams to hide their program’s maliciousness, even though most antivirus scanners can check in alternate data streams if configured to do so. ADSScan will let you scan and view a file’s alternate data streams.

• IntegCheck: This is a file system integrity checker, which is great for creating baselines or noting differences.

• Log tools: These are tools for detecting various types of intrusions—LogUser for invalid user accounts, LogShares for nonallowed shares on a computer, LogServices for nonallowed services, LogStartup for suspicious autorun items, and LogProc for rogue processes running in memory.

• SecurIT Intrusion Detection Kit: This is an open-source collection of useful monitoring and log programs. It includes ADSScan, IntegCheck, and the Log utilities.

SecurIT, a Quebec-based company, has several other useful utilities, both open-source and commercial versions. As useful as the open-source versions are, the $99 commercial versions of the same programs will run in the background and are not readily viewable to the end user (although I would not call them stealth programs). Figure 10-6 shows several SecurIT utilities monitoring various system processes.

Figure 10-6: Several SecurIT utilities monitoring system processes

DiamondCS Utilities

Along with Port Explorer and OpenPorts, which I described in the “Network Baselines” section earlier in this chapter, DiamondCS (http://www.diamondcs.com.au/index.php?page=console) offers more than a dozen other useful forensic utilities. Here is a sampling:

• Autostart Viewer: This is a competitor to Sysinternal’s Autoruns. You can view, modify, and control programs that start automatically.

• Advanced Process Manipulation: This tool allows control over target processes.

• RegistryProt: This is a real-time monitor of Registry activity. It goes one step further than Sysinternal’s Regmon by providing Registry protection against modification.

• Passdump: This will dump asterisk-protected passwords typed in on the screen to a log file.

• SHA-160 Hash: This will calculate SHA-1 file hashes.

• IPList: This enumerates network interfaces. It shows all bound IP addresses and their net masks (a prettier version of Microsoft’s ROUTE PRINT command).

• XWhois: This is an advanced domain registration query tool that can be used to trace hackers.

• Sendmail: This is small footprint SMTP server that can be used to send alerts and other messages.

• Uptime: This determines how long the computer has been up since a reboot.

PC Magazine’s InCtrl5

InCtrl5 (http://www.pcmag.com/article2/0,4149,9882,00.asp) is one of PC Magazine’s most popular utilities and is used for forensic analysis of malware. Developed primarily for undoing installation program changes, InCtrl5 can be used to track changes to common Registry keys (HK_CU, HK_CC, and HK_CR) plus other text files, like Autoexec.bat and Config.sys.

Users run InCtrl5 to take a snapshot of the computer configuration, and then take another snapshot later on to compare against the previous baseline. Changes can be reported to HTML and CSV file formats. (A previous version, InCtrl4 did real-time reporting, but it required the use of an undocumented VxD file, so it could run on only Windows 2000.) InCtrl5 requires the name of the executable that it will track the changes to, so you cannot just tell it to track all changes made by any program, which limits its overall usefulness. Its primary use in a honeypot system environment is to track changes made by discovered malware. In most instances, this type of analysis is done during the cleanup phase of the honeypot when trying to discover all the hacker’s tricks.

Note 

Users may need to register and subscribe to PC Magazine’s periodical or web site to gain access to the downloadable files.

Protection for Monitoring Communications

Whenever possible, in-band monitoring traffic headed to external monitoring machines should be protected using signing and encryption. Signed data communications ensures the data hasn’t been tampered with en route between the honeypot system and the management workstation. Encrypting the data ensures that the hacker won’t be able to read the detection stream.

Many security monitoring tools use some form of Secure Shell (SSH), which equates to an encrypted telnet session. The most common Windows SSH program is Putty (http://www.chiark.greenend.org.uk/~sgtatham/putty). No matter which SSH program you use, make sure it is one of the recent versions coded to withstand the latest malicious exploits.

Windows default encryption communication’s protocol is IP Security (IPSec). IPSec can be enabled on any Windows computer 2000 and above. IPSec can be difficult to configure for the first-time user, but if used in conjunction with digital certificates (other authentication mechanisms include Kerberos and preshared secrets), it is quite secure. Microsoft has many excellent and detailed implementation guides (including http://www.microsoft.com/windowsserver2003/technologies/networking/ipsec/default.mspx) that can be found at http://www.microsoft.com/security/default.mspx by searching on the “IPSec” keyword. Microsoft also encourages the use of Secure Sockets Layer/Transport Layer Security (SSL/TLS), but it requires the installation and use of IIS. If you decide to use that method, you can use SSL on a port other than the default port of TCP 443.

Windows 2000 and above computers can also be remotely managed using the Remote Desktop Protocol (RDP). RDP is used for Windows’s various Terminal Services technologies including Remote Desktop for Administration (Windows Server 2003), Remote Desktop (Windows XP), and Terminal Server for Applications. RDP normally runs over TCP port 3389, but you can edit the Registry to make it work across any port (http://support.microsoft.com/default.aspx?scid=kb;en-us;555031). I use this trick on many of my publicly addressable honeypots. I place the port up high and random, usually above 40,000, where hackers don’t know what to make of the port. RDP encrypts the password and all traffic by default.