Intrusion Detection Systems

Intrusion Detection Systems (IDS) play a critical role in the protection of the IT infrastructure Intrusion detection involves monitoring network traffic, detecting attempts to gain unauthorized access to a system or resource, and notifying the appropriate individuals so that counteractions can be taken. This section starts by discussing how IDS systems work; then IDS tools and products are discussed; and finally IDS evasion techniques are discussed.

IDS Types and Components

Objective:

Identify IDS components

Identify IDS alert types

Intrusion detection was really born in the 1980s when James Anderson put forth the concept in a paper titled "Computer Security Threat Monitoring and Surveillance." IDS systems can be divided into two broad categories: network-based intrusion-detection systems (NIDS) and host-based intrusion-detection systems (HIDS). Both can be configured to scan for attacks, track a hacker's movements, or alert an administrator to ongoing attacks. Most intrusion detection systems consist of more than one application or hardware device. IDS systems are composed of the following parts:

• Network sensors Detects and sends data to the system.
• Central monitoring system Processes and analyzes data sent from sensors.
• Report analysis Offers information about how to counteract a specific event.
• Database and storage components Performs trend analysis and stores the IP address and information about the attacker.
• Response box Inputs information from the previously listed components and forms an appropriate response.

The key to what type of activity the IDS will detect depends on where the network sensors are placed. This requires some consideration because after all, a sensor in the demilitarized zone (DMZ) will work well at detecting misuse there but will prove useless for attackers who are inside the network. Even when you have determined where to place sensors, they still require specific tuning. Without specific tuning, the sensor will generate alerts for all traffic that matches a given criteria, regardless of whether the traffic is indeed something that should generate an alert. An IDS must be trained to look for suspicious activity. Figure 10.1 details the relationship between IDS systems and the types of responses they can produce.

Figure 10.1. IDS true/false matrix.

IDSHandle with Care

I was lucky to work on most of England's Internet banks. Apart from the general excitement that always surrounded a new ecommerce project, the banks were risk adverse organizations that rarely cut corners on security, which allowed me to delve deep into the areas where I worked.

On one of these assignments, I was asked to review and improve the existing security controls. I had made all the necessary improvements to the firewalls and the routers. The IDS was the last component that needed to be reviewed, and this was not going to take place until the morning of the first day that the bank was scheduled to go live. The system administrators were going to install and configure the IDS a few days before the site launched. The rationale was that the IDS was only a detective control, so the bank could survive it being fully configuredit wasn't like it was a really important detective control. Remember that detective controls don't prevent problems; they only alert when problems occur.

When I arrived at the worksite, it was chaos. Nothing was working, no email, no web accesseverything was at a standstill. The bank only had a limited amount of time to look at the IDS configuration and figure out what was wrong. On inspection of the IDS policy, I had found every box ticked and therefore enabled. This included commands such as, HTTP get, HTTP put, and SMTP HELLO.

This was definitely not good. Every time anyone sent an email or accessed a web page, the IDS would trigger an alarm. Looking at the action setting for each of these events revealed the problem. Each event had every conceivable action set, including the RESET option, which sends a Transmission Control Protocol (TCP) reset to the sending address every time the event fires. So every time a user connected and tried to access the bank's web page, the IDS terminated the session and sent a flood of mail and log messages.

It transpired that the poor administrator had never seen an IDS before and had little in-depth protocol experience. He thought he was making it extra secure by just ticking every box! While explaining the problem to the unfortunate administrator, he repeated the immortal phrase "doesn't it affect only bad packets." Presumably, if you pay extra, you get, "wickedness detection as well!"

There is a moral to this story; when tuning an IDS, know your protocols and understand the attack signatures. This was an easy problem to solve, but it isn't always so easy. It's possible to get one signature wrong and hunt for it for months. Always run the IDS in passive mode until you are confident that you have got it right and are sure that you've got the thresholds right. Only enable positive block actions, whether Shunning, black listing, or just dropping one packet, with logging and alertingthis allows you to diagnose any problems.

This "in the field" segment was contributed by Mark "Fat Bloke" Osborn. He is the developer of WIDZ, the first open source wireless IDS.

 

Pattern Matching and Anomaly Detection

Objective:

Describe anomaly detection IDS

Explain the differences between pattern matching IDS and protocol decoding IDS

Pattern matching, protocol decoding, and anomaly detection are some of the basic characteristics and analysis methods used by IDS systems. Each type takes slightly different approaches to detecting intrusions. A graph showing the relationship of these types and the vendors that use each method is shown in Figure 10.2.

Figure 10.2. IDS types.

Anomaly detection systems require the administrator to make use of profiles of authorized activities or place the IDS into a learning mode so that it can learn what constitutes normal activity. A considerable amount of time needs to be dedicated to make sure that the IDS produces few false negatives. If an attacker can slowly change his activity over time, the IDS might actually be fooled into thinking that the new behavior is actually acceptable. Anomaly detection is good at spotting behavior that greatly differs from normal activity. As an example, if a group of users who only log in during the day suddenly start trying to log in at 3 a.m., the IDS can trigger an alert that something is wrong.

Exam Alert

A false negative is the worst type of event, as it means that an attack occurred but that the IDS failed to detect it.

Somewhere in the middle of the spectrum of intrusion detection is protocol decoding. Protocol decoding IDS systems have the capability to reassemble packets and look at higher layer activity. If the IDS knows the normal activity of the protocol, it can pick out abnormal activity. Protocol decoding intrusion detection requires the IDS to maintain state information. As an example, let's look at the domain name server (DNS) service. DNS is a two-step process. Therefore, a protocol matching IDS can detect that when a number of DNS responses occur without a DNS request, a cache poisoning attack might be happening. To effectively detect these intrusions, an IDS must reimplement a wide variety of application-layer protocols to detect suspicious or invalid behavior.

On the opposite end of the scale, there is pattern matching. Snort is a good example of a pattern matching IDS. Pattern matching IDS systems rely on a database of known attacks. These known attacks are loaded into the system as signatures. As soon as the signatures are loaded into the IDS, the IDS can begin to guard the network. The signatures are usually given a number or name so that the administrator can easily identify an attack when it sets off an alert. Alerts can be triggered for fragmented IP packets, streams of SYN packets (DoS), or malformed Internet Control Message Protocol (ICMP) packets. The alert might be configured to change to the firewall configuration, set off an alarm, or even page the administrator. The biggest disadvantage to the pattern matching system is that the IDS can only trigger on signatures that have been loaded. A new or obfusticated attack might go undetected. Obfusticated attacks are those that are disguised.

Snort

Objective:

Explain Snort

Describe basic Snort rules

Know how to decode Snort Alerts

Snort is a freeware IDS developed by Martin Roesch and Brian Caswell. It's considered a lightweight, network-based IDS that can be set up on a Linux or Windows host. Although the core program has a command-line interface, two popular GUIs can be used. They include SnortSnarf and IDS Center. Snort operates as a network sniffer and logs activity that matches predefined signatures. Signatures can be designed for a wide range of traffic, including Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), and Internet Control Message Protocol (ICMP).

Snort rules are made up of two basic parts:

• Rule header This is where the rules actions are identified.
• Rule options This is where the rules alert messages are identified.

Here is a sample rule:

Alert tcp any any -> any 80
(content: "hacker"; msg: "Hacker Site Accessed";)

The text up to the first parentheses is the Rule Header. The first part is known as the rule action. Alert is the action used in the preceding sample rule; rule actions can include

• Alert
• Log
• Pass
• Activate
• Dynamic

The next item is the protocol. In the example, TCP was used. After the protocol is the source address and mask. Although the example uses any any, it could have been a specific network such as 10.10.0.0/16. This is followed by the target IP address and mask. The final entry of the rule header designates the port. This example specifies 80.

The section enclosed inside the parentheses are the rule options. Rule options are not required but are usually the reason for creating the rule. The rule options are as follows (content: "hacker"; msg: "Hacker Site Accessed";). The first portion specifies the action, which is to examine port 80 traffic for the word "hacker". If a match occurs, a message should be generated that reads, "Hacker Site Accessed," and the IDS would create a record that a hacker site might have been accessed. The rule option is where Snort has a lot of flexibility. Table 10.1 lists some common keywords Snort can use.

Table 10.1. Snort Keywords

Keyword	Detail
content	Used to match a defined payload value
ack	Used to match TCP ack settings
flags	Used to match TCP flags
id	Matches IP header fragment
ttl	Used to match the IP header TTL
msg	Prints a message

Although the CEH exam will not expect you to be a Snort expert, it is a good idea to have a basic understanding of how it works and to understand basic rules. A few of these are shown in Table 10.2.

Table 10.2. Basic Snort Rules

Rule	Description
Alert tcp any any -> 192,168.13.0/24 (msg: "O/S Fingerprint detected"; flags: S12;)	OS fingerprint
Alert tcp any any -> 192,168.13.0/24 (msg: "NULL scan detected"; flags: 0;)	Null scan
Alert tcp any any -> 192,168.13.0/24 (msg: "SYN-FIN scan detected"; flags: SF;)	SYN/FIN scan
Alert udp any any -> any 69 (msg "TFTP Connection Attempt)";)	Trivial File Transfer Protocol attempt
[View full width] Alert tcp any any -> 192,168.13.0/24 (content: "Password"; msg: "Password Transfer Possible!";)	Password transfer

Although these are good examples of basic Snort rules, they can be much more complex. Following is an example of one developed to alert upon detection of the Microsoft Blaster worm:

[View full width]

alert tcp $EXTERNAL_NET any -> $HOME_NET 135 (msg:"NETBIOS DCERPCISystemActivator bind attempt"; low:to_server,established; content:"|05|";distance:0; within:1; content:"|0b|"; distance:1; within:1;byte_test:1,&,1,0,relative; content:"|A0 01 00 00 00 00 00 00 C0 00 00 00 00 00 00 46|"; distance:29; within:16; reference:cve,CAN-2003-0352;classtype :attempted-admin; sid:2192; rev:1;)

Building Snort rules is only half the work. After a Snort alert occurs, it is important to be able to analyze the signature output. To really be able to determine what attackers are doing and how they are doing it, it is important to be able to perform signature analysis. The goal of the signature is to be able to identify malicious activity and be able to track down the offender. This activity can be categorized as

• Scans and enumeration
• Denial of service (DoS) attacks
• Exploits

If you have never used an IDS, you might be surprised at the number of alerts it produces in just a few hours after you connect to the Internet. Shown in Figure 10.3 is the signature of an Nmap ACK scan.

Figure 10.3. Nmap ACK scan log.

As you can see, the attacker is located at 192.168.13.236 and is scanning 192.168.13.235. On the third line of each scan, you should see the ***A***, which indicates the ACK scan. The other telltale sign is repeating sequence and acknowledgement numbers. That is not the normal behavior for TCP.

If this section has raised your interest in getting to know more about Snort, there are a host of books that can help you through the process. One good book by Syngress is Snort 2.1 Intrusion Detection, by Jay Beale. Now let's look at some of the ways that hackers attempt to bypass IDS and prevent it from detecting their activities.

IDS Evasion

Objective:

Identify IDS evasion techniques and tools

Attackers can use a range of techniques to attempt to prevent detection. These include techniques such as

• Flooding
• Evasion
• Session splicing

One of the most basic is to attempt to overload the IDS by flooding. Flooding is nothing more than attempting to overload the IDS by flooding it with traffic. The attacker might even insert a number of low priority IDS triggers to attempt to keep it busy while a few more damaging attacks slip by. Generating such a huge amount of traffic forces the administrator to sort through all the data and somehow try to make sense of it all. The real target and intent of the attacker might be totally lost within the blizzard of messages, beeps, and alerts generated.

Evasion is another effective technique. It occurs when an IDS discards the packet that is accepted by the host it is addressed to. As an example, TCP segments marked with a SYN flag might also include data. Because this is an infrequent occurrence, an IDS might ignore the contents of these packets, thereby allowing the packets to go undetected.

Session splicing works by delivering the payload over multiple packets, which defeats simple pattern matching without session reconstruction. This payload can be delivered in many different manners and even spread out over a long period of time. It is really a form of fragmentation. By breaking up the payload over many different packets, many IDS systems will fail to detect its true purpose. IP fragments typically arrive in the order sent, but they don't have to. By sending the packets out of order and playing with fragment IDs, reassembly can become much more complicated. If the IDS cannot keep all fragments in memory for reassembling, an attacker could slip by.

IDS Evasion Tools

Several tools are available that can be used to evade IDS systems. Most of these tools exploit one or more of the techniques discussed in the previous section. Some of the better known tools are discussed in the following:

• Stick Uses the straightforward technique of firing numerous attacks to purposely trigger IDS events. Although the IDS system attempts to keep up with the new flood of events, it could eventually become flooded and a DoS of the IDS might result.
• ADMutate Borrows ideas from virus writers to create a polymorphic buffer-overflow engine. An attacker feeds ADMutate a buffer-overflow exploit to generate hundreds or thousands of functionally equivalent exploits, but each has a slightly different signature.
• Mendax Builds an arbitrary exploit from an input text file and develops a number of evasion techniques from the input. The restructured exploit is then sent to the target.
• NIDSbench Includes fragrouter, tcpreplay, and idstest. Fragrouter fragments traffic, which might prevent the IDS from detecting its true content.
• Nessus Can also be used to test IDS systems and has the capability to perform session splicing attacks.

IDS systems are not perfect and cannot be expected to catch all attacks. Even when sensors are in the right location to detect attacks, a variety of tools and techniques are available to avoid detection. For IDS systems to be effective, the individuals responsible for them must continually monitor and investigate network activity to stay on top of changes in hacking tools and techniques.

Challenge

As you have seen, intrusion detection is an important tool. For IDS to be useful, you will need to know current attack information so that you can properly tune the IDS. One good site to obtain IDS information from is www.DShield.org. DShield is a site that gathers log files from users around the world. This challenge exercise will explore DShield.

1. Open Internet Explorer or the browser you use on your Windows computer.
2. Enter the URL for the DShield site: www.dshield.org/reports.php
3. Click on the link to the right side of the screen for Top 10 Reports.
4. After you're on the Top 10 page, review the most common ports probed. At the time of writing, the number one port is 137.
5. Now, determine if your computer has connections to open on port 137. One quick way to do so is by using netstat. To start netstat, open a command prompt and enter netstat a; then press Enter. A list of open connections will appear.
6. Observe how many connects to port 137 are present. Are these computers in your own domain? Remember that port 137 is used by NetBIOS and can represent a security threat if accessible by attackers. If connections to your computer are present by computers outside your domain, write down the IP address of these connections.
7. With these IP addresses recorded, proceed to www.dshield.org/ipinfo.php/. This is the location of DShields' IP address database. Enter the IP addresses you discovered in step 6. Where are the remote computers located?
8. If you would like to participate in the DShield process, you can download a client that will allow your firewall alerts to be added to the DShield database. The client is available at www.dshield.org/windows_clients.php. It works with most firewalls.

Firewalls