10.5 Intrusion Detection Systems

IDSs are increasingly popular additions to network security. An IDS is used to search for patterns that may indicate an attack on a network. Unlike firewalls, which are designed to block suspect information, an IDS only issues a warning.

Generally, an IDS is placed at the edge of the network (Figure 10.7), so it can monitor all traffic in and out of the WAN. This is known as a network IDS (NIDS). An IDS installed on a server that is used to monitor connections to that server only is known as a host-based IDS. Most networks use a combination of host-based IDS and NIDS, because a single NIDS on the edge of the network may have trouble processing all of the incoming traffic. The NIDS is used to get the state of the network in a big picture fashion, while the host-based IDS are used on critical servers to watch for potential problems on those servers.

Figure 10.7. A typical NIDS design. The public side of the NIDS does not have an IP address and is plugged into the same switch as the firewall. The switch is configured for port-mirroring; all alerts gathered by the NIDS are sent to a device on the private network.

How does an NIDS work? The NIDS is placed between the routers and firewalls, usually plugged into a Layer 2 switch, or even a hub. If a switch is used, the port mirroring needs to be set up on the switch to allow traffic from the monitored ports, or VLANs, to be mirrored on the port the NIDS is plugged into. On a Cisco 2900 switch port mirroring can be accomplished in a manner similar to this:

 Core1(config)#int fa0/24  Core1(config-if)#port monitor fastEthernet 0/1  Core1(config-if)#port monitor fastEthernet 0/2  Core1(config-if)#port monitor fastEthernet 0/3  Core1(config-if)#port monitor fastEthernet 0/4 

This will allow Fast Ethernet Port 24 to monitor all traffic coming through Ports 1, 2, 3, and 4. Different vendors and even different product lines within the same vendor have different ways of configuring port mirroring. Information about how to configure port mirroring should be included on the vendor website, or in its documentation.

NOTE

While hubs can be used, and some people prefer them because there are no configuration requirements ”they are just plug and play ”hubs do not have the processing power that switches do. It is possible to flood a hub on an enterprise network and slow down traffic.

Port mirroring is how network intrusion detection is accomplished from a design perspective, but the more important question is, how do NIDS process data. An NIDS is, basically, a packet sniffer. It sorts through the traffic on the network looking for patterns, which may be representative of an attack. These patterns are called signatures. A signature is a set of events indicative of a network attack ”the events may not actually be an attack, but there is a preponderance of evidence to suggest it is. The way signatures are matched against network traffic depends on the type of NIDS in use.

Based on how the problem of detecting attacks is approached, there are two types of NIDS systems: signature and anomaly. These types of NIDS can be broken into smaller categories, but the NIDS in each category all follow the same architecture. Signature-based NIDSs rely on internal databases with common attack patterns. If one of these patterns is matched, a flag is set off alerting administrators. Anomaly-based NIDSs rely on changes in traffic patterns to determine whether an attack is occurring.

NOTE

To increase the level of network security, most administrators do not assign an IP address to the port on the NIDS that is doing the sniffing. That interface is often outside the firewall, so there is nothing to protect it ”if there is no IP address it is much more difficult for an attacker to detect and compromise the NIDS.

10.5.1 Signature-Based NIDS

Most commercial NIDSs are signature-based. It requires a great deal of effort to keep databases current with the latest security information, so the attack patterns can be included in the database. While this is not outside the realm of possibility for an open -source project, to date it has not been done. There are three types of signature-based NIDSs: pattern matching, stateful pattern matching, and protocol decode.

Pattern matching is the simplest of NIDSs; it is a basic pattern-matching system. The database consists of a series of rules, and the NIDS inspects each packet looking for a match. When a match, or series of matches, is found they will be reported .

Pattern-matching rules are very simple; they usually consist of the protocol, port, and the actual pattern. Some pattern-matching NIDSs also allow administrators to include things like source and destination IP address. The following is an example of a typical signature, prepared for the SNORT database:

 alert UDP $EXTERNAL any -> $INTERNAL 53 (msg:  "IDS489/dns_named-exploit-tsig-lsd"; content: "3F 909090 EB3B 31DB  5F 83EF7C 8D7710 897704 8D4F20"; classtype: system-attempt;  reference: arachnids,489;) 

This is based on the BIND transaction signature vulnerability (CVE CAN-2001-0010) in BIND Version 8.0. An attacker exploiting this vulnerability would be able to gain root access to the DNS server. If a match to this pattern were found, the NIDS would alert administrators to a potential problem.

Administrators have to be careful with simple pattern-matching NIDSs. If a pattern is too general, the NIDS will generate a lot of false positives from normal traffic. If a pattern is too specific, it may not catch all attempts.

The problem with pattern-matching NIDSs is that they are too rigid. If a packet does not match the information in the database exactly, it will not be reported. This can be problematic , especially when attackers can change small portions of an attack. For example, if an attack is based on buffer overflow vulnerabilities in PHP, an attacker may construct an attack that makes an HTTP request:

 POST http://www.example.com/script.php?efffff7d2b9bfff 

Obviously, that is not a fully formed HTTP request, but it will do for example purposes. In fact, there is a security hole in PHP, with a pattern that looks a lot like this:

 alert TCP $EXTERNAL any -> $INTERNAL 80 (msg:  "IDS431/web-cgi_http-php_strings_exploit-atstake"; flags: A+;  content: "ba49feffff f7d2 b9bfffffff f7d1";) 

If a signature match based on the pattern in the example is added to the NIDS database, there is nothing stopping the attacker from changing the attack to:

 POST http://www.example.com/script.php?efffff7d2b9bfff16932 

Especially in the case of HTTP-based attacks, it is relatively simple to adjust the attack to avoid matching an entry in the NIDS database. So, while pattern-matching NIDSs are great because of their simplicity and the relatively low number of incidences of false positives, they are too easy for a savvy attacker to fool. Simple pattern matching can also be problematic when dealing with streaming protocols, such as HTTP. Because simple pattern matching inspects each packet individually, it does not take into account attacks that span across several packets, so these attacks may be missed.

Stateful pattern-matching NIDSs have a different approach to signature tracking. These NIDSs search for the same information, but they track session streams, not just individual packets. By maintaining session information, stateful pattern-matching NIDSs can track much more complex attacks, and return fewer false positives and fewer false negatives .

To understand the benefits of stateful pattern matching, look at the PHP exploit in the previous example. Suppose an attacker were able to separate the query into two packets; the first packet contains ba49feffff f7d2, while the second has b9bfffffff f7d1. A simple pattern-matching NIDS will return a false negative, because the data is in two packets, neither of which matches the database entry. A stateful pattern-matching NIDS will spot the match and report it, even across multiple packets.

The benefits to this type of pattern matching are obvious and, if managed properly, a stateful pattern-matching NIDS will report fewer false negatives than a simple pattern-matching NIDS. While this type of pattern matching is an improvement, it is still susceptible to alterations in the attacks. If the details of an attack are altered , it is no longer a match for the information in the database, and it can evade detection.

10.5.2 Anomaly-Based NIDS

An anomaly-based NIDS is an IDS that baselines normal traffic patterns and generates alerts when that pattern alters significantly. Rather than monitor for specific types of traffic, the NIDS looks at changes in the behavior of the network.

Anomaly-based NIDS offer the advantage of not relying on existing signatures, so they are less susceptible to new attacks than signature-based NIDSs. If a new attack is launched on the network, the NIDS will detect the change in traffic patterns ” assuming the attack is different enough to generate an alarm ” and alert administrators to the problem.

Anomaly-based NIDSs work best when used in conjunction with a signature-based NIDS. The signature-based NIDSs should catch known attacks, while the anomaly-based NIDS will report suspicious activity.

One of the best-known anomaly-based NIDSs is the Statistical Packet Anomaly Detection Engine (SPADE), a plug-in for SNORT that was developed by Silicon Defense (www.silicondefense.com/). SPADE is a preprocessor for SNORT; it monitors the network for anomalous packets. If a suspect packet is discovered SPADE logs the packet, then generates an alert using the standard SNORT mechanisms.

Anomaly-based NIDSs require a good deal more time to set up than signature-based NIDSs. It takes a while to examine the traffic patterns of a network and determine when to generate alarms. It can often take several weeks, or even months, to set the triggers in an anomaly-based NIDS so that it is not generating alarms to frequently, or infrequently.