What Is IPS?

Simply stated, intrusion prevention technology adds an active layer of defensive technology. Intrusion detection technology generally only reports attacks against monitored systems, although since 1997 active responses have been available, such as forging resets to blow away TCP connections (the so-called session sniping or reset kill). Intrusion prevention technology, by contrast, will attempt to stop the attacks before they are successful.

As IPS continues to mature, it will probably evolve to be something more like a capability than a single product. For instance, consider the logic behind Cisco Secure IDS, CiscoWorks Management Center for firewalls and VPNs, and Cisco Security Agent (CSA). A suspicious packet enters the network and is perhaps detected with the Cisco Secure IDS technology. The attack is classified by the operating systems vulnerable to that attack. The CiscoWorks Security Information Management (SIM) console is consulted to see if the attack and the destination are a match. For instance, if the attack is a Solaris exploit and the target system is a Sun Microsystems workstation running that version of Solaris, it would constitute a match.

To double-check, an agent on the target can be consulted in case the console is out of date. If the attack does not match the target (for example, an SGI attack against a Windows XP box), the alert is deprecated.

However, if there is a match, the agent can be consulted to determine the potential for an active vulnerability. If the box is patched and not vulnerable, again, we can deprecate, or reduce the priority of the alert. If the box is vulnerable, we can test to see if a compromise has occurred. If the file system has changed, we can begin forensics and incident response.

To be sure, this doesn't sound like intrusion prevention; it is more like modern SIM-enabled intrusion detection. This capabilitymodern intrusion detection with strong multiproduct coordination featuresis exactly what well-funded organizations across the globe are trying to implement using Intellitactics, netForensics, Huntsman, ArcSight, and other database-driven consoles. However, next we add the defensive layers based on the former Okena StormWatch product purchased by Cisco and renamed Cisco Security Agent (CSA), a host intrusion prevention system (HIPS). CSA can stop the attack in the network, just like a personal firewall does. However, what if the user is surfing the web using Internet Explorer and malicious code compromises the system via an Internet Explorer vulnerability? CSA also has an operating system shim so that if the malicious code activates in the file system and misbehaves, CSA can detect and stop the application. Wrong behavior, as we discussed in Chapter 10, "Host Defense Components," could range from trying to initiate a network connection to making a call for interrupt 13H to make a direct write to the hard disk. You can expect to see similar console and agent capabilities from Symantec and possibly Guidance Software. Encase, by Guidance, is a fully capable incident-handling tool already; it gives you the ability to detect signatures of malicious activity and to quarantine a file.

Because both host-based and network-based intrusion prevention systems are active technologies in the sense they directly interact with packets and are extremely capable of causing denial of service conditions, we can easily deduce the requirements for a successful product:

• It has to be fast.

• It has to keep state.

• It has to have some knowledge of application protocol or application behavior.

• It has to be accurate and up to date.

• It has to nullify an attack.

Let's take a look at these requirements in greater detail.

An IPS Must Be Fast

Perhaps you remember the early days of routers and a company named Cisco that offered a router that operated at "wire speed." Once they had wire speed, nothing else would do. The same is true for a NIPS. If it starts adding latency, it becomes a denial of service deviceand companies do not intentionally purchase denial of service devices. This likely means doing as much processing in hardware as possible and even better processing in parallel, which is going to add to the cost of the device. Cheaper, single-threaded NIPS may be able to perform much of their processing in kernel space to gain speed. Does a HIPS have to be fast? Certainly. Who would tolerate a security layer that slowed down his or her machine?

An IPS Must Keep State

The state of a communication flow affects the correct interpretation of a packet. This is fairly simple at the IP/TCP/UDP header level, but the IPS must be able to create state tables so that sufficient state is available to interpret packets. Perhaps you will recall that the original stateful firewallsCheck Point FireWall-1 and Cisco PIXhad gobs and gobs of problems with state. It is a lot harder than it sounds because various TCP stacks do not always perfectly implement RFCs, and state is not always easy to predict and requires lots of memory and processing.

Perhaps you have heard the marketing term Deep Packet Inspection. If such a thing exists, it stands to reason there must also be "Shallow Packet Inspection"and there is. Shallow Packet Inspection is a field-by-field analysis of the IP/TCP/UDP header. Because all these fields are fixed length, we can do this very fast, and anomalous values can be detected with a high degree of accuracy. The craft of Shallow Packet Inspection was first created by Stephen Northcutt with the DoD Shadow team and brought to maturity by Judy Novak at the Army Research Laboratory. It's now incorporated by most network analysis tools. Deep Packet Inspection requires understanding of the protocol itself and dealing with fields that may not be fixed-length. The earliest significant work was the BRO freeware intrusion detection system by Vern Paxton. Although there are application-specific IPSs especially for web servers, as a general rule of thumb you can expect it will be several years until an IPS that can actually monitor a large number of protocols becomes available.

You can see the reason why we were careful to point out the hype in the intrusion prevention space. This is very hard stuff to do, and there are significant limitations that affect the performance possible at a given price point.

An IPS Must Be Accurate and Up to Date

Because it is an active device, an IPS runs the risk of creating a "self-inflicted" denial of service condition. Therefore, it must be nearly 100% accurate for the attack signature it takes an active role in defending. This difference may be one reason the pre-Gartner report IDS vendors were a bit sloppy with their signatures; if they were wrong, nothing broke. Clearly keeping state is a large part of that; knowing how the application is supposed to perform is also important. However, the accuracy of the signatures depends on far more than that. The signature language and rule-processing engine must be fully industrial strength. Also, they have to be up to date. The rule of IPS signatures is simple: The cycle time from the moment a vulnerability is announced to develop and distribute a new signature with a prescription must be less than the cycle time to develop a worm to take advantage of the vulnerability.

An IPS Must Have the Ability to Nullify an Attack

We realize this is a stretch, but there actually are products that meet the acid test requiring an IPS to be fast, keep state, know the application protocol or behavior, be accurate, and be up to date. Now we come to the core issue: Can an IPS reliably stop attacks? There are two basic methods: eliminate only the very awful packets and kill the stream. One approach, as you will learn later, is a "bump in the wire" approach to network-based intrusion prevention. For a NIPS to nullify an attack, all the packets to a segment of the network must pass through the network deviceeither a switch or a chokepoint next-generation firewall-type device. IPS must be fast, stateful, and hopefully accurate and up to date.

Ideally, IPS see the attack set up and drop only the payload of the buffer overflow or shell code. This approach has the minimum possible impact on network traffic yet keeps you safe. In contrast, there are NIPS and HIPS technologies that look for signatures of problems and, at the first sign of trouble, terminate (or refuse to pass) the communication stream. The classic example of this approach is the UNIX Sentry Tools (http://sourceforge.net/projects/sentrytools). PortSentry, one of the most employed HIPS ever, detects an attack, can disallow the attacking IP from making further connections, and can null-route the path back to the attacker. Both approaches are fine, but the bottom line is that to be truly considered an IPS, it must implement a far more robust active mechanism than a TCP reset kill.

Note

We mentioned PortSentry in Chapter 10 in the context of an IDS with active response capabilities and that one could classify it as an IPS as well.