Intrusion Detection

Intrusion detection has its roots in the financial audits of mainframe computers. The scarcity and expense of mainframes meant that access to their computing power had to be tightly controlled (and users had to be accurately billed). In the late 1970s, financial audits were adapted for security purposes, enabling administrators to review logs for anomalies that might indicate misuse, such as unauthorized file changes.

Further developments led to real-time detection capabilities in the mid-1980s, and to network monitoring in 1990. Today, network managers can choose from a variety of solutions and vendors , but the general principles of an Intrusion Detection System (IDS) remain the same.

The idea behind an IDS is simple: an agent monitors file activity on a host or traffic on a network, and reports strange behavior to an administrator (see Figure 1).

Figure 1: Electronic Eyes. An Intrusion Detection System (IDS) monitors network traffic or file activity on a host for attacks, anomalous behavior, and misuse. An IDS logs intrusions, sends real-time alerts, and in some situations can halt the attack.

The IDS market is divided into two primary groups: host-based and network-based systems. This tutorial examines the basics of host- and network-based detection systems, discusses how to implement an IDS, and outlines a proper incident-response plan.

Host-Based IDS

Host-based IDSs add a targeted layer of security to particularly vulnerable or essential systems. An agent sits on an individual system-for example, a database server-and monitors audit trails and system logs for anomalous behavior, such as repeated login attempts or changes to file permissions. The agent may also employ a checksum at regular intervals to look for changes to system files. In some cases, an agent can halt an "attack" on a system, though a host agent's primary function is to log events and send alerts.

The primary benefit of a host-based system is that it can detect both external and internal misuse, something that network monitors and firewalls can't do. The appeal of such a tool is obvious, as security breaches are more likely to come from an internal user than from a hacker outside the network. Host agents are powerful tools for addressing the authorization and access issues that make internal security so complex.

Agents install directly on the host to be monitored , so they must be compatible with the host's OS. Memory requirements and CPU utilization will vary from vendor to vendor, so be sure to learn ahead of time the demands the agent will place on the system.

Network-Based IDS

A network-based IDS sits on the LAN (or a LAN segment) and monitors network traffic packet by packet in real time (or as near to real time as possible), to see if that traffic conforms to predetermined attack signatures. Attack signatures are activities that match known attack patterns. For example, the TearDrop Denial of Service (DoS) attack sends packets that are fragmented in such a way as to crash the target system. The network monitor will recognize packets that conform to the TearDrop signature and take action.

The IDS vendor provides a database of attack signatures, and administrators can also add customized signatures. If the IDS recognizes an attack, it alerts an administrator. In some cases, the IDS can also respond, for example by terminating a connection. In addition to its monitoring and alarm functions, the IDS also records attack sessions for later analysis. Network IDSs can also be linked to other security features, such as firewalls, to make sure those systems haven't been breached.

A network monitor has two main benefits. The first is the real-time nature of the alarm, which can give administrators an opportunity to halt or contain an attack before it does significant harm. This is especially valuable for DoS attacks, which must be dealt with immediately to mitigate damages.

The second benefit is evidence collection. Not only can administrators analyze the attack to determine what damage might have been done, the attack session itself can point out security flaws that need addressing. (This is also true for host-based systems). Because many hackers first scan a target network for known vulnerabilities, a hacker's choice of attack may indicate that such vulnerabilities exist on your network. A simple example is an operating system that has yet to be secured with the latest vendor patch.

Network monitors are OS-independent. Basic requirements include a dedicated node that sits on the segment to be monitored and a NIC set to promiscuous mode. You may also want to set up a secure communications link between the monitor and its management console.

Establishing An IDS

The first step in establishing an IDS is to incorporate it into your security policy. (If you don't have a security policy, now would be a good time to develop one. See "Create Order with a Strong Security Policy," Network Magazine July 2000, page 62 for more information.) In brief, a security policy defines the basic architecture of the network, describes how the network will be secured, and establishes a hierarchy of user access to data resources.

When incorporating an IDS into your security policy, you should define how the IDS will fit into the overall security architecture, outline procedures for maintaining and responding to the IDS, and assign resources (software, hardware, and humans to manage the technology).

You'll also have to choose a network- or host-based system, or a combination of both. A combination provides the most comprehensive security; however, this decision will be colored by the level of security you require, the budget at your disposal, and the in-house resources on hand to manage the system.

Generally speaking, network monitors cost significantly more than host-based agents. However, depending on the size of your network, a single monitor can offer substantial network coverage. Conversely, host-based agents cost less, but are limited to a single host.

Other factors play in deciding to implement either or both solutions. For example, network monitors may have difficulty with encrypted traffic. A network monitor functions by reading packet headers and data payloads. If this information is encrypted, the IDS can't detect attacks. Encryption doesn't hinder host agents because the data is decrypted before a host agent sees it.

Network sensors can also become a bottleneck on high-speed LANs, degrading performance and frustrating users. According to an ICSA paper, a network-based IDS can handle up to 65Mbits/sec of traffic before the analysis engine's performance drops (see Resources).

Care And Feeding

Regardless of which solution you implement, you must be prepared to properly maintain the system. IDSs can generate reams of data that have to be reviewed regularly. Most products come with reporting software to help with this task.

Depending on the number of monitors and agents you employ, be prepared to dedicate substantial storage to IDS log files. You also must secure audit logs so that intruders can't tamper with them to erase or obscure evidence of the intrusion.

Like anti-virus products, an IDS's attack signature database must be updated regularly. Vendors will provide new attack signatures, but be sure to query them on the frequency of updates, especially in response to newly discovered attacks. Be aware that slight variations to a known attack may be enough to slip past even an IDS with the most current signatures.

You can also be proactive by monitoring security sites for new attack signatures and exploits.

Dealing With Incidents

Once you install your IDS, be prepared for the possibility that it will work! That is, have a plan in place for dealing with intrusions once you detect them.

The first step is to create an incident-handling team to respond to intrusions. The size and capabilities of your team will vary with the size of your organization, but each member of the team should have clearly-defined roles and responsibilities (for example, a Windows NT specialist, a Unix specialist, and so on).

You'll also want to create an incident-handling policy that outlines the response procedures and lists contact information for team members . Procedures include backing up an affected hard drive and determining whether it is necessary to enlist outside expertise or contact law enforcement agencies.

The decision to involve the law is a complicated one, so it's best to have policies in place beforehand. It may not be worth your time to call the cops on a script kiddy who Pings your network.

Is An IDS Right For You?

Intrusion detection is still a maturing technology, and not everyone in the security community is convinced of its viability. Some observers have compared intrusion detection to the so-called Star Wars missile defense program-that is, expensive and ineffective .

Of course, "expensive" is a relative term : While an IDS doesn't run cheap, the cost of a network outage from a DoS attack (and the attendant bad press, dissatisfied customers and business partners , and furious executives) can easily justify the vendor's price tag.

As for effectiveness, an IDS is not a "set it and forget it" proposition. Security policies must be in place, attack signature databases must be updated, and logs have to be reviewed regularly to gain the full benefits. If you can meet those requirements, intrusion detection is a valuable tool for protecting your data resources.

Resources

The ICSA's Intrusion Detection Systems Consortium has an informative white paper on Intrusion Detection Systems (IDSs). You can download the paper at www.icsa.net/html/communities/ids/White%20paper/index.shtml.

Intrusion Detection by Rebecca Gurley Brace (Macmillan Technical Publishing, 2000) is an informative and comprehensive guide to the concepts and principles of IDSs.

The IDS vendor Network Security Wizards has a good collection of papers and articles at www.securitywizards.com/library.html. Network managers may be particularly interested in the rebuttal to "50 Ways to Defeat Your Intrusion Detection System." CERT maintains an IDS checklist at www.cert.org/tech_tips/intruder_detection_checklist.html.

Network Magazine has published several useful articles on IDSs. Check out "Deploying an Effective Intrusion Detection System" (September 2000, page 60), "Security Reality Check" (July 1999, page 80) and "Can Intrusion Detection Keep an Eye on Your Network's Security?" (April 1999, page 36). In addition, the article "Incident Handling" (January 2000, page 80) provides a good overview of how to build an incident-handling team. Archived article can be found at www.networkmagazine.com.

This tutorial, number 149, by Andrew Conry-Murray, was originally published in the December 2000 issue of Network Magazine.