IDS Theory

In this section we introduce a practical mathematical model for evaluating and deploying IDSs in your network. This section is based on methods from statistics, which we have adapted to the information security realm.

IDS Limitations

Because of the nature of IDSs, they will always be at a disadvantage . Hackers can always engineer new exploits that are not yet detected by existing signature databases. In addition, as with virus scanners , keeping signatures up to date is a major problem. Furthermore, network IDSs are expected to cope with massive bandwidth. Maintaining state in a high-traffic network becomes prohibitive in terms of memory and processing cost.

Moreover, monitoring "switched networks" is problematic because switches curtail the IDS's sensors. There have been attempts to compensate for this by embedding the IDS in the switch, or by attaching the IDS to the switch monitor port. However, such solutions have so far proven mostly ineffective .

Another limitation of IDSs is that they are extremely vulnerable to attack or evasion. For example, denial-of-service (DoS) attacks such as SYN floods or smurf attacks can often take down an IDS with ease. A SYN flood exploits the standard TCP connection establishment sequence when the malicious sender forges the source address in the packets being directed at the IDS. The IDS then begins to consume resources waiting for the nonexistent host to respond to the IDS synchronization packets. Similarly, slow scans or IP address spoofing will frustrate many IDSs.

Later in this chapter, we will discuss ways to hack through IDSs. However, before completely discouraging you from using them, we will first provide some mathematical models that show you how IDSs can help protect your network. The following section will introduce statistical methods for evaluating the effectiveness of IDSs. Based on your statistical evaluations, you will then be able to intelligently implement different flavors of IDSs at different points in your network.

Sensitivity Versus Specificity

This section discusses the properties of diagnostic software, and their implications for interpreting test results. By understanding these concepts and how they apply to IDSs, you can make better judgments about how to deploy and interpret IDSs in your system.

Consider a typical IDS report monitor as represented by the 2x2 table in Figure 14.1. One axis called Intrusion represents whether an intrusion has really occurred . For example, on this axis, the "+" means there really was an intrusion, while the "-" means there was no intrusion.

Figure 14.1. Sensitivity versus specificity.

TP = True Positive = "Intrusion Correctly Detected"

FP = False Positive = "False Alarm"

FN = False Negative = "Intrusion Missed"

TN = True Negative = "Integrity Correctly Detected"

The other axis is called IDS Response and represents whether or not the IDS thinks it has detected an intrusion. For example, on this axis, the "+" means the IDS thinks there was an intrusion, while the "-" means the IDS thinks there was no intrusion. As in the real world, this model shows that the IDS is not always correct. We can use the incidence of each quadrant of the 2x2 table to help us understand the statistical properties of an IDS.

Sensitivity

Sensitivity is defined as the true positive rate (for example, the fraction of intrusions that are detected by the IDS). Mathematically, sensitivity is expressed as follows :

 True Positives / (True Positives + False Negatives) 

The false negative rate is equal to 1 minus the sensitivity. The more sensitive an IDS is, the less likely it is to miss actual intrusions.

Sensitive IDSs are useful for identifying attacks on areas of the network that are easy to fix or should never be missed. Sensitive tests are more useful for "screening"; that is, when you need to rule out anything that might even remotely represent an intrusion. Among sensitive IDSs, negative results have more inherent value than positive results do.

For example, you would need a sensitive IDS to monitor host machines sitting deep in the corporate LAN, shielded by firewalls and routers. In Figure 14.2, this is represented by Area 2. At this heavily buffered point in the network, you should not have any intrusions whatsoever. Thus, it would be important to have a high sensitivity to screen for anything amiss. As you will see later, specificity is less important here, because at this point in the network all anomalous behavior should be investigated. The IDS does not need to discriminate, because a human operator is obligated to investigate each alarm by hand.

Figure 14.2. Sample network.

Specificity

Mathematically, specificity is expressed as follows:

 True Negatives / (True Negatives + False Positives) 

True negatives represent an IDS that is correctly reporting that there are no intrusions. False positives occur when an IDS mistakenly reports an intrusion when there actually is none. The false positive rate is equal to 1 minus the specificity.

Specific IDSs have the greatest utility to the network administrator. For these programs, positive results are more useful than negative results. Specific tests are useful when consequences for false positive results are serious.

You would choose an IDS with a high specificity for an area of the network where automatic diagnosis is critical. For example, in Figure 14.2, Area 1 represents the corporate firewall that faces the Internet. In this case, you would need an IDS that has a high specificity to detect DoS attacks, because they can be fatal if not detected early. At this point in the network, you care less about overall sensitivity, because you are " ruling in" an attack, rather than screening the mass of normal Internet traffic for any anomaly.

Accuracy

Often, a trade-off occurs between sensitivity and specificity that varies on a continuum dependent on an arbitrary cutoff point. A cutoff for abnormality can be chosen liberally or conservatively.

However, there are situations when you need to spend the extra money to achieve both a high sensitivity and a high specificity. Accuracy is a term that encompasses both specificity and sensitivity. Accuracy is the proportion of all IDS results (positive and negative) that are correct.

For example, you might need a high-accuracy IDS in an area of the network such as Area 3 in Figure 14.2. In this case, your Web server is under constant attack, and it would also cause the most immediate embarrassment and financial loss if compromised. In this case, you need to process any slight anomaly, and you need to do it automatically because of the high traffic volume. In fact, to achieve the highest sensitivity and specificity here, you might need to combine layers of different IDSs.