Factors to Consider when Setting up an Intrusion Detection System

---

An intrusion detection system is a critical component of a complex information security system. Correctly selecting the positions for its components seriously influences how efficiently it operates. Suppose that you have chosen where you want to detect attacks (for example, let us say it is the demilitarized zone). What do you need to know in order to be able to place sensors efficiently? It is not enough to simply understand which traffic will be controlled and which will pass through. Besides this, it is necessary to answer several questions, which I'd like to cover now. Generally, since an intrusion detection system (especially its network component) relates to network equipment, it is necessary to consider various aspects of the network when choosing the places for its sensors. Considering the fact that planning locations for security tools must be kept in mind at the stage of planning the network layout, this should not cause any serious dilemmas.

Network Workload and Traffic Intensity

These parameters are rather important. Without knowing them, you would be unable to tell for sure whether or not a sensor installed in a controlled segment is able to process the controlled traffic. If a threshold value is exceeded, the sensor will not be able to perform its tasks. It will either start overlooking attacks (false negative) or simply fail to operate (these aspects will be covered in more detail in Chapter 12). You are lucky if the sensor notifies you during a false positive event (for example, RealSecure Network Sensor provides such a mechanism) and you will be able to understand that some steps must be taken. But what if you are not so lucky?

In order to forecast and predict situations that might cause the IDS to fail, one should know the network workload's average and peak values. For example, the sensor might handle an average workload of 40 Mbit/sec, but at peak workload of 90 Mbit/sec it might start to pass suspicious packets. The frequency of peak workloads depends on various parameters, including the traffic type.

Types of Traffic in the Controlled Segment

The traffic in a controlled segment can be classified by the following two types: flow traffic and pulsating traffic. Applications that generate flow traffic are characterized by highly predictable workload peaks. Traffic such as IP telephony or multimedia (audio and video) data transmission, is ideal for a network intrusion detection system, since data are transmitted at a predictable speed, thus reducing the relation of peak-to-average workload to practically zero. Pulsating traffic (e.g. FTP protocol, e-mail transmission, Web browsing, etc.) is characterized by unpredictable peaks of the network workload - from zero to peak values. Unfortunately, most attacks are implemented in segments with pulsating traffic. This is the main reason why users should test intrusion detection systems at average and peak network workloads.

Average and Peak Processor Workloads

In contrast to network sensors, one should consider the processor workload's average and peak values for host-level sensors. It might even be impossible to install system sensors on servers running applications that heavily load the CPU. I would also advise you to thoroughly assess the processor workload when purchasing and testing the IDS. This will prove to be useful not only when planning sensor positions within the corporate network, but also in thwarting internal attacks. The collected statistics will help you to prove to the specialists from the IT department that performance problems that they are experiencing are not related to the intrusion detection system.

Average Packet Size in a Protected Segment

Let us suppose that you believe the manufacturer's claims that their intrusion detection system can process 100 Mbit traffic without problems. You install this system in your network and use it to protect the IP telephony segment or DNS server managing the whole corporate network (more than 1000 addresses). And what happens? The system can not even handle half of the planned workload. Why did this happen and who is responsible? The answer lies in the fact that the size of the DNS query and DNS response packets is usually not larger than 50 and 90 bytes, respectively (neglecting the sizes of UDP and IP headers). Thus, you have not solved the problem, but the manufacturer did not deceive you. The problem is that the manufacturer measured IDS performance under laboratory conditions, and in your environment, the average packet length proved to be significantly smaller. However, some manufacturers prefer to pull an even simpler trick - they measure performance at the maximum possible values of packet length rather than at the average values (Table 10.2). Knowing the average packet size allows you to detect various anomalies, including attacks, as was shown in Chapter 4 in the description of a Loki attack. It should be noted that there is no universal average packet size. In each segment, these values can significantly differ. For example, in the NSS tests [NSS1-02], the average packet size was equal to 300 bytes, while in other tests conducted by Shieply [Shipley1-99], 40% of all packets had lengths of less than 75 bytes, and only 30% were more than 1,426 bytes.

Table 10.2. Average Packet Lengths for Different Protocols

Protocol	Average packet size (in bytes)
ARP	28
ICMP Address Mask Request and Reply	12
ICMP Timestamp Request and Reply	20
ICMP Destination Unreachable Error	36
ICMP Echo Request	64
ICMP Time Exceeded, Parameter Problem, Source Quench, Redirect	36
DNS Query	40-50
DNS Response	70-90
TFTP	516
BOOTP	300
HTTP	400-1500
Telnet	64-1500
NFS	64-1500
Multimedia	400-700

When determining the type of traffic controlled by the sensor, you must not only take into account the network's average packet sizes and higher-level protocols, but also the MTU values (Table 10.3) for the type of network architecture used in the controlled segment [RFC1-90]. Notice that typical packet sizes for different protocols can vary in different networks. For example, in the Ethernet network, the size of an IP packet is limited to 1,500 bytes, while in the FDDI network this value increases to up to 4,772 bytes.

Table 10.3. MTU Values for Different Networks

Network type	MTU value (in bytes)
Token Ring (16 Mbit/sec)	17,914
Token Ring (4 Mbit/sec)	4,464
FDDI	4,352
Ethernet	1,500
Ethernet IEEE 802.3/802.2	1,492
X.25	576
NetBIOS	512
Point-to-Point	1,500 or 296
SLIP	1,006

Thus, you must ask the manufacturers at which packet length they measured the performance of their intrusion detection system. Practically all manufacturers process packets with a length of 1,500 bytes (maximum size of the Ethernet frame without a header), but not every product can handle 46-byte packets (minimum Ethernet frame without a header). Furthermore, some intrusion detection systems even fail when processing such small packets. Although small packets (runts) are quite rare (IP-telephony has not yet been developed enough), one should not exclude this possibility. Some manufacturers (such as Cisco Systems) specify this information in their promotion materials. Besides needing to know the average packet size, one should also have an idea of the number of such packets processed by the intrusion detection system per second. Based on these two parameters, one can approximately evaluate the throughput of the network-level intrusion detection system. This value, measured in Mbit/sec, can be calculated using the following simplified formula:

    Throughput = number of processed packets * packet size (in bits) / 1000000 

For example, if the packet size is 1,500 bytes, and the intrusion detection system processes 30,000 packets per second, the throughput of this IDS will be 360 Mbit/sec. At first glance, this is not too bad. However, in reality, the traffic comprises packets of different sizes, including service packets that are significantly smaller in size. Taking this fact into account, if the system processes the same number of packets per second, but the packet size drops to 64 bytes, the IDS throughput will be only 15.36 Mbit/sec. Actually, there is another aspect that ought to be considered when evaluating the IDS throughput, but we will leave this to be covered in detail in Chapter 12.

Acceptable Response Time

You should remember that, despite its importance, ensuring security is secondary to ensuring business continuity. Because of this, the installed intrusion detection system must not introduce any delays or otherwise influence the circulation of traffic. This especially relates to those active intrusion detection systems installed between protected and public networks. The list of applications susceptible to delays includes:

• Financial and other transaction processing applications

• Scheduling systems

• Systems for exchanging audio and video data

• IP telephony

• Multiplayer games

• Any other interactive applications

The Presence of Networks with Switching

This problem was already covered in detail earlier in this chapter. However, you should remember that there is no universal intrusion detection solution in networks with switching. It is possible that you will need to implement other methods of intrusion detection. In particular, you can use network analyzers (including RMON-based ones). The solution from NetSout (NetScout Probe) is an example of such an analyzer, in that it analyzes the data from which the n Genius Performance Management System can make the decision as to whether network anomalies exist [NetScout1-02]. Thus, it is possible to detect the malicious activity of worms such as Red Code or Nimda.

The Presence of Asymmetric Routes

This problem, already considered earlier in this chapter, is similar to the one that arises when using splitters, and will be covered in detail in Chapter 12. Consequently, it is necessary to know whether asymmetric routing is used within a controlled segment.

Scalability and Extensibility

Each network changes with time: information flows change their routes, new hosts and network segments are added, etc. This all influences the design of the intrusion detection system's infrastructure. As nothing in the world remains constant and unchanged, it is wise to consider future changes when choosing locations for IDS components. Obviously, it is impossible to predict and forecast everything. However, it is desirable to avoid situations in which adding another sensor will require you to move the management console to another location. Furthermore, any upgrade requires resources (time, human, and financial), which might exceed all possible limits if you do not plan the IDS infrastructure with scaling in mind.

Compatibility with Existing Software and Hardware

You can spare both effort and expense if you efficiently integrate the intrusion detection system with the existing network equipment and software. For example, instead of terminating the connection to the attacking host, the intrusion detection system can send an appropriate command to the firewall or filtering router. Let us consider another example. If your network is mainly oriented towards Microsoft DBMS products (MS SQL Server), then selecting an intrusion detection system using an Oracle database as its data store is not the best solution (although this decision might be made on account of other reasons).

The Availability of Channel and Subscriber Encryption

As will be shown in Chapter 12, like any other conversion of controlled traffic, encryption leads to the inability to detect any attack indications within it. One can proceed quite easily by simply disabling the transmission of encrypted traffic within the corporate network. However, what should you do in respect to access to protected web servers? Or what about for VPN connections between headquarters and remote offices? You will have to either stop using the IDS network component and solve the problem by installing a host-level component, or try to find where traffic is transmitted in plain-text mode. In any case, you must know beforehand whether or not encryption is used in the controlled segment.

Distribution between the Management Console and the Sensor

If you plan to install IDS agents in remote offices (away from headquarters, where the management console is installed) with access via the Internet, then it is necessary to bear in mind the following two aspects: the amount of data transmitted between the console and the sensor, and this data's protection. In contrast to the first issue, which has begun to lose its significance (most large companies now have communication channels of a significant bandwidth), the second problem must never be underestimated. The information circulating between the sensor and the console is both very important and confidential, since knowledge of system vulnerabilities would certainly help intruders implement a successful attack on your corporate network.

Security Tools between the Console and the Sensor

Even if the information exchange between agents and the management console is performed via the corporate network and stays there, one important problem still remains: the presence of the security tools between them. As was already mentioned, the presence of a firewall or IDS between the security scanner and scanned hosts can lock some attacks (including DoS attacks). It also might be a problem if the firewall locks all unknown traffic. Such traffic might include all interactions between the management console and the sensor. Because of this, you should know in advance which ports are used by the IDS components, and what the directions of interaction are. This data is required to create appropriate rules for the firewall or filtering router.

Dynamic Address Assignment in the Controlled Segment According to the DHCP Protocol

The arrival of the DHCP protocol has significantly simplified the process of automatic assignment of IP addresses. However, this protocol introduces specific features into the deployment of the IDS infrastructure. With the dynamic assignment of addresses, each host is assigned an IP address for a limited time - a so-called lease duration. When this period expires, this address can be assigned to another host. There are some possible issues that may have to be dealt with, including:

• Those related to the detection of the real address of an internal intruder.

• Problems with respect to the analysis of the attacking host addresses (which change dynamically).

• Problems regarding the reconfiguration of network equipment and security tools. To tell the truth, it is not a good idea to employ these tools in networks with dynamic address assignment.

• Issues related to security analysis by IP address

Since log files of the intrusion detection system store the information on the IP address of the attacked or attacking host taken from the packet IP header, this IP address might already be assigned to another host while this information is being analyzed. As a result, it will be impossible to get an adequate response to the incidents. Therefore, it is extremely important to register not only the IP address, but also the MAC addresses along with DNS and NetBIOS names.

Keep in mind that if you use static (both manual and automatic) address assignment, these problems never arise.

Interaction with the IT Department

All the problems described above are related mainly to technical aspects, and successfully solving them depends on the existence of a previously composed network map, described in detail in Chapter 7. However, do not forget that besides the technical component, the network's infrastructure also has an organizational component, which is no less and sometimes even more important. One of the most serious organizational problems that might arise is interaction with the IT department (if introduction of the sensors is delegated to the IT department). One possible conflict was already described earlier in this chapter - IT department employees might form the opinion that the intrusion detection system degrades network performance. By the way, it may very well do so if you place the network sensors and connect them to the SPAN port of the switch incorrectly. Another conflict might arise between the IT and information security departments, as both may demand to use the precious Gigabit SPAN port. You need it for the intrusion detection system, but the IT department also needs it to connect to the network analyzer. Solutions to most of these and other problems strongly depend on your skills at finding mutually acceptable solutions and in a word - compromising. Note that it is only by working in this way that you will be able to build a truly efficient intrusion detection infrastructure.

---