Module Objectives

In today's context where hacking and computer system attacks are common the importance of intrusion detection and active protection is all the more relevant. This module takes up a discussion on IDSs, Firewalls and Honey pots. After the completion of this module, you will be familiar with the following topics:

• Intrusion Detection System

• System Integrity Verifiers

• How is Intrusions Detected ?

• Anomaly Detection

• Signature Recognition

• How does IDS match Signatures with incoming Traffic?

• Protocol Stack Verification

• Application Protocol Verification

• Hacking Through Firewalls

• IDS Software Vendors

• Honey Pots

Intrusion Detection Systems (IDS)

• Intrusion Detection Systems (IDS) monitors packets on the network wire and attempts to discover if a hacker/hacker is attempting to break into a system (or cause a denial of service attack).

• A typical example is a system that watches for large number of TCP connection requests (SYN) to many different ports on a target machine, thus discovering if someone is attempting a TCP port scan.

 

Note  

An intrusion detection system (IDS) inspects all inbound and outbound network activity and identifies suspicious patterns that may indicate a network or system attack from someone attempting to break into or compromise a system. There are several ways to categorize an IDS:

• Misuse detection: In misuse detection, the IDS analyze the information it gathers and compares it to large databases of attack signatures. Essentially, the IDS looks for a specific attack that has already been documented. Like a virus detection system, misuse detection software is only as good as the database of attack signatures that it uses to compare packets against.

• Anomaly detection: In anomaly detection, the system administrator defines the baseline, or normal, state of the network's traffic load, breakdown, protocol, and typical packet size . The anomaly detector monitors network segments to compare their state to the normal baseline and look for anomalies.

• Network-based: In a network-based system, or NIDS, the individual packets flowing through a network are analyzed . The NIDS can detect malicious packets that are designed to be overlooked by a firewall's simplistic filtering rules. A Network Intrusion Detection System is responsible for detecting anomalous, inappropriate, or other data that may be considered unauthorized occurring on a network. An NIDS captures and inspects all traffic, regardless of whether it's permitted or not. Based on the contents, at either the IP or application level, an alert is generated. Network-based intrusion detection systems tend to be more distributed than host-based IDS.

• Host-based systems: In a host-based system, the IDS examines at the activity on each individual computer or host. Host-based systems collect and analyze data and aggregate them so that they can be analyzed locally or sent to a separate/central analysis machine. One example of a host-based system is programs that operate on a system and receive application or operating system audit logs. These programs are highly effective for detecting insider abuses . Residing on the trusted network systems themselves , they are close to the network's authenticated users. If one of these users attempts unauthorized activity, host-based systems usually detect and collect the most pertinent information in the quickest possible manner. In addition to detecting unauthorized insider activity, host-based systems are also effective at detecting unauthorized file modification.

  

• Passive system: In a passive system, the IDS detect a potential security breach, logs the information and signals an alert.

• Reactive system: In a reactive system, the IDS respond to the suspicious activity by logging off a user or by reprogramming the firewall to block network traffic from the suspected malicious source.

Intrusion detection is different from traditional firewalls because it involves the detecting of a security breach. In a firewall, if the traffic matches an acceptable pattern, it is permitted regardless of what the packet contains.

In general, network-based systems are best at detecting the following activities:

• Unauthorized outsider access: When an unauthorized user logs in successfully, or attempts to log in, they are best tracked with host-based IDS. However, detecting the unauthorized user before their logon attempt is best accomplished with network-based IDS.

• Bandwidth theft/denial of service: These attacks from outside the network single out network resources for abuse or overload. The packets that initiate/carry these attacks can best be noticed with use of network-based IDS.

Some possible downsides to network-based IDS include encrypted packet payloads and highspeed networks, both of which inhibit the effectiveness of packet interception and deter packet interpretation. Examples of network-based IDS include Shadow, Snort, Dragon, NFR, RealSecure, and NetProwler.

Furthermore, somewhere in the range of 80 - 85 percent of security incidents originate from within an organization. Consequently, intrusion detection systems should rely predominantly on host-based components , but should always make use of NIDS to complete the defense. In short, a truly secure environment requires both a network and host-based intrusion detection implementation to provide for a robust system that is the basis for all of the monitoring, response, and detection of computer misuse.

System Integrity Verifiers (SIV)

• System Integrity Verifiers (SIV) monitor system files to find when an intruder changes.

• Tripwire is one of the popular SIVs.

• SIVs may watch other components such as Windows registry as well as chron configuration to find known signatures.

 

Network intrusion detection systems (NIDS) monitors packets on the network wire and attempts to discover if a hacker/cracker is attempting to break into a system (or cause a denial of service attack). A typical example is a system that watches for large number of TCP connection requests (SYN) to many different ports on a target machine, thus discovering if someone is attempting a TCP port scan. A NIDS may run either on the target machine who watches its own traffic (usually integrated with the stack and services themselves), or on an independent machine promiscuously watching all network traffic (hub, router, probe).

System integrity verifiers (SIV) monitors system files to find when an intruder changes them (thereby leaving behind a backdoor). An integrity monitor watches key system structures for change. For example, a basic integrity monitor uses system files or registry keys as "bait" to track changes by an intruder. Although they have limited functionality, integrity monitors can add an additional layer of protection to other forms of intrusion detection.

The most popular integrity monitor is Tripwire, available for both Windows and UNIX. It can monitor a number of attributes, including file additions, deletions, or modifications; file flags; last access time; last write time; create time; file size; hash checking etc. A SIV may watch other components as well, such as the Windows registry and cron configuration, in order to find well known signatures. It may also detect when a normal user somehow acquires root/administrator level privileges.

Log file monitors (LFM) monitor log files generated by network services. The simplest form of IDS is a log file monitors, which attempt to detect intrusions by parsing system event logs. In a similar manner to NIDS, these systems look for patterns in the log files that suggest an intruder is attacking. A typical example would be a parser for HTTP server log files that looking for intruders who try well-known security holes, such as the "phf" attack.

Intrusion Detection

Anomaly Detection	Signature Recognition
The idea behind this approach is to measure a "baseline" of such stats as CPU utilization, disk activity, user logins, file activity, and so forth. The benefit of this approach is that it can detect the anomalies without having to understand the underlying cause behind the anomalies.	This means that for every hacker technique, the engineers code something into the system for that technique. This can be as simple as a pattern match. The classic example is to examine every packet on the wire for the pattern "/cgi-bin/phf?" which indicates an attempt to access this vulnerable CGI script on a web-server.

 

Note  

Anomaly Detection: Anomaly detection involves establishing a baseline of "normal" system or network activity and then sounding an alert when a deviation occurs. Because network traffic is constantly changing, such a design lends itself more to host-based IDS than network IDS. Anomaly detection provides high sensitivity but low specificity. An administrator can create profiles of the system and determine what their normal operational parameters are, and then look out for events that are out of profile. Therefore if someone who doesn't have the right credentials attempts to log into a system on the network - and is denied access, an application will set off an alarm on some other console to say that there is an unauthorized access. At another level, anomaly detection can investigate user patterns, such as profiling the programs executed daily. If a user in the graphics department suddenly starts accessing accounting programs or compiling code, the system can properly alert its administrators.

Note  

Signature Detection: Signature based works on a host or a network basis where a particular type of packet or data stream is looked for. This method uses specifically known patterns of unauthorized behavior to predict and detect subsequent similar attempts. These specific patterns are called signatures. An example is of a signature "three failed logins" used in host-based intrusion detection. For network intrusion detection, a signature can be as simple as a specific pattern that matches a portion of a network packet. When the scanner sees it, it generates an alarm. The issue with signature based applications is that they're really only as good as the signatures are. So they leave a large area to be desired in terms of total effectiveness. Moreover, the occurrence of a signature might not signify an actual attempted unauthorized access. Depending on the robustness and seriousness of a signature that is triggered, some alarm, response, or notification should be sent to the proper authorities.

How does an IDS match signatures with incoming traffic?

• Traffic consists of IP datagrams flowing across a network.

• An IDS is able to capture those packets as they flow by on the wire.

• An IDS consists of a special TCP/IP stack that reassembles IP datagrams and TCP streams. It then applies some of the following techniques:

  • Protocol stack verification

  • Application protocol verification

  • Creating new loggable events

 

Note  

Traffic consists of IP datagrams flowing across a network. A NIDS is able to capture those packets as they flow by on the wire. A NIDS consists of a special TCP/IP stack that reassembles IP datagrams and TCP streams. It then applies some of the following techniques:

Protocol stack verification A number of intrusions use violations of the underlying IP, TCP, UDP, and ICMP protocols in order to attack the machine. A simple verification system can flag invalid packets. This can include valid, by suspicious, behavior such as severally fragmented IP packets.

Application protocol verification A number of intrusions use invalid protocol behavior, such as DNS cache poisoning , which has a valid, but unusually signature. In order to effectively detect these intrusions, a NIDS must re-implement a wide variety of application-layer protocols in order to detect suspicious or invalid behavior.

Creating new loggable events A NIDS can be used to extend the auditing capabilities of the network management software. For example, a NIDS can simply log all the application layer protocols used on a machine. Downstream event log systems can then correlate these extended events with other events on the network.

Signatures range from very simple - checking the value of a header field - to highly complex signatures that may actually track the state of a connection or perform extensive protocol analysis. Signature analysis systems have a few key strengths. They are very fast, since packet matching is a relatively non-processor intensive task. The rules are easy to write and understand, as well as very customizable. These systems excel at catching low level, simple attacks since they tend to employ prepackaged exploits that are easy to recognize.

Protocol Stack Verification

• A number of intrusions, such as "Ping -O-Death" and "TCP Stealth Scanning" use violations of the underlying IP, TCP, UDP and ICMP protocols in order to attack the machine.

• A simple verification system can flag invalid packets. This can include valid, by suspicious, behavior such as severally fragmented IP packets.

 

Many network protocols are simple and easy to analyze. They involve one system sending a single request to another, and waiting for that system to respond. For example, a network monitor can easily determine the purpose of a single UDP DNS query by looking at one packet.

Other protocols are more complex, and require consideration of many individual packets before a determination can be made about the actual transaction they represent. In order for a network monitor to analyze them, it must statefully monitor an entire stream of packets, tracking information inside each of them.

Note  

Protocol analysis is a technique used by protocol based IDS to detect the presence of attack signatures. Protocol analysis takes advantage the high degree of order in network protocols and uses this knowledge to quickly detect the presence of an attack. This highly efficient technique results in an enormous reduction in the amount of computation required. As a result a greater number of attacks can be detected and analyzed in greater detail without missing a single packet, even on a fully loaded 100 Mbps line.

In a Protocol Analysis IDS, since the protocols are being decoded it is easy to check the fields of the various layers for illegal or suspicious values. And since the protocol is decoded, it is also easy to check the IP or TCP fragmentation bits. If the fragmentation bit is set the packets are first reassembled and then searched for attack signatures. By reassembling the packets the system is easily able to detect attacks hidden using packet fragmentation techniques. Protocol decoding also eliminates the false positives so common in pattern matching systems. False positives occur when a byte string in a packet matches a pattern signature, but the string is in fact not an attack at all.

However, due to the preprocessors required for advanced protocol examination, protocol analysis can be fairly slow to begin with. Furthermore, the rules for a protocol system are difficult to write and understand. Systems differ in their implementation of protocol standards and RFCs, making it difficult for IDS developers to write accurate processors.

Initially protocol-based IDSs appear slower than signature-based systems, though they address scalability and performance better. Furthermore, since they search for generic violations, protocol analysis engines can detect zero-day exploits unlike signature based system. Regrettably, they can also miss out obviously deviant events, such as a root Telnet session, that do not violate any protocol. Protocol-based systems keep the false alarms to a minimum, since they log real violations.

Example: Consider the following signature used by the ISS RealSecure IDS to detect the SQL Slammer worm. Note that the signature was in place nearly five months before the worm appeared, making it possible for users of ISS to detect the worm way ahead.

 SQL_SSRP_StackBo is (  udp.dst == 1434  ssrp.type ==4  ssrp.name.length > ssrp.threshold) 

Where ssrp.type is first-byte of packet

where ssrp.name is nul- terminated string starting at second byte

Where ssrp.threshold defaults to 97

ISS decoded Microsoft's SSRP protocol, and tested it for conditions - specifically vulnerability. This shows that using protocol analysis IDS doesn't have to wait till exploits appear to write signatures, but can instead test against the vulnerability itself.

Application Protocol Verification

• A number of intrusions use invalid protocol behavior, such as "WinNuke", which uses NetBIOS protocol (adding OOB data or DNS cache poisoning, which has a valid but unusual signature.

• In order to effectively detect these intrusions, an IDS must re- implement a wide variety of application-layer protocols in order to detect suspicious or invalid behavior.

 

Note  

We have seen how network IDS can detect varying signatures and how traffic can be analyzed at the protocol level. Evasion at the application layer is an aspect that is being addressed by NIDS better. A number of intrusions use invalid protocol behavior or valid, but unusual signature. In order to effectively detect these intrusions, a NIDS must re-implement a wide variety of application-layer protocols in order to detect suspicious or invalid behavior.

However, evasion at the application layer is a complex NIDS problem. The NIDS must completely mimic the application protocol interpretation. An attacker can use the differences between the application and the IDS as a gap that can be exploited for gaining access into the network or system. Signature-based NIDSs can find it difficult to deal with the complexities of application interactions. The potential for evasion at the application layer is increasing because new protocols are becoming more complex with support for features such as Unicode, which provides a unique identifier for every character in every language to facilitate uniform computer representation of the world's languages.

The major part of the problem at the application layer is that it is very difficult to synchronize the parsing done by the NIDS with the parsing done by the application. Intruders often exploit application protocol weakness for crashing applications or breaking into hosts . Attacks such as WinNuke and invalid packets that cause DNS cache corruption fall into this category. Host-based IDS that has access to the application logs is much more effective at detecting application layer attacks.

What happens after an IDS detects an attack?

1. Configure firewall to filter out the IP address of the intruder.

2. Alert user / administrator (sound / e-mail / Page).

3. Write an entry in the event log. Send an SNMP Trap datagram to a management console like HP Openview or Tivoli.

4. Save the attack information (timestamp, intruder IP address, Victim IP address/port, protocol information).

5. Save a tracefile of the raw packets for later analysis.

6. Launch a separate program to handle the event

7. Terminate the TCP session - Forge a TCP FIN packet to force a connection to terminate.