Signature Triggers

The heart of any IPS signature is the mechanism that causes it to trigger. These triggering mechanisms can be simple or complex, and every IPS incorporates signatures that use one or more of these basic triggering mechanisms to trigger signature actions. These triggering mechanisms can be applied to both atomic and stateful signatures. Current IPSs incorporate various triggering mechanisms when developing signatures, including the following:

• Pattern detection

• Anomaly-based detection

• Behavior-based detection

Trigerring Mechanism

Triggering mechanisms refer to the conditions that cause an intrusion system to generate a signature action. For example, the triggering mechanism for a burglar alarm might be a motion detector that detects the movement of an individual entering the alarmed room. A Network IPS might trigger a signature action if it detects a packet with a payload containing a specific string going to a specific port. A Host-based IPS might trigger a signature action when a specific function call is invoked. Anything that can reliably signal an intrusion or security policy violation can be used as a triggering mechanism.

Protocol Decodes

Another common triggering mechanism is called protocol decodes. Instead of simply looking for a pattern anywhere in a packet, protocol decodes involve breaking down a packet into the fields of a protocol and then searching for specific patterns in a specific protocol field or some other malformed aspect of the protocol fields. The advantage of protocol decodes is that it enables a more granular inspection of traffic and reduces false positives.

Table 2-1 shows the relationship between the various signature types and triggering mechanisms.

Table 2-1. Signature Type Versus Signature Trigger

Signature Trigger	Signature Type
	Atomic Signature	Stateful Signature
Pattern detection	No state required to examine pattern to determine if signature action should be applied	Must maintain state or examine multiple items to determine if signature action should be applied
Anomaly detection	No state required to identify activity that deviates from normal profile	State required to identify activity that deviates from normal profile
Behavior detection	No state required to identify undesirable behavior	Previous activity (state)required to identify undesirable behavior

The following sections explain the signature triggering mechanisms in detail. Table 2-2 and Table 2-3 provide example signatures that illustrate the various combinations of signature types and triggering mechanisms to help clarify how the different signature types and triggers combine to create useful signatures.

Table 2-2. Host-Based Signature Examples

Signature Trigger	Signature Type
	Atomic Signature	Stateful Signature
Pattern detection	Searching for the string confidential in a data file	Searching for the string SELECT FROM in a URI
Anomaly detection	Detecting a function call that is not part of the normal profile	Two function calls that are part of the normal profile, but have never been called within 1 second of each other
Behavior detection	Searching for any invocation of cmd.exe	Searching for an e-mail application (program that has previously generated or received e-mail traffic) invoking command.com

Table 2-3. Network-Based Signature Examples

Signature Trigger	Signature Type
	Atomic Signature	Stateful Signature
Pattern detection	Detecting for an Address Resolution Protocol (ARP) request that has a source Ethernet address of FF:FF:FF:FF:FF:FF	Searching for the string confidential across multiple packets in a TCP session
Anomaly detection	Detecting traffic that is going to a destination port that is not in the normal profile	Verifying protocol compliance for HTTP traffic
Behavior detection	Detecting abnormally large fragmented packets by examining only the last fragment	Searching for RP Crequests that do not initially utilize the PortMapper

Each of these triggering mechanisms has its benefits and drawbacks. Using the correct triggering mechanism in the appropriate situation greatly improves its efficiency. IPS devices that support multiple triggering mechanisms can more adequately support efficient signatures for a wide variety of activities without significantly impacting the performance of the IPS device.

By understanding the mechanisms that a signature can use to identify an activity, you can more efficiently determine a product's true capabilities.

Pattern Detection

The simplest triggering mechanism is identifying a specific pattern. This pattern can represent a textual or binary string or it can be other patterns, such as a sequence of function calls. Besides simple string patterns, most systems provide enhanced pattern detection using the following mechanisms:

• Regular expression (regex) patterns

• Deobfuscation techniques

Specifying string patterns using regex provides the ability to efficiently search for textual patterns (using a single regular expression) while making it harder to bypass the pattern without detection.

Regular Expression

A regex is a pattern-matching language that enables you to define a flexible search pattern. Using regex, you can easily define complex search patterns. Many different programs use regex to enable you to define custom search strings. In the UNIX world, for example, the grep, command is a common program that utilizes regex to search for text inside of files (or other output). For example, to perform a case-insensitive search for the word attack in the file named output.results, you can use the following command:

     grep [Aa][Tt][Tt][Aa][Cc][Kk] output.results 

This command finds various permutations of the word attack, such as ATTAck, attack, AttaCk, and so on.

Besides searching using flexible string patterns, many protocols (such as HTTP) accept multiple encoding mechanisms for input data. Some of the common encoding mechanisms include the following:

• American Standard Code for Information Interchange (ASCII)

• Hexadecimal

• Unicode

Note

Hackers often combine these encoding methods in an effort to trick IPS devices and other monitoring applications. In the past, various web servers could be tricked into accessing data and applications from other directories in what is known as a directory traversal attack.

Unless your IPS can decode the input data correctly (before performing a pattern search), it will miss valid strings that have been obfuscated. For example, the following patterns each request the same web page:

• http://10.10.10.10/index.html/../etc/password

• http://10.10.10.10/index.html/%2e%2e/etc/password

• http://10.10.10.10/index.html/%c0%ae%c0%ae/etc/password

In each of these requests, the .. is being represented differently. If your IPS correctly deobfuscates these requests (before performing a pattern search), then it can correctly detect the attempt to request a web page outside of the root of the web server's directory tree in each of these web requests (which all request the same web page).

Pattern Matching Considerations

Depending on the fidelity of the patterns that your signatures use, one of the problems with pattern matching can be the rate of false positives generated, especially with atomic signatures.

One of the benefits of pattern matching is that the signature clearly correlates to a specific attack. When the signature triggers, you know which attack it is detecting. This is different from anomaly detection in which the signature indicates only that something outside of the configured normal parameters has been detected.

Host-Based Examples

A Host-based IPS product might have the capability to examine the data inside a file on the hard disk. An example of a pattern-based signature trigger is the word confidential being detected in any data file. A stateful signature with the same triggering mechanism might look for the word confidential in inbound network traffic. The pattern is the same, but can be matched only if multiple packets have been re-assembled.

Network-Based Examples

Most Network-based IPS devices include a robust pattern-matching capability because many of the attack signatures involve searching for patterns in different network protocols. Most of these pattern-matching signatures are also stateful signatures because the information being examined can occur across multiple packets.

Note

Examining network traffic using pattern matching does not work when the traffic stream is encrypted. However, Host-based IPS might be able to examine the traffic depending on where it is examining the traffic (before or after the decryption).

The following regex string searches for an attempt to change the working directory to the root directory during an FTP session:

        [ \t]*[Cc][Ww][Dd][ \t]+[~]root 

Examining traffic with a destination port of 21 (the default FTP server port) detects FTP sessions in which the user attempts to change the working directory to root.

Note

If you have FTP servers listening on ports other than TCP port 21, then you need to configure your IPS to monitor these non-standard ports to have the IPS identify FTP attacks to these other ports.

Anomaly-Based Detection

Anomaly-based (also known as profile-based detection) signatures are not based on a specific event. Instead these signatures trigger when a certain activities deviate from what is considered normal. To utilize an anomaly-based signature, you must first determine what normal activity means for your network or host. This is usually accomplished by monitoring your network (or specific applications on your host) for a specific period of time to observe what is considered normal activity. Once you define normal activity, then you can configure your anomaly-based signature to trigger whenever activity on the network or a specific host deviates from your defined normal profile by a certain amount.

Anomaly-Based Detection Considerations

One of the biggest limitations of anomaly-based signatures is that you need to learn (or define) what is considered normal. As your network evolves, your definition of normal might also have to change. Furthermore, you need to guarantee that during your learning phase, your network is free of the attack traffic that you are going to detect (otherwise, this activity will be considered normal traffic). Furthermore, just defining normal can sometimes be difficult because most networks comprise a heterogeneous mixture of systems, devices, and applications that continually change.

Another potential drawback to anomaly-based systems is that when a signature generates an alert, it might be very difficult to correlate that alert back to a specific attack, because the alert indicates only that non-normal traffic has been detected. More analysis is required to determine whether the traffic represents an actual attack and what the attack actually accomplished. Furthermore, if the attack traffic happens to be similar to normal traffic, the attack might go undetected.

Because anomaly-based signatures do not look for specific attacks, they can be used to detect previously unpublished attacks. This is a major advantage for anomaly-based detection. Instead of having to define a large number of signatures for various attack scenarios, you simply define a profile for normal activity. Any activity that deviates from this profile is then abnormal and triggers a signature action. The drawback is that an alert from an anomaly signature does not necessarily indicate an attack. It indicates only a deviation from the defined normal, which can sometimes occur from valid user traffic.

Host-Based Examples

A good example of anomaly detection is using the Profiler utility from the Cisco Security Agent (CSA) software. This tool allows you to monitor an application over a period of time. During this time, it records all of the functions calls that the application uses. You can then use this information to build a profile for the application that can then be used to identify when the application makes anomalous function calls (any function call that is not in the profile).

Note

The Profiler utility is useful to analyze custom applications that are unique to your network. Most common applications (for a specific operating system) have already been incorporated into CSA's predefined rules. As of CSA version 4.5, the Profiler utility is now referred to as Application Behavior Investigation.

Sophisticated Host-based IPS tools might be able to apply stateful conditions to function calls recorded during the learning process. If two function calls that are a part of the normal profile occur within 1 second of each other and this has never happened before, this could trigger a signature. This is an example of a stateful signature with an anomaly-based triggering mechanism.

Network-Based Examples

A Network-based IPS can have various anomaly-based signatures. Some simple examples of anomaly signatures with the Cisco IPS solution are its flood signatures. These signatures detect floods of various types of traffic on your network. For example, you might want to monitor the amount of Internet Control Message Protocol (ICMP) traffic on your network. First, you need to measure the amount of ICMP traffic that appears on your network during normal operation. Then the flood signature triggers a signature action whenever the ICMP traffic on the network exceeds the configured maximum threshold for a specific length of time.

Other anomaly-based signatures involve ensuring that traffic to a set of ports matches a defined protocol specification. In this situation, the protocol specification defines what is considered normal. Any traffic that does not conform to this protocol specification is abnormal and triggers a signature action.

Note

The Cisco application inspection and control (AIC) signatures provide signatures that provide you with anomaly detection signatures for both HTTP and the FTP by verifying protocol compliance.

Behavior-Based Detection

Behavior-based detection is similar to pattern detection, but instead of trying to define specific patterns, you are defining behaviors that are suspicious based on historical analysis. The behaviors define classes of activity that are known to be suspicious. For example, an e-mail client running shell commands using the Windows cmd.exe program normally indicates something unusual such as a virus attempting to do something to your system.

Behavior-Based Detection Considerations

Similar to pattern matching, behavior-based signatures must be defined before you can use them. It takes a lot of research to determine behaviors that do not occur normally and can accurately indicate suspicious behavior.

The use of behaviors enables a single signature to cover an entire class of activities without having to specify each individual situation. For example, having a signature that triggers a signature action when an e-mail client invokes cmd.exe enables you to apply the signature to any application whose behavior mimics the basic characteristics of an e-mail client without having to apply the signature to each e-mail client application individually. Therefore, if a user installs a new e-mail application, the signature still applies.

Host-Based Examples

Behavior-based signatures are easy to visualize at the host level. A simple example involves the cmd.exe program. This program is frequently used by malicious programs to execute commands and scripts on the system. It is also frequently used by users during their normal day-to-day operations. Using a behavior-based signature, you can detect suspicious invocations of cmd.exe while ignoring the normal uses of this program. For example, an e-mail client that invokes cmd.exe is performing suspicious activity (because the e-mail client does not need to run cmd.exe to operate). Incorporating a signature that triggers a signature action whenever cmd.exe is invoked by any e-mail client application is an excellent example of a behavior-based signature.

Network-Based Examples

At the network level, you might find it a little more difficult to identify the behavior-based signatures. For example, a signature that identifies attempts to directly access RPC applications is behavior-based, because the normal behavior is to first communicate with the PortMapper.

RPC Protocol

The Sun Remote Procedure Call (RPC) protocol enables one system to run applications on another system across the network. These RPC applications are not bound to well-known ports. Instead, a program called the PortMapper keeps track of which RPC application is operating on which port. When a system wants to communicate with an RPC application on a remote system, it first contacts the PortMapper to find the port that the RPC application is operating. Then the system can communicate directly with the application through that port.