Analyzing Network Traffic

After receiving network traffic, your IPS sensors must analyze that traffic and then perform certain actions based on the results of that analysis. IPS sensor network traffic analysis falls into the following categories:

• Atomic operations

• Stateful operations

• Protocol decode operations

• Anomaly operations

• Normalizing operations

Atomic Operations

Some attack signatures identify situations in which the entire attack signature can be observed by analyzing the contents of a single packet. These signatures are the easiest to process because the IPS sensor has to examine only a single packet to determine if the signature is present. Whenever the attack signature is seen in a packet, the signature triggers, regardless of what traffic came before that packet or the packets that follow that packet.

Stateful Operations

Not all signatures can be processed without maintaining state on the traffic that the IPS sensor has previously observed on the network. For example, most TCP-based signatures require the traffic to be part of a valid TCP connection to actually trigger the signatures. Therefore, the IP sensor must maintain a list of the valid connections that it has seen. This list of valid connections includes information for each connection, such as the following fields:

• Source port

• Destination port

• Current source sequence number

• Current destination sequence number

• Connection state

Usually, this state information is maintained only for a specific amount of time. For example, inactive connections usually time out after a configured length of time. If the space claimed by inactive sessions were not reclaimed after a preset period of time, it would be a small matter for an attacker to consume significant resources on the sensor (by initiating connections and never terminating them). This stale state information would then limit the effectiveness of the sensor to analyze new connections.

Protocol Decode Operations

Instead of searching for known bad text strings, many intrusion systems decode traffic based on a specific protocol definition. This enables the analysis to check for bad values in specific fields of the protocol. When you verify that the traffic matches the protocol specification, it enables signatures to identify traffic that does not conform to the protocol specification, such as peer-to-peer (P2P) traffic with a destination port of 80, which is supposed to be HTTP traffic. Furthermore, decoding traffic for a specific protocol also enables the signatures to more accurately identify attack traffic, thus minimizing false positives.

Note

One drawback to utilizing protocol decodes is that not all applications are completely compliant with the RFC. In some situations, this noncompliance can result in the IPS sensor impacting the flow of legitimate traffic.

Anomaly Operations

For anomaly operations, the sensor triggers a signature when it observes traffic that deviates from a configured normal value. For example, assume that the normal amount of Internet Control Message Protocol (ICMP) traffic on your network is 100 K per second, with bursts of 1 M per second for no more than a second. You can create a signature that establishes the normal amount of ICMP traffic based on these parameters. Now, suppose the amount of ICMP traffic is 500 K per second for 5 seconds. This exceeds the normal definition, so the ICMP anomaly signature triggers.

Normalizing Operations

When your IPS sensors operate in in-line mode, you can actually alter the traffic that it receives. This functionality is useful in various situations, such as removing Time to Live (TTL) anomalies in a TCP connection. Normalizing traffic operates as follows:

Step 1.	Sensor captures packet on an interface.
Step 2.	Sensor analyzes packet.
Step 3.	Signature configured for normalization triggers.
Step 4.	Sensor generates normalized packet.
Step 5.	Sensor transmits normalized packet on outbound interface.

A good example of the way in which normalization modifies packets can be illustrated with the TTL field in a packet. Whenever a source system sends a packet across the network, it initializes the TTL field to a specific value. As each router or other infrastructure device forwards this packet, the TTL is decremented. If the TTL reaches 0 before the packet reaches the target system, the packet is dropped.

An attacker might attempt to send packets that have different initial TTL values in the hope that the packets with smaller TTL values will be processed by the IPS sensor, but not reach the target system. If this happens, the IPS sensor's analysis will be inaccurate because the analysis is not based on the same packets that the target system received and processed (some packets were dropped before reaching the target because of the TTL reaching 0).

Using normalization, the IPS in-line sensor can automatically adjust the TTL values for the packets to match the smallest TTL observed on a specific connection between two systems. Therefore, the analysis performed by the sensor matches the same packets that the host system receives and processes.