Using IPsec for Network Protection

Using IPsec for Network Protection

Although 802.1X is ok for initial authentication to an entire network (or network segment), think of IPsec as a layer of authorization that permits particular machines to communicate with each other (and within this layer, IPsec performs its own machine-to-machine authentication, too). Remember, given the attack we described previously, 802.1X alone, on wired networks, isn't good enough; combine it with IPsec to achieve the notion of authentication to the network (come in the front door) and authorization to resources (entering individual offices; using the photocopier and popcorn machine).

Several RFCs define IPsec; RFC 2401 outlines the basic structure and includes references to additional RFCs for the various individual protocols. Our explanation here will help you understand how it works at a relatively high level and give you some very useful scenarios where IPsec solves actual security problems. Generally our explanation describes the features that the Windows IPsec implementation supports. If you want to learn everything you can, see the references in the various footnotes throughout this section. (Also search on "IPsec" at http://www.rfc-editor.org to see everything that's available, including some recent proposed standards for implementing things such as dynamic routing, AES encryption in IKE, a new policy model, and so on.)

IPsec Operation

When two computers ( peers ) use IPsec to communicate, they create two kinds of security associations . In the first, called main mode or phase one , the peers mutually authenticate themselves to each other, thus establishing trust between the computers. In the second, called quick mode or phase two , the peers negotiate the particulars of the security association, including how they digitally sign and encrypt traffic between them. Packet signing ensures that the data hasn't been tampered with in transit; packet encryption ensures that the data isn't vulnerable to eavesdropping attacks.

A computer can have only one IPsec policy assigned at a time. The policy can have any number of rules, each of which has a filter list and a filter action . Filter lists contain one or more filters that specify the characteristics of the traffic that the rule should process: source and destination addresses, source and destination port numbers , and protocol types. Filter actions specify the behaviors of the rule: whether to permit traffic, block traffic, or negotiate the pair of IPsec security association. Actions that specify negotiating security can have many options, including encryption suites, per-packet authentication methods , how often to generate new keys, how to respond to incoming insecure requests , and whether to communicate with computers that don't support IPsec.

Each rule in an IPsec policy combines one filter list with one filter action. Traffic that matches a particular filter list is processed according to the settings in the linked filter action. Rules also indicate the security association's mode (transport or tunnel, explained later) and one of three phase-one authentication methods:

• Preshared keys. Included only for RFC conformance, it's a good idea to use preshared keys only when testing your IPsec policies. Every peer that participates in the same security policy needs the same preshared keyand remember the earlier comment about the problem with shared secrets. Furthermore, they're stored in the clear in the registry and visible to anyone with administrative privileges on the computer.

• Digital certificates. So long as each peer possesses an IPsec or computer certificate signed by an authority the other peer trusts, the peers authenticate to each other. Note where the trust lies: in the signer of the certificate. The actual name on the certificate is unimportant in this case. Digital certificates are much preferred over preshared keys because each peer can have its own certificate and a multilevel certificate hierarchy can help create more granular IPsec policies. For example, Super-secure Machine A might accept only certificates signed by High-value Authority X, while Sort-of-secure Machine B might accept certificates signed either by High-value Authority X or Medium-value Authority Y.

• Kerberos. Windows IPsec can also use Kerberos for the initial computer-to-computer authentication if both peers are in the same Active Directory forest. Kerberos is appropriate if you don't have a PKI and don't need to establish IPsec security associations between computers outside a single forest.

IPsec Modes and Methods

There is no such thing as an "IPsec tunnel." It bears repeating: there is no such thing as an IPsec tunnel. Yes, it's extremely common language and everyone thinks they know what it means, but in fact it's a meaningless phrase that lacks specificity. The proper terminology is "transport mode" or "tunnel mode," which describe the two kinds of phase one security association modes :

• Transport mode. This is the more common of the two, and is often what people have in mind when thinking of an IPsec tunnel. In IPsec transport mode, two peers authenticate each other (phase one) and establish the traffic signing and encryption parameters (phase two). Any traffic between the computers that matches the characteristics specified in the filter list gets signed and/or encrypted according to the details of the linked filter action. Transport mode ensures that communications between two computers remains tamper-free and private. Transport mode doesn't create new packets; instead, it secures existing packets. Interestingly enough, in L2TP+IPsec VPNs, it's transport mode, not tunnel mode, that secures the L2TP traffic between a client and a VPN server. ^[18]

  ^[18] See "Securing L2TP using IPsec" (ftp://ftp.rfc-editor.org/in-notes/rfc3193.txt). L2TP+IPsec is the only IETF-approved method for client remote-access VPNs: L2TP authenticates the user and assigns the client an IP address; IPsec encrypts the L2TP tunnel. Some other products implement Xauth and Mode Config, IPsec add-ons that were rejected by IETF for being cryptographically insecure.

• Tunnel mode. IPsec tunnel mode (not just IPsec tunnel) is intended for secure site-to-site communications over an untrusted network. Each site has an IPsec gateway configured to route traffic to the other site. When a computer in one site needs to communicate with a computer in the other site, the traffic passes through the IPsec gateways (and possibly through intervening routers in each site before reaching the local gateway). At the gateway, the outbound traffic is encapsulated inside another complete packet and secured according to the details of the filter action in the rule. Of course, the gateways have already performed their phase one authentication and established their phase two signing/encryption security associations. In Windows IPsec, tunnel mode is supported only for site-to-site VPNs on RRAS gateways and not for any kind of client-to-client or client-to-server communications.

A filter action can specify one of the three behaviors: permit traffic, block traffic, or negotiate security. The first two don't really do any kind of integrity or privacy checking: if traffic matches a filter list that's linked to a "permit" filter action, the traffic is allowed to pass; if traffic matches a filter list that's linked to a "block" filter action, the traffic is dropped. Filter actions that negotiate security can choose one or both of two different phase two security methods:

• AH (Authentication Header). ^[19] AH security associations are useful when the requirement is only for integrity but no confidentiality. AH computes a SHA1 (preferred) or MD5 digital signature across the entire packet (including the IP header that contains the source and destination addresses) and adds this signature to the packet. The receiver computes its own version of the signature and compares that with the signature stored in the header; if they match, the packet hasn't been modified. The IANA (Internet Assigned Numbers Authority) has assigned IP protocol number 51 to IPsec AH.

  ^[19] "IP authentication header" (ftp://ftp.rfc-editor.org/in-notes/rfc2402.txt).

• ESP (encapsulated security payload). ^[20] Use ESP security associations when you have a need for confidentiality. ESP negotiates a DES or 3DES (triple DES, preferred) session key that's exchanged between the peers and used for encrypting the traffic between them. You can also specify a SHA1 or MD5 digital signature in ESP, too. Note that both the ESP encryption and signature computations include the payload and the TCP/UDP header portions of each packet but not the IP header; compare to AH whose digital signature covers the entire packet. IPsec ESP is IP protocol 50.

  ^[20] "IP encapsulating security payload" (ftp://ftp.rfc-editor.org/in-notes/rfc2406.txt).

There are no dependencies between modes and methods. Both transport mode and tunnel mode security associations can use AH, ESP, or AH and ESP together. (An older, now deprecated, version of the specification for AH didn't define AH tunnel mode.)

IKE

IKE (Internet Key Exchange) is the mechanism by which IPsec security associations negotiate their protection suites and exchange signing or encryption keys. IKE defines how the peers communicate policy information and how authentication messages are constructed and exchanged. IKE is fairly complicated; to fully understand it it's helpful to possess multiple advanced degrees in mathematics and cryptography and to have copious amounts of spare time to read many detailed yet highly valuable resources. ^[21] IKE runs on UDP port 500 and is itself already encrypted; IKE is always exempted from IPsec processing ( otherwise , how could the security associations get set up?).

^[21] Naganand Doraswamy and Dan Harkins, IPSec 2nd Edition, Prentice Hall PTR (2004). See also RFCs 2403, 2404, 2405, 2407, 2408, 2409, 2410, and 2412.

IPsec over NAT

One of the biggest deployment blockers for IPsec is the presence of NAT (network address translation). IPsec authenticates computers; NAT hides them. So if you think about it, IPsec and NAT are fundamentally at odds: they are opposing technologies designed to do completely different things. Nevertheless, there's a huge demand for getting IPsec to work correctly over NAT devices. Indeed, making the two work together will rapidly accelerate the adoption of IPsecbut unfortunately won't rapidly accelerate the disappearance of NAT (which is merely an address preservation tool, not a security mechanism).

Several people from Microsoft spent some time thinking about all of the necessary requirements to make IPsec work over NAT. ^[22] It's more difficult than you might think. Three problems loomed over IPsec and NAT:

^[22] Most recently documented in "IPsec-network address translation (NAT) compatibility requirements" (ftp://ftp.rfc-editor.org/in-notes/rfc3715.txt).

• AH integrity violation. AH computes its digital signature of the packet before it leaves the sending peer. Then if that packet passes through a NAT (either local or remote), its IP header gets modified. The receiving peer, when it computes its own version of the packet's signature, generates a different result because the NAT modified the source address. Therefore the receiving peer drops the packet.

• IPsec "helpers." Many small/home office NAT gateways include a feature called IPsec helper (or IPsec passthrough). Originally designed for tunnel mode, these act on transport mode also. If multiple computers behind the gateway create IPsec security associations to destinations outside the gateway, the gateway forwards all incoming IPsec traffic to the first computer that created its security association. The "helper" function simply remembers which computer on the inside initiated an IPsec conversation and forwards all incoming traffic there, without modification.

• IKE fragmentation. It's common that the payload of a digital certificate exceeds the size of an Ethernet frame. Whenever an application generates a data packet larger than the network's frame size , IP fragments the packet so that each fragment fits in a single frame. Although this is fine inside a local network, network border devices (including many NATs) drop fragments because carefully crafted malicious fragments are a popular way to circumvent some firewalls. NATs that drop fragments prevent IKE from working properly.

Turns out, however, that by defining a mechanism to encapsulate ESP (but not AH) inside UDP, it's possible to forward IPsec traffic through a NAT without it getting rejected. Each side sends some discovery packets to the other to determine whether there is a local NAT present and whether both sides are capable of performing NAT traversal ("NAT-T"). If one or both sides is behind a NAT and both can perform NAT-T, IPsec first shifts the IKE exchange to UDP port 4500 (to avoid IPsec helper conflicts) and then encapsulates the entire IPsec security association in the same UDP conversation. To the NAT the traffic looks like ordinary UDP traffic, so the NAT can handle it just fine. Each side also exchanges its NAT details with the other so that they can properly reassemble the IP headers of received traffic. Remember that NATs change the headers, so receivers need to "remanufacture" the original header before decrypting traffic and checking signatures. The NAT-T setup process provides each side with the necessary information to do this.

UDP-ESP (as the specification is called) knows how to handle multiple IPsec security associations behind a single NAT device. When multiple computers make outgoing connections using the same protocol, NAT devices use unique source port numbers for each computerthis is how the NAT knows where to forward incoming return traffic. In each client, NAT-T maintains a table of source portsecurity association pairs and matches up traffic accordingly as it flows in and out.

In January 2005, NAT-T finally made it out of draft and onto the IETF standards track. ^[23] The standard doesn't solve the first problem; NAT-T is not defined for AH because there's no way to effectively work around the AH integrity violation problem. And IETF decided not to address the third problem, IKE fragmentation; if one side is blocking fragments, then that side's administrator must change the NAT programming or the side simply won't be able to participate in NAT-T. The Microsoft implementation of NAT-T includes a prefragmentation workaround that engages if both sides support it. IKE fragments the key into smaller portions before sending the data to the IP layer; each portion is placed into a fully formed IP packet and sent on its way. The receiving end is expecting this and delivers each portion to IKE, which reassembles the portions back into the complete key. The NAT discovery phase also checks for support of IKE fragmentation and uses it only if both devices are capable.

^[23] "Negotiation of NAT-Traversal in the IKE" (ftp://ftp.rfc-editor.org/in-notes/rfc3947.txt) and "UDP encapsulation of IPsec ESP packets" (ftp://ftp.rfc-editor.org/in-notes/rfc3948.txt).

Using IPsec to Stop Worms

It sounds condescending, but the best way to stop worms is not to get them in the first place! Alas, because not all people understand or even care about the threats and risks associated with electronic mail and Web surfing, worms and other kinds of malicious code are simply a fact of life right now. Given that, how can we reduce the damage that such code inflicts?

You can thwart malicious code in three different ways: prevent the code from installing, prevent the code from executing, or prevent the code from communicating. The first is the most difficult; although host firewalls and virus/spyware scanning utilities can stop some of it, it's often still the responsibility of the user to decide whether to permit the code to install. Certain features in Windows XP Service Pack 2 create additional barriers against malicious code but still rely on users either leaving the features enabled (which they are by default) or making correct decisions when the operating system raises prompts. You can eliminate some of these decisions by configuring many of the features through group policy, however.

SRPs (software restriction policies) help prevent malicious code from executing. Using AD group policy you can apply restrictions to a computer that allows only authorized programs to run. Resist the urge to think in the other directionthat is, allowing everything to run except known bad stuff. How can you really know everything that's bad? Combine an organization-wide allow-to-execute list with nonlocal administrator users and you've got an environment in which malicious code that does manage to get on a computer is unable to do anything dangerousthe SRP simply won't allow it to run. SRP is a heavy hand, and although it's nontrivial to implement, we're big fans of SRP and encourage you to investigate using it in your organization.

In some instances, your only option might be to prevent malicious code from communicating. IPsec policies help here both by limiting what kinds of traffic a computer accepts as well as what kinds of traffic a computer generates. Rules with filter actions that specify simply to block or allow trafficthat is, create no security associationscan build effective basic packet filters on individual computers. Assign these to computers using group policy to reduce the amount of malicious traffic propagating throughout your network.

Your choice of IPsec policies depends on which operating system you're running. Windows XP and Windows Server 2003 include host-based firewalls that are more effective than IPsec for blocking inbound traffic, so your IPsec policies would block only outbound traffic. Windows 2000 doesn't include a host-based firewall, so you should consider IPsec policies that block traffic both inbound and outbound.

Consider the "Slammer" worm. Slammer searched for computers running SQL Server or MSDE and therefore would have been listening on UDP port 1434. Because patching all computers can take some time, using AD group policies to quickly assign IPsec policies to all computers to block inbound traffic to the vulnerable port is an excellent alternate mitigation. To prevent a computer from getting infected by Slammer, you can assign a policy that blocks all inbound traffic from anywhere to the computer's own IP address with destination port 1434/UDP:

• Filter list with one filter from any-address:any-port to my-address:1434/udp

• Filter action block

• Rule link the list with the action; all interfaces; no tunnel; any authentication method (it doesn't matter because there's no IPsec security association here)

You can also script IPsec policies using command-line tools. There are three different tools that you can use depending on your operating system. For Windows 2000, the tool is ipsecpol.exe from the Resource Kit; ^[24] for Windows XP it's ipseccmd.exe from the Support Tools folder on the CD-ROM or through a download; ^[25] for Windows Server 2003 it's netsh ipsec included with the operating system. (All the command-line tools are very picky about syntax and upper-/lowercase. Be sure to study the built-in help before you begin experimenting with them.)

^[24] http://www.microsoft.com/windows2000/techinfo/reskit/default.asp .

^[25] http://support.microsoft.com/?kbid=838079.

You can apply the Slammer filter in Windows XP with this command:

[View full width]

 
 [View full width] 
 ipseccmd -w REG -p "Block UDP 1434 Filter" -r "Block Inbound UDP 1434 Rule" -f *=0:1434  :UDP -n BLOCK x 

(In Windows 2000 the command is ipsecpol with the same syntax.) This command creates and assigns a static policy called "Block UDP 1434 Filter" with a single rule called "Block Inbound UDP 1434 Rule" containing the same filter list as above linked to a "block" filter action. Static policies are stored in the Registry and persist between reboots. The policy won't apply until the next boot or restart of the IPsec policy agent, so if you want the policy to apply immediately, your script should also stop and restart the service called "policyagent." If you use the UI to create the policy, it applies immediately. Remember that if you're using Terminal Server to administer the computer!

In case a computer does get infected with Slammer, you can use a different IPsec rule to prevent the computer from infecting other computers by blocking outbound communications to destination port 1434/UDP:

• Filter list with one filter from my-address:any-port to any-address:1434/udp

• Filter action block

• Rule link the list with the action; all interfaces; no tunnel; any authentication method (it doesn't matter because there's no IPsec security association here)

Note the subtle difference here: in the first rule, the filter list is from any-address:any-port to my-address:1434/udp, where in the second rule the filter list is from my-address:any-port to any-address:1434/udp. The second rule blocks any outbound traffic that's destined for UDP port 1434 on any computer.

Use this command to script that rule and add it to the same policy as the first:

[View full width]

 
 [View full width] 
 ipseccmd -w REG -p "Block UDP 1434 Filter" -r "Block Outbound UDP 1434 Rule" -f 0=*:1434  :UDP -n BLOCK 

The -x is omitted here because this command is adding another rule to the existing policy.

You can also use the command-line utilities to apply dynamic policiesthese stay in effect only so long as the system is running. They are lost if the policy agent is restarted or the computer is rebooted. These two commands create a dynamic policy that does the same thing as the static policy above:

 ipseccmd f[*=0:1434:UDP] ipseccmd f[0=*:1434:UDP] 

The square brackets around the filter specification indicate that this is traffic the policy engine should block.

So far we've been exploring how to use IPsec to block traffic from and to known "bad" destinations. You could be more restrictive with your IPsec policies and block all traffic, and then create rules that permit certain traffic to certain locations. You will need to think very carefully about what kinds of traffic should be allowed where, plan extensively, and thoroughly test your ideas before launching into production. Expect stuff to break at first!

Using IPsec to Protect Servers

One very good use of a block-all-except policy is for server protection. Suppose you're building a Web server. Why should that server accept any inbound traffic that isn't Web traffic, at least on its Internet- facing connection (if dual- homed )? You can use an IPsec policy to build a rudimentary packet filter that discards everything except that which makes sense for the purpose of your serversin the example here, everything except traffic destined for TCP port 80 (and 443 if some pages use HTTPS). There's one thing to note: on extremely heavily loaded servers, performance under IPsec block/allow policies can suffer. RRAS packet filters do better but they aren't as manageable. Here again, as with many security mitigations, which one to choose is a matter of selecting the tradeoffs that make the most sense for you.

Such policies buy you time to test and deploy patches. If someone discovers a vulnerability in the operating system but an IPsec policy isn't permitting access to the vulnerable service, you don't need to patch right away . Reread that; yes, it says what you think it says. We aren't advocating that you should never patch. But whenever possible, if there are other mitigating factors that can protect you, it's better to do those right away so that you remain protected. Now you've got time to test and deploy according to your outage schedules and not affect any service level agreements that might be in place. You might even think you could wait until the next service pack, but we don't recommend that, given the time interval between service packs .

Let's continue the Web server example. Your IPsec policy would have two rules:

• Rule 1:

  Filter list with one filter from any-address:any-port to my-address:any-port

  Filter action block

  Rule link the list with the action; all interfaces; no tunnel; any authentication method (it doesn't matter because there's no IPsec security association here)

• Rule 2:

  Filter list with two filters from any-address:any-port to my-address:80/tcp and from any-address:any-port to my-address:443/tcp

  Filter action permit

  Rule link the list with the action; all interfaces; no tunnel; any authentication method (it doesn't matter because there's no IPsec security association here)

To script this, use these commands:

[View full width]

 
 [View full width] 
 ipsecpol -w REG -p "Allow Web Traffic" -r "Block Everything" -f *+0 -n BLOCK x ipsecpol -w REG -p "Allow Web Traffic" -r "Permit Inbound TCP 80" -f *+0:80:TCP f *+0  :443:TCP -n PASS 

Note in these examples that a + replaces the = between the source and destination address/port/protocol specifications. This tells the policy agent to build "mirror" rules, which are required for reply traffic to leave the Web server. Without the mirror, you'd need separate rules permitting outbound traffic from the server's ports 80 and 443. When you create rules in the GUI they're mirrored automatically.

Think about the roles of various servers in your organization and start to develop IPsec policies that are appropriate for those roles. Use AD group policy to assign the IPsec policies based on organizational units that reflect each role. You will measurably increase the security of your environment simply by implementing a method to limit the traffic that can enter a server.

Using IPsec for Domain Isolation

If you're using Active Directory, you know who your users are: they have to authenticate when they want to use network resources. But what about the computers? Sure, some of the computers are joined to the domain. The architecture of Windows doesn't require this, however. As long as a user possesses valid credentials, he or she can access network resources from any computer on the network. And Windows XP's credential manager makes it even easier to live with a non-domain-joined computer!

The concept of "domain isolation" is becoming more and more popular. We first started mentioning it in late 2001; it's now running across all of Microsoft's corporate network ^[26] and in the networks of many customers. If you haven't considered it yet, we encourage you to think about it now. Recently Microsoft published full prescriptive guidance that will help you implement domain isolation in your network; download your copy from http://www.microsoft.com/technet/security/topics/architectureanddesign/ipsec/default.mspx .

^[26] See "Improving security with domain isolation" ( http://www.microsoft.com/technet/itsolutions/msit/security/ipsecdomisolwp.mspx ) for full details on the implementation of domain isolation at Microsoft.

Domain isolation is important for many reasons. Domain-joined computers are computers that you can trust, at least somewhat, because you can take advantage of things such as group policy, security templates, software restriction settings, IPsec policies, Systems Management Server, and any other security- related technologies that you can centrally control and manage. Computers whose configuration you control are computers that do only what you allow them to do. ^[27] These are computers that are far less dangerous in your environment than rogue unmanaged machines that you have no knowledge of either their existence or their configurations.

^[27] Provided, of course, the users don't run as local administrators.

Here you're essentially saying that no user can bring up a rogue machine and access domain resourcesonly authorized computers can communicate with other authorized computers. It's easier than you might think to implement this across your domain. First add this IPsec policy to your default domain group policy:

• Filter list use the existing All IP Traffic example filter list

• Filter action ESP only, null encryption, SHA-1 integrity; require security; don't communicate with non-IPsec machines

• Rule link the list with the action; all interfaces; no tunnel; Kerberos authentication; no default response

You could use AH rather than ESP-null, but then your policy wouldn't work with devices that must communicate over NATs. You also need to create a rule that exempts your domain controllers, because you need to communicate with them to authenticate and get the Kerberos ticket that's used for all other communications:

• Filter list filters with the addresses or address ranges of your domain controllers

• Filter action permit

• Rule link the list with the action; all interfaces; no tunnel; any authentication method (it doesn't matter because there's no IPsec security association here)

You also need similar exceptions for devices that can't participate in IPsec, such as network printers, computers with older operating systems that can't run IPsec, and so on.

Test this first! You'll probably make a mistake or two. After you test and deploy these policies, nondomain machines (that aren't specifically exempted) will be unable to communicate with any domain-joined machine. Why? Domain members require IPsec. You can't get the policy unless you join the domain. You can't "roll your own" policy and try to remain outside the domain because the policy requires Kerberos, which works only if you're in the domain. All domain members receive the policy and will therefore be able to communicate with all other domain members .