Chapter 10: Planning, Implementing, and Maintaining Internet Protocol Security

Introduction

Securing sensitive or mission-critical data is an important part of the network administrator’s job. Data is especially vulnerable to interception as it travels across the network. Windows Server 2003 includes Microsoft’s implementation of the Internet standard IP Security (IPSec) protocol, for the purpose of protecting data in transit. This chapter deals with how to work with Windows Server 2003’s IPSec. We start by introducing IPSec terminology and concepts and explaining how IPSec works “under the hood” to secure data in transit over the network. We discuss the purposes of IPSec encryption: authentication, integrity, and confidentiality. You’ll learn about how IPSec operates in either of two modes: tunnel or transport.

Although we refer to IPSec as a protocol, it is actually a framework, or a collection of protocols and standards designed to protect IP data in transit. In this chapter, you’ll learn about the protocols used by IPSec. These include the two primary protocols: the Authentication Header (AH) protocol and the Encapsulating Security Payload (ESP) protocol. We’ll also discuss the roles of additional protocols used by IPSec, including the Internet Security and Key Management Protocol (ISAKMP), Internet Key Exchange (IKE), the Oakley key-determination protocol, and the Diffie-Hellman key-agreement protocol. You’ll learn about Windows Server 2003’s IPSec components—the IPSec driver and the IPSec Policy Agent service. We’ll also discuss the relationship of IPSec to Internet Protocol version 6 (IPv6).

Next, we’ll show you how to deploy IPSec on your network, taking into consideration organizational needs and security levels, and help you determine the appropriate authentication methods. You’ll learn about managing IPSec, and we’ll walk you through the process of using the IP Security Policy Management Microsoft Management Console (MMC) snap-in as well as the command-line tools. We’ll discuss the role of IPSec policies, including default and custom policies, and we’ll show you how to assign and apply policies. We’ll also talk about IPSec security considerations and issues, including the use of a strong encryption algorithm (Triple Data Encryption Standard, or 3DES), authentication methods, firewall packet filtering, unprotected traffic, Diffie-Hellman groups, and the use of pre-shared keys. We’ll show you how to use the Resultant Set of Policy (RSoP) and the RSoP MMC snap-in to view policy assignments and to simulate policy assignments for deployment planning.