Lesson 2: Planning Encryption of Transmitted Data

Protecting data from being modified during transmission is critical, but sometimes you must further protect the transmitted data by encrypting it. Encryption protects the transmitted data from inspection by unauthorized users.

---

After this lesson, you will be able to

• Design a strategy for encrypting transmitted data by using application layer technologies

Estimated lesson time: 30 minutes

---

Planning Secure E-Mail Encryption

Although digital signing protects e-mail messages from modification, it doesn't prevent someone from inspecting them during transmission across the network. The default protocol used for sending e-mail messages is Simple Mail Transfer Protocol (SMTP). SMTP doesn't include any extensions for the encryption of email. As such, the contents of e-mail messages are vulnerable to inspection.

Figure 11.7 illustrates a plaintext e-mail message that has been intercepted during transmission.

Figure 11.7 Network monitor capture of an e-mail message

The graphic shows that the contents of the e-mail message are transmitted in clear text. From the message we can determine that [e-mail] sent his e-mail password to [e-mail] and that the password is X454eg!!GG.

NOTE

---

The actual password isn't shown in the SMTP process. It's only because the user put the password in the text of the message that the password is exposed to the attacker using a network monitor. Never share your password with another user under any circumstances.

Analyzing the E-Mail Encryption Process

For e-mail encryption to be successful, the recipient requires a private/public key pair. The e-mail encryption process takes place as shown in Figure 11.8.

Figure 11.8 The e-mail encryption process

1. The sender of the e-mail creates a plaintext message.
2. The sender encrypts the message by using the recipient's public key. To perform the encryption, the sender must have access to the recipient's public key. The public key can be either retrieved from a directory or exchanged by having the recipient send a digitally signed message to the sender.
3. The encrypted message is sent to the recipient of the message using the SMTP protocol.
4. The recipient decrypts the message using the recipient's own private key. This use of the public and private keys ensures that only the holder of the private key (the recipient) can view the contents of an encrypted message.
5. The decrypted message is displayed in the recipient's e-mail application.

As with digital signatures, the recipient's public key has to be deployed for email encryption to take place. The public keys must be made available either by exchanging digitally signed messages, or in the case of a Windows 2000 network, the public keys can be retrieved from Active Directory.

Determining Encryption Levels for E-Mail Encryption

E-mail messages can be encrypted by using different algorithms. Supported algorithms in Microsoft Outlook 2000 include

• Rivest's Cipher v2 (RC2). RC2 is a secret-key block encryption algorithm developed by Ron Rivest at RSA Security that uses 64-bit input and output blocks. The key size can be varied up to 128 bits in length with most implementations using 40-bit or 128-bit length keys. RC2 is optimized for speed and encrypts messages faster than DES or 3DES on slower computers.
• Data Encryption Standard (DES). DES is the most widely used encryption algorithm in the world. DES takes 64-bit blocks of plaintext and applies a 56-bit key to each block of plaintext. This key is the recipient's public key. The encrypted package is decrypted using the recipient's private key.
• Triple DES (3DES). 3DES increases the strength of DES by using an encrypt-decrypt-encrypt process that uses three keys. The 64-bit plaintext message block is first encrypted with the first key. Then the encrypted result is decrypted using a second key. Finally, the result of the decryption process is encrypted using a third key. The formula to arrive at the encrypted packet is E_k3[D_k2[E_k1[Plaintext]]] where E_k3, E_k2, and E_k1 are the three separate encryption keys. The resulting encryption strength is 168 bits (3 × 56-bits).

IMPORTANT

---

Use of RC2 (128 bit) and 3DES require the Windows 2000 High Encryption Pack to be installed. The installation of the Windows 2000 High Encryption Pack is subject to your country's import and export laws. The United States allows the export of high encryption to nonembargoed nations. For more information, see www.microsoft.com/exporting/.

Select the e-mail encryption algorithm that meets your organization's security needs. The encryption algorithm you select must abide by all the laws on the export and import of strong encryption products that your organization uses.

Determining Protocol Choices for E-Mail Encryption

As with digitally signing messages, you have to choose between S/MIME and PGP for your encryption protocol. The decision is based on the e-mail applications implemented in your organization. You can't mix encryption protocols for e-mail. If you use S/MIME to encrypt the message, you have to use S/MIME to decrypt it.

Making the Decision

Include the following considerations in your security plan when deploying e-mail encryption:

• Determine all approved e-mail applications that are in use. The decision on whether to use S/MIME or PGP is determined by the security protocols supported by your primary e-mail system. You may not be able to support all email applications deployed within an organization.
• Determine who can use secure e-mail. If you don't have a private/public key pair for e-mail encryption, you can't receive encrypted e-mail. Your organization must draft a set of criteria that, if met, qualifies an individual to acquire a private/public key pair for encrypting e-mail.
• Determine where the private/public keys will be acquired. You must choose between implementing your own internal CA for certificate distribution or purchasing certificates from a public CA such as Verisign or Thawte. If you choose your own CA, remember that the certificates may not be trusted by other organizations. If you choose a public CA, costs are associated with each certificate acquired from the CA.
• Establish guidelines for the distribution of public keys to recipients outside the organization. Recipients outside the organization won't have access to your Exchange or Windows 2000 directory.
• Establish an external public point for CRLs if using an internal CA. External e-mail recipients need to verify the revocation status of any encrypted messages. Ensure that the CRL is available to the outside world by publishing the CRL to an externally available service, such as your external Web server.

  NOTE

  ---

  For more information on designing CRL publication points, see Chapter 10, "Planning a Public Key Infrastructure."

• Train users on when to encrypt messages. You should set guidelines outlining the types of e-mail that should be encrypted when sent on the network. Guidelines reduce the risk of critical data being sent in an unencrypted format.

NOTE

---

While you could simply implement a policy that all e-mail messages be encrypted when sent, this isn't necessary in most cases and necessitates more processing at both the client and the recipient.

Applying the Decision

Fabrikam will require encryption of e-mail sent to the Defense Department and between project members on the Sonar System project. The same infrastructure that's required for digitally signing e-mail messages works for encrypting e-mail messages.

Because the encryption requirements are for e-mail messages sent to other organizations, it's recommended that you acquire the Mail certificates either from a public CA, such as Verisign or RSA, or ensure that the CAs that issue the certificates to the mail users have their CRLs available on the Internet for certificate revocation checking.

The users involved in the two projects should be trained in how to encrypt messages when the messages are sent to recipients in other companies. The process may require that a digitally signed message is first sent between the two users who require encrypted mail. If the message is digitally signed, it contains the certificate and public key that's used to encrypt messages. Remember that the public key of the recipient is used to encrypt messages sent to that recipient.

Planning Application-Level Encryption with SSL/TLS

Applications other than e-mail, such as Web pages containing sensitive data, also require encryption of data when it's transmitted. For example, Windows 2000 supports two forms of application-level encryption: SSL and Transport Layer Security (TLS).

• SSL. Provides encryption services to several applications by using public and private keys to encrypt data transmitted between a server and a client. While most commonly associated with Web browsers, SSL is also used to provide encryption services to Lightweight Directory Access Protocol (LDAP) queries, Network News Transfer Protocol (NNTP) authentication and news group transfers, Post Office Protocol v3 (POP3) authentication and e-mail retrieval encryption, and Internet Message Access Protocol v4 (IMAP4) authentication and e-mail retrieval encryption. As with secured Web browsing, the applications must support the use of SSL encryption.

  Configuring an Application to Support SSL Encryption

  Applications must include options to take advantage of SSL or TLS protection. For example, Microsoft Outlook Express supports the use of SSL encryption for POP3 by providing a check box to enable SSL encryption of POP3 credentials, as shown in Figure 11.9.

  

  Figure 11.9 Enabling POP3 SSL protection in Outlook Express

  If the application isn't enabled to recognize SSL encryption, you can't configure the application to do so. To use SSL, the application must be programmed to support SSL.

• TLS. TLS is very similar to SSL in that it provides communications privacy, authentication, and message integrity by using a combination of public key and symmetric encryption. TLS uses different encryption algorithms than SSL. TLS is an IETF draft standard. Windows 2000 uses TLS to encrypt smart card authentication information transmitted when using Extended Authentication Protocol (EAP).

NOTE

---

TLS also supports the option of reverting to SSL support if needed. TLS may replace SSL in the future.

Both SSL and TLS are implemented between the TCP and application layers, as illustrated in Figure 11.10.

Figure 11.10 SSL and TLS encrypting data between the transport and application layers

When an application is SSL-enabled, the application listens for client connections on a different port than the usual port used by the application. Table 11.4 shows the ports used by protocols when SSL is implemented at the server.

Table 11.4 Standard and SSL Ports

Protocol	Standard Port	SSL Port
Hypertext Transfer Protocol (HTTP)	80	443
Internet Message Access Protocol v4 (IMAP4)	143	993
Lightweight Directory Access Protocol (LDAP)	389	636
Network News Transfer Protocol (NNTP)	119	563
Post Office Protocol v3 (POP3)	110	995
Simple Mail Transfer Protocol (SMTP)	25	465

Deploying SSL and TLS

Both SSL and TLS require the server hosting the application that uses SSL or TLS to acquire a private/public key pair for encrypting the data. Figure 11.11 shows the encryption process for Web applications.

Figure 11.11 SSL Encryption for Web–based transactions

1. The Web client attempts to connect to the Web server by using SSL. No additional software is required for the client. The client simply changes the protocol in the URL from http: to https:.
2. The Web server returns the Web server's certificate and public key to the Web client. The Web client requires the public key to encrypt any transmissions sent to the Web server. The certificate is included so that the Web client can access the public key and determine CRL locations if revocation checking is enabled on the Web client computer.
3. The Web client and Web server enter into a negotiation to determine encryption levels. The Web server and Web client negotiate to determine if 40-bit, 56-bit, or 128-bit encryption will be used for the session key.
4. The Web client generates a session key and encrypts the session key with the Web server's public key. The session key is set to be the length negotiated between the Web client and the Web server. Once the session key is encrypted, the encrypted session key is transmitted to the Web server.
5. The Web server decrypts the session key using the Web server's private key. Only the Web server has access to this private key, ensuring that the connection attempt isn't intercepted by an attacker.
6. The session key is used to encrypt all further data exchanged between the Web client and the Web server.

The benefit of using application-level security is that the encryption requires no additional work by the user. The only noticeable change is that the user must use https: in the URL rather than http:.

NOTE

---

Only applications that are programmed to use SSL and TLS can implement the two forms of encryption. Only applications that recognize SSL and TLS can make the appropriate calls to programming interfaces to encrypt and decrypt data.

Making the Decision

Use the decision matrix shown in Table 11.5 to design application-level encryption of data when using SSL or TLS.

Table 11.5 Designing SSL and TLS Encryption

To	Include the Following in Your Design
Enable secure Web communications	Acquire a Web server certificate from a trusted CA. The CA can either be a private CA or a public CA, depending on the application.
Enable secure Web communications for a public Web site	Acquire a Web server certificate from a public CA such as Verisign or RSA. The use of a public CA certificate increases consumer confidence in your Web site.
Enable secure communications for a private Web site	Acquire a Web server certificate from a private CA within your organization's PKI. This action reduces the costs of the Web site deployment.
Secure authentication to your Web site and support any browser	Configure the Web server to use basic authentication and require SSL encryption for access. The SSL encryption encrypts the plaintext authentication used by basic authentication. In a Microsoft Internet Explorer deployment, consider deploying Windows Integrated Authentication, which isn't supported by other browsers. Deploy client certificates and configure the Web server to use certificate–based authentication.
Define the level of encryption to use for a Web site	Determine the type of information that will be available at the Web site. Determine how export and import laws affect the use of strong encryption. Always use the strongest level of encryption allowed by law to ensure protection of the data.
To enable strong encryption at a Windows 2000 Web server	Acquire a server certificate for the Web server to enable SSL communications. Configure the Web to require 128-bit encryption to allow server connections.
To enable strong encryption at a Windows client	For Windows 2000 clients, install the Windows 2000 High Encryption Pack to enable 128-bit encryption at the Windows 2000–based computer. For older versions of Windows clients, install the strong encryption patch for the version of Internet Explorer installed on the computer. For other companies' browsers (such as Netscape), download the strong encryption patch for the browser.
Minimize reduction in performance due to encryption of transmitted data	Enable SSL only for the Web pages that require encryption. To minimize excessive CPU utilization, reduce the amount of Web content being encrypted. In other words, don't automatically encrypt the entire Web site. Minimize the amount of graphics contained in secured Web pages. Typically, you don't need to encrypt graphics embedded in HTML pages.

Applying the Decision

You must protect three separate Web sites with SSL encryption to ensure that information entered into or downloaded from Web pages isn't compromised during transmission. These Web sites include the Defense Department Web site, the time input Web site for the Sonar System project, and a secured Web site for the Sonar System project server.

Defense Department Bidding Web Site

The first Web site that uses SSL encryption is the Defense Department bidding Web site. Although the Defense Department administers this Web site, the network administrators must ensure that any users who connect to the Web site from Fabrikam are enabled for strong encryption. If the computer used to connect to the Defense Department is a Windows 2000–based computer, the Windows 2000 High Encryption Pack must be applied to enable 128-bit encryption for the browser. If the computer runs Windows 95, Windows 98, or Windows NT 4.0, the 128-bit encryption patch for the current version of its browser must be installed to enable strong encryption.

Sonar Project Time Sheet Web Site

The Sonar System project members who work at A. Datum Corporation must enter the number of hours they work into a Web–based time recording system hosted in the Fabrikam extranet.

The Web server hosting the Time Sheet application must have a Web server certificate. Because only members from a partner organization are connecting to the Web site, an internal CA can issue the Web server certificate. If an internal CA is used, the CRL for the CA and any CAs that issued certificates in the certificate chain must be made available at a public location as indicated in the issued certificate. This action ensures that if the client Web browser is configured to perform certificate revocation checking, the checking won't cause the certificate to fail the revocation check.

You could configure the Web server to require 128-bit encryption, but you don't have to.

The Sonar System Project Server

The Sonar System project server at the San Francisco office requires that all communication with the server is protected from interception.

Making the data accessible by using a Web server allows the data to be accessed by a mix of Windows 95, Windows 98, Windows NT 4.0, and Windows 2000 clients. Web pages containing links for key files or directory browsing must be enabled at the Web server to allow access to the data.

The project server requires the acquisition of a Web server certificate. Because the server is required only for internal project use and the partners at A. Datum Corporation won't require access, you can use an internal CA to acquire the necessary Web server certificate to enable SSL encryption. The client computers may require additional configuration if the Web server is configured to require 128-bit encryption.

Lesson Summary

The encryption of data at the application layer allows data to be protected from inspection as it's transmitted across the network. Application-layer security benefits the network administrator because little configuration of the client computers is needed to deploy the application-layer security.