Encryption Schemes

Encryption is the process of transforming regular, readable data, or plaintext , into scrambled, or unreadable, form, called ciphertext . Decryption is the reverse process ”transforming ciphertext into plaintext. The process of encryption can be used in various ways to ensure privacy, authenticity, and data integrity:

• Privacy No one should be able to view the plaintext message except the original sender and intended recipient.

• Authenticity The recipient of an encrypted message should be able to verify with certainty the identity of the message sender.

• Data integrity The recipient of the message should be able to verify that it has not been tampered with or altered in any way while in transit.

Encryption is accomplished using an encryption algorithm, typically a pair of closely related mathematical functions that perform the actual encryption and decryption on the data provided to them. Modern encryption algorithms, including the ones used in Check Point NG AI, utilize what is called a key (or keys) to aid in the encryption or decryption process. There are two types of encryption algorithms: symmetric and asymmetric.

Encryption Algorithms: Symmetric vs. Asymmetric Cryptography

In what is called symmetric encryption , the encryption algorithm itself is public while the key is a secret. Anyone who discovers the key and has knowledge of the algorithm can decrypt any messages encrypted with that key. Since both the sender and recipient need to know the secret key before they can communicate, you must have a secure method of exchanging the key. Sometimes you will hear the term sneakernet used to describe this key exchange process, meaning that the exchange takes place via phone, fax, or in person, since an online exchange cannot be encrypted prior to the sharing of the key. Sometimes you will hear this key referred to as a shared secret . Symmetric encryption is typically very fast, but it has some disadvantages:

• As stated, anyone discovering the secret key can decrypt the messages.

• Since each sender/recipient pair (we will call them users ) needs a separate secret key, the number of separate keys that need to be managed increases rapidly as the number of users increases . Mathematically, we need n ( n “1) / 2 keys for a network of n users. Using this formula, a network of 500 users, for example, requires 124,750 unique keys.

Asymmetric encryption was developed to solve the problem of secure key exchange and to improve key management. It is called asymmetric because the encryption and decryption keys are different. In one form of asymmetric encryption, called public key encryption , the sender and recipient each have two keys, one of which is public and can be openly shared and another of which is private and is kept secret and never shared. If Alice wants to send an encrypted message to Bob, for example, she and Bob only need to exchange their public keys. The method used for the exchange need not be private in this case. Alice encrypts the plaintext message to Bob using Bob s public key. When Bob receives the message, he decrypts it using his private key. This method of public key encryption, invented in 1976 by Whitfield Diffie and Martin Hellman, is sometimes called the Diffie-Hellman algorithm .

Another form of asymmetric encryption, called RSA encryption, is used by Check Point NG AI for generating digital signatures.

As we can see, asymmetric encryption solves the problem of key exchanges needing to be done in private. Users need only share their public keys to encrypt messages to one another. Asymmetric encryption does suffer one serious drawback, however: It is much, much slower than symmetric encryption (on the order of 1,000 times slower). For this reason, real-life encryption schemes tend to use a hybrid form of public key exchange and private (symmetric) key encryption. Check Point NG AI is no different in this regard. A Diffie-Hellman key pair is used to generate and exchange a shared secret key, which is used for all encryption and decryption after the initial public key exchange. The shared secret key in this case is sometimes called a session key . The shared key can be regenerated at periodic intervals to lessen the chance of its compromise.

An encryption algorithm s security is completely dependent on its keys and how they are managed. Strong encryption that has a flawed key management algorithm is actually weak encryption. You will often hear of an encryption algorithm described as using a 128-bit key, for example. What this means is that, if the algorithm is implemented properly, someone who tried to enumerate every possible key in order to break your encryption (called a brute-force attack ) would have to try 2128 different key combinations to be guaranteed success. This is not computationally feasible for the foreseeable future. In practice, cryptanalysts typically attack an algorithm s key generation or key management scheme instead, attempting to find a flaw such as a predictable sequence of keys to exploit. An example of this is a very technical but small flaw that enables an attacker to dramatically reduce the number of possible keys when decrypting WEP used for wireless networks.

The moral of all this is that you should pay attention to an algorithm s implementation rather than to its key size exclusively. The latter will not guarantee your security. Note that asymmetric encryption schemes typically have key sizes that are much larger than symmetric ones (1024 bits, for example). The strength of these keys cannot be equated to the strength of symmetric keys, since they use different mathematical principles. The original Diffie-Hellman public key scheme, for example, was based on the difficulty of factoring very large prime numbers .

Check Point makes available several encryption algorithms. They are enumerated in Table 10.1, along with their shared key sizes and whether they are based on a public standard or are proprietary.

Table 10.1: Check Point Encryption Algorithms

Algorithm	Key Length in Bits	Standard
CAST	40	Public
DES	56	Public
3DES	168	Public
AES	256	Public

IKE and ISAKMP

The Internet Security Association and Key Management Protocol (ISAKMP), or Internet Key Exchange (IKE), is an Internet encryption, authentication, and key exchange standard put forth by the Internet Engineering Task Force (IETF). In today s Internet, it is widely used for implementing VPNs. Because ISAKMP is a standard, a Check Point firewall utilizing it will be able to interoperate with other third-party VPN products. Check Point firewalls are known to interoperate with Linux gateways (Free/SWAN), OpenBSD, SonicWall, and Watchguard firewall products, as examples. The ISAKMP key exchange process is divided into two phases and utilizes what are called security associations (SAs) to facilitate encryption and key generation. Keys and SAs are regenerated on a periodic basis.

IKE uses what is called tunneling-mode encryption . This means that each packet that is to be sent over a VPN is first encrypted in its entirety (both header and data payload are encrypted) and then encapsulated with a new header. The new header will differ based on whether the packet is just being encrypted, just being authenticated, or both. This tunneling mode slightly degrades network performance because it increases the size of each packet, reducing the amount of actual data transferred per packet, but it is more secure than in-place encryption.

Before standards for VPN technologies were available, Check Point developed the first widely used encryption scheme for encrypting network traffic. Though no longer used, FWZ was a Check Point proprietary key exchange scheme that utilizes another proprietary protocol, Reliable Datagram Protocol, or RDP (not the same as the RDP described in RFC 1151), to negotiate encryption and authentication methods between gateways.

FWZ used what is called in-place encryption , in which packet bodies are encrypted, leaving the original TCP/IP headers in place. This method of encryption is faster (because it does not inflate the size of the packets by encrypting the headers) than tunneling mode but at the expense of security, since original header information is left in a readable state, including IP addresses, which are internal to an organization. Note that because FWZ does not encapsulate packets before sending them through a VPN, FWZ could not be used in situations in which any networks participating in the VPN domain have nonroutable addresses.

Hash Functions and Digital Signatures

A hash function , also known as a one-way function , is a mathematical function that takes a variable-length input and generates a fixed-length output, which is typically much smaller than the input. If we pass a plaintext message through a hash function, we produce what is called a message digest . A good hash function is one that, if we are given the message digest, is impossible to reverse and deduce the original message. It is also one in which for any two different function inputs (two different messages, in this context), the output should be unique to the input. To put it another way, the message digests for two different messages should also be different. As we will see, this principle can be used to ensure the integrity of a message. If a hash function generates the same message digest for two different inputs, we call this a collision . A good hash function will minimize collisions. When we talk about hash functions, we usually specify the length of the message digest in bits. This roughly corresponds (strength-wise) to the length of a symmetric encryption key. For example, a commonly used hash function, MD5, produces a 128-bit message digest for any size input it is given.

The output to a hash function is usually much smaller than the original message as well. MD4 and MD5 are good examples of hash functions. You may have heard of an MD5 checksum. This checksum is the result of sending a file through the MD5 hash algorithm.

Another important note about hash functions is that the output is unique to the message. If the original message were tampered with in any way, a different message digest would result. Since you cannot decrypt a message digest, you run the algorithm against the message and compare the two digests to verify that the message is intact. This is how data integrity is achieved .

A digital signature is an attachment to a message that utilizes a hash function and enables the receiver to authenticate the sender and verify data integrity. Digital signatures can be attached to encrypted messages. Check Point NG generates digital signatures using an RSA private key and a hash function, as shown in this example, where Alice wants to send a digitally signed message to Bob:

1. Alice sends the (unencrypted) message through a hash function, producing a fixed-length message digest.

2. Alice encrypts the message digest with her private RSA key and sends it on its way, along with the encrypted message. The encrypted message is now signed by Alice.

3. Bob decrypts the message as usual and passes it through the same hash function Alice used when it was sent. Bob compares this message digest he just generated with the decrypted message digest sent to him, making sure they match. Alice s public key is used to decrypt the message digest in this case.

A match in this case means that Bob can be sure that Alice sent the message and that no one tampered with it in transit. We are assuming here that Bob trusts that he is using Alice s public key; this trust is usually provided by a certificate authority who will certify public keys.

The two hash functions offered by Check Point are MD5 and SHA-1. MD5 is a 128-bit hash function; SHA-1 is considered more secure, with a 160-bit message digest length.

Certificates and Certificate Authorities

A certificate authority (CA) is a trusted third party from which we can reliably obtain a public key. A certificate is issued by a CA and contains reliable information about the entity that wants to be certified authentic . This entity could be a person s or a firewall s public key or a secure Web server host name and domain.

In the case of Check Point VPNs, certificates can be used by encrypting gateways to exchange public keys and to authenticate one another. Typically, the gateways themselves act as CAs in this regard.

Types of VPNs

There are logically two types of VPN: site-to-site and client-to-site . Site-to-site VPNs are what we normally think of when we think of a VPN: two gateways communicating across an insecure network (usually the Internet), with encrypted traffic passing between them.

Client-to-site VPNs, on the other hand, have a fixed gateway at one end and a mobile client on the other, perhaps with a dynamic IP address. This type of VPN is implemented by Check Point s SecuRemote or SecureClient products.

VPN Domains

We can define a VPN domain as a group of hosts and/or networks behind a firewalled gateway that participate in a VPN. In a site-to-site VPN, each gateway has its own VPN domain defined and is also aware of the other gateway s VPN domain. Any traffic coming from one VPN domain and going to the other (behind the opposing gateway) will be encrypted outbound and then decrypted inbound at the other end.

VPN domains are defined on each gateway s firewall object and must be set up with certain rules in mind. We talk about this concept in more detail when we discuss VPN implementation.

Designing & Planning
VPN Domains

It is important not to include either peer s gateway object in their respective VPN domains; otherwise , traffic to or from each gateway will be encrypted, which is not what we want, nor can it work, since key exchange has not yet taken place. Contrast this with single entry point (SEP) configurations, in which gateways must be a member of each VPN domain. Furthermore, for nonroutable VPN domains, make sure opposing subnets are not identical. In large deployments in which you may have more than one gateway, each with a unique VPN tunnel, make sure the VPN domains don t overlap or include the same hosts/networks in both domains. Both gateways will want to encrypt traffic in cases where traffic passes through more than one gateway on the way to its destination. It is better to use a SEP configuration for this, with some dynamic routing protocol inside you local network.