16.1 SECURITY-ENHANCED APPLICATION PROTOCOLS

Team-Fly

There are several application protocols that have been enhanced to provide integrated security services. In the following subsections, we address security enhancements for terminal access, file transfer, electronic mail, WWW transactions, DNS, and distributed file systems. Take them as examples; there are many other application protocols that can be enhanced with security features but are not addressed in this book (e.g., SNMP).

16.1.1 Terminal Access

Telnet [1] and Rlogin [2] are the protocols of choice to provide terminal access services for systems running the TCP/IP communications protocol stack. From a security point of view, Telnet is the preferred protocol, as it employs a password-based authentication scheme (contrary to Rlogin, which employs an address-based authentication scheme that depends on and makes use of trusted hosts).

Referring to the introductory remarks, it is possible to use an authentication and key distribution system to provide security services for a particular application protocol, such as Telnet or Rlogin. More accurately, an authentication and key distribution system provides an API that applications can use to incorporate and provide the security services they require. While something like Kerberos-mediated Telnet encryption [3] might be an ultimate goal, it takes a big effort to deploy. The same is true for any other authentication and key distribution system one may think of (e.g., SPX [4]).

Instead of waiting until authentication and key distribution systems are used and widely deployed on the Internet, several security-enhanced Telnet software packages have been developed in the past. In Chapter 15, for example, we learned about the SSL and TLS protocols being able to secure any application protocol layered on top of TCP. Consequently, these protocols can also be used to secure Telnet. Exemplary implementations include SSLtelnet and SSLTel. In fact, the port number 992 has been assigned by the IANA to run Telnet on top of SSL/TLS.

In addition to these possibilities, there are a number of technological approaches and corresponding products that can be used to secure applications that provide terminal access. For example, we also mentioned in Chapter 15 that Steven Bellovin and Matt Blaze developed the ESM that may serve as a secure Telnet replacement for 4.4BSD UNIX [5]. The ESM implements a Diffie-Hellman key exchange and the Interlock protocol originally invented and proposed by Ron Rivest and Adi Shamir [6] to protect against the man-in-the-middle attack. Because the software is not widely deployed on the Internet, some other software packages are described next. In either case, it is important to use such a software package as a replacement for Telnet or Rlogin in an intranet environment. It is even more important to use such a replacement software package if users regularly connect to intranet hosts from the Internet.

Secure RPC Authentication

The Secure RPC Authentication (SRA) software package was originally developed and distributed by a group of researchers at the Texas A&M University (TAMU) [7]. SRA is based on Sun Microsystems' Secure RPC implementation to protect Telnet and FTP connection establishment handshakes against password sniffing attacks.

When an SRA-enhanced client connects to an SRA-enhanced Telnet (or FTP) server, a Diffie-Hellman key exchange is performed to negotiate a session key. This session key is then used to encrypt the user's authentication information, which typically consists of a username and password. Unfortunately, the remainder of the Telnet (or FTP) traffic is left unencrypted. There are two major problems related to the use of SRA:

The length of the modulus used with Sun Microsystems' implementation of the Secure RPC Diffie-Hellman key exchange is only 192 bits and thus far too small to protect against more serious cryptanalytical attacks [8].
The Secure RPC and hence also SRA are vulnerable to the man-in-the-middle attack. An attacker can decrypt and reencrypt user identification and authentication information such as a password or an encrypted time stamp.

In addition to that, the SRA software package also contains a DES library that was not exportable from the United States and Canada. Nevertheless, there had been software packages available overseas that combined the SRA basic software with publicly available DES libraries.

Secure Telnet

Secure Telnet (STEL) is another Telnet replacement that was developed at the University of Milan in cooperation with the Italian CERT [9].

Again, the purpose of STEL is to provide its users remote terminal access, very similar to Telnet and Rlogin. The main differences, however, are that the authentication mechanisms employed by STEL are much stronger than their traditional counterparts and that all data traffic between the client and the server is transparently encrypted. The client software is intended to be directly run by users, whereas the server software can be run as a standalone daemon by the superuser or automatically be invoked by inetd.

Similar to SRA, STEL uses a Diffie-Hellman key exchange with a shared prime number p (either 512 or 1,024 bits) and a shared generator g = 3 to have a client and a server agree on a common session key. The actual session key is derived from the Diffie-Hellman value by hashing it with MD5. To protect against the man-in-the-middle attack, STEL also implements the Interlock protocol. In addition, a variety of methods are currently supported by STEL to provide client authentication (e.g., UNIX passwords, S/Key, and SecurID). With regard to UNIX passwords, it should be noted that the communication channel between the client and server is transparently encrypted and that this encrytion provides a reasonable level of security for the transmission of the password.

After the Diffie-Hellman key exchange and client authentication, all data traffic is transparently encrypted using the specified encryption algorithm keyed with the session key. The default encryption algorithm is DES, because it is fast, but Triple-DES and IDEA are also available. Because the STEL software has been developed entirely outside the United States and Canada, there is no problem in distributing the software package internationally.

Secure Shell

Probably the most important and most widely used and deployed replacement for Telnet and Rlogin on the Internet is the Secure Shell (SSH).^[1] SSH is a relatively simple program that can be used to securely log into a remote system, execute commands on that system, and move files from one system to another. SSH provides strong authentication and secure communications over insecure network connections. As such, it was originally designed to replace the Berkeley r-tools (e.g., rlogin, rsh, rcp, and rdist), but can also be used to replace Telnet in many cases. Furthermore, any application protocol layered on top of TCP (including, for example, the X protocol) can be secured using the integrated port forwarding mechanism of SSH. There are currently two versions of the SSH protocol:

SSH version 1, or SSH V1, refers to the protocol that was originally developed by Tatu Ylönen at the Helsinki University of Technology in Finland [10]. The corresponding implementation, including source code, documentation, and configuration scripts, was made publicly and freely available on the Internet. It was ported to many platforms and is in widespread use today. For example, most Linux software distribution packages come along with SSH V1 support. In addition, there are many commercial implementations of SSH V1 that are being sold by various companies, including, for example, the SSH Communications Security Corporation,^[2] a company founded by Ylönen after the public release of the first SSH software package. Refer to [11] for an overview about SSH V1 implementations.
SSH version 2, or SSH V2, refers to the protocol that is specified by the participants of the IETF Secure Shell (SECSH) WG.^[3] The first Internet-Draft specifying SSH V2 was submitted to IETF in 1997 and has been further refined in a series of Internet-Drafts since then. Again, there are many commercial and noncommercial implementations of SSH V2 [11]. For example, the commercial version that is being marketed by the SSH Communications Security Corporation is called the SSH Secure Shell (currently in version 2.4). Also, another Finnish company, called F-Secure, distributes a commercial version of SSH that is widely used and deployed on the Internet today.

In either case, there are open source implementations of SSH commonly referred to as OpenSSH.^[4] OpenSSH is being developed under the auspices of the OpenBSD project^[5] and is freely usable under a BSD license.

SSH V1 and V2 both utilize a generic transport layer security protocol.^[6] When used over TCP/IP, the server normally listens for TCP connection establishment requests on port 22. This port number has been registered with the IANA and is officially assigned for SSH. In short, the SSH protocol provides support for both host authentication and user authentication, together with data compression and data confidentiality and integrity protection.

The SSH V1 protocol is simple and straightforward. It requires the server be equipped with two public key pairs:

A long-term host public key pair with a relatively long key size (typically 1,024-bit RSA);
A short-term server public key pair with a shorter key size (typically 768-bit RSA).

Contrary to the host public key pair, the server public key pair changes periodically (i.e., every hour by default). The aim of having two public key pairs and having one of them (i.e., the server public key pair) change periodically is to provide something like PFS (refer to Chapter 14 for a discussion of PFS and key management protocols that provide PFS). Note that the purpose of the host public key is to bind the connection to the desired server host, whereas the purpose of the periodically changing server public key is to make decrypting recorded traffic impossible even in the case of a private host key being compromised. To achieve PFS, however, it must be ensured that the server private key never be saved to disk.

When a user wants to use SSH V1 to get terminal access to an SSH server, the SSH client sends an authentication request to the server. The server, in turn, sends back both of its public keys. The client now compares the received host key against its own database of manually distributed and preconfigured host keys. The client normally accepts the key of an unknown host and stores it in its database for future use. This makes the use of SSH practical in most environments. However, in more secure environments it is also possible to configure the SSH client to refuse access to any host whose public key has not been properly registered in the client database. If the client accepts the host key, it generates a 256-bit random number that serves as a session key. Furthermore, the client chooses an encryption algorithm from those supported by the server, typically Blowfish, DES, or 3DES in CBC mode. The client pads the session key with random bytes, double encrypts it with the public host and server RSA keys, and sends the result to the server. The server, in turn, decrypts the RSA double encryption and recovers the session key accordingly. Both parties can now start using the session key to transparently encrypt and decrypt data transmitted over the connection. The server sends an encrypted confirmation to the client. Receipt of this confirmation tells the client that the server has been able to successfully decrypt the session key, and must therefore hold its private keys. From that moment on, the client assumes the server to be authentic and transport layer encryption and integrity protection to be in proper use.

In certain cases, user authentication may also be required. The corresponding message exchange is initiated by the client who sends an authentication request to the server. The request declares the user name to log in. Depending on the authentication method in use, the dialogue that takes place between the client and the server may look different. There are two authentication methods supported in SSH V1:

In the case of password authentication, the user password is transmitted over the communication channel in transparently encrypted form.
In the case of RSA authentication, the server challenges the client with a random number that is encrypted with the public key of the user. In this case, the server must also have access to a database of manually distributed and preconfigured public keys for registered users. The client can only decrypt the challenge if it knows the private key of the user. It therefore requests a passphrase that is needed to temporarily unlock the user's private key. To authenticate to the server, the client must respond with a correct MD5 hash value of the decrypted challenge and some additional data that binds the result to the current session.

In either case, the server must respond with an authentication success or failure. If client authentication is not required, or if the client has been able to successfully authenticate to the server, it can now request a service. In particular, it can securely log in to a remote system (using the slogin command), remotely execute commands (using the ssh command), transfer files (using the scp command), and so on. Session keys can also be reexchanged dynamically.

SSH V2 is being standardized by the participants of the IETF SECSH WG. In a first step, the protocol used in SSH version 1 was divided into three subprotocols:

The SSH transport layer protocol (i.e., SSH-TRANS);
The SSH authentication protocol (i.e., SSH-USERAUTH);
The SSH connection protocol (i.e., SSH-CONN).

All of these subprotocols are being specified in separate Internet-Drafts. A fourth document specified in an Internet-Draft describes the overall SSH protocol architecture (i.e., SSH-ARCH).

SSH V1 and SSH V2 differ considerably. For example, in SSH V2 all algorithms can be negotiated between the client and the server. Also, SSH V2 uses a Diffie-Hellman key exchange to provide PFS (instead of encrypting the session key with an additonal server key). Furthermore, SSH V2 is designed to provide support for public key certificates. Most importantly, SSH V1's weak CRC-32 integrity check has been replaced in SSH V2 with a stronger MAC construction.^[7]

In summary, SSH provides a well established and widely used and deployed alternative for Telnet and the Berkeley r-tools (e.g., rlogin, rcp, rsh, …). As such, its use is highly recommended and server systems should be remotely administered only using SSH. The major disadvantage of SSH in the past was the lack of certificate-based key management, meaning that public keys had to be manually configured on both the client and server side. This disadvantage has been resolved and the latest releases of SSH are also able to handle public key certificates (either X.509 or PGP certificates).

16.1.2 File Transfer

The File Transfer Protocol (FTP) [12] is the protocol of choice for transferring files in TCP/IP-based networks. We briefly touched upon FTP when we discussed the uncompleteness of static packet filtering techniques in Part II (in fact, we used the FTP to motivate to use dynamic packet filtering techniques).

Many things that have been said for secure terminal access also apply for secure file transfer:

It is possible to use an authentication and key distribution system, such as Kerberos or SPX, to secure the transfer of files. In this case, however, the corresponding software on either side (i.e., the client and server software) must be modified to make use of the authentication and key distribution system and the corresponding API.
It is possible to layer FTP on top of SSL/TLS. The corresponding protocol has been acronymed FTPS and the IANA has assigned the two port numbers 989 (for the FTPS data connection) and 990 (for the FTPS control connection) to it (instead of 20 and 21 as in the case of the FTP data and control connections). For example, SSLftp implements FTPS.
SRA and SSH can be used to securely transfer files in TCP/IP-based networks. For example, the SSH utility scp can be used to securely transfer (i.e., copy) files from one system to another, and SSH V2 provides an sftp utility that implements FTP on top of the SSH V2 transport layer security protocol.

It would be very important to use any of these possibilities to securely transfer files in TCP/IP-based networks. Unfortunately, this is seldom done and file transfers often occur without proper protection put in place.

16.1.3 Electronic Mail

There are two RFC documents that collectively define the mode of operation of an Internet-based electronic mail (e-mail) system:

RFC 822 (STD 11) specifies the formats and syntactical rules for text-based e-mail messages [13].
RFC 821 (part of STD 10) specifies the SMTP used to send and receive e-mail messages by user agents and message transfer agents [14].

In addition, there are several protocols that either enhance the capabilities of SMTP or provide a possibility to access a user's message store (e.g., POP3 and IMAP4). All of these protocols are fully described in [15, 16] and summarized in Chapter 3 of [17].

Again, there are many possibilities to secure communications in an e-mail system. For example, it is possible to layer the various protocols on top of SSL/TLS. SMTP on top of SSL/TLS combine to form the acronym SMTPS and has a port number of 465. Similarly, POP3 (IMAP4) on top of SSL/TLS has the acronym POP3S (IMAPS) and has a port number of 995 (993). Also, it is possible to use the Kerberos authentication system to have users access their message stores in an authenticated way. For example, the Eudora user agent software provides support for Kerberos-based authentication.

All the possibilities mentioned above target the e-mail system. In addition, there is always the possibility to transform and secure the messages that are sent or received in an e-mail system in a way that is transparent to both the e-mail system and the users of the system. This possibility requires the use of a message security scheme, and we further elaborate on such schemes in Chapter 17.

16.1.4 WWW Transactions

The continuing growth and popularity of the Internet is mainly driven by the HTTP and the WWW. As further explored in [18], there are many technologies that can be used to secure the WWW and corresponding Web transactions. For example, the Digest authentication method has been added to the HTTP to provide a mechanism for strong user authentication [19]. Also, the SSL and TLS protocols are extensively used on the WWW to provide transaction security. As we saw in Chapter 15, HTTP layered on top of SSL/TLS has the acronym HTTPS and has a port number of 443 from the IANA.

In addition to the SSL and TLS protocols and the work that is being done within the IETF TLS WG, the IETF Web Transaction Security (WTS) WG^[8] was chartered in 1995. The goal of the WTS WG was (and still is) to "develop requirements and a specification for the provision of security services to Web transaction." The starting point was the specification of the Secure Hypertext Transfer Protocol (S-HTTP) that was developed and originally proposed by Eric Rescorla and Allan Schiffman on behalf of the CommerceNet consortium in the early 1990s.^[9] S-HTTP version 1.0 was publicly released in June 1994 and distributed by the CommerceNet consortium. Since 1995, the S-HTTP specification has been further refined under the auspices of the IETF WTS WG. In August 1999, the S-HTTP was specified and released in an experimental RFC document [20]. In addition, the IETF WTS WG released an informational RFC document specifying considerations for Web transaction security in January 1997 [21]. Because S-HTTP is not used and deployed on the Internet, we are not going to delve into the technical details of the corresponding protocol specification. You may refer to the first edition of this book or the RFC document mentioned above to obtain the details.

Finally, there are some complementary proposals to secure WWW transactions. For example, the use of a general-purpose common client interface (CCI) to enhance the security of WWW transactions with PGP messaging was proposed in [22]. The CCI was an early attempt at extending the functionality of WWW browsers. Now largely abandoned, CCI was an experimental protocol that allowed some versions of the NCSA Mosaic WWW browser to be controlled by an HTTP server. Today, many of the more useful functions of CCI are also present in programming and scripting languages, such as Java and JavaScript. The PGP-CCI application was basically a link between a WWW browser and the functionality provided by the PGP software.

16.1.5 Domain Name System

As mentioned in Chapter 4, the DNS has specific vulnerabilities that may be exploited in DNS spoofing attacks. Against this background, the IETF has worked out a couple of security extensions for the DNS, and these security extensions are commonly referred to as DNS Security, or DNSSEC. The extensions are specified in standards track RFC 2535 [23] and are currently being implemented and deployed.

In short, DNSSEC uses public key cryptography and digital signatures to provide data integrity and authentication for the information contained within the DNS and transferred with the DNS protocol. To achieve this goal, DNSSEC incorporates some new RR types:

The KEY RR for public keys;
The CERT RR for public key certificates;
The SIG RR for digital signatures;
The NXT RR for the nonexistence of specific data.

As such, digital signatures can be used to secure DNS zone data, dynamic updates, and transactions, as well as to prove the nonexistence of DNS data. Furthermore, DNSSEC provides an option to use secret key cryptography rather than public key cryptography to secure DNS transactions. The digital signatures and the public keys used for signature verification are retrieved through queries and responses, just like any other piece of information within the DNS.

What DNSSEC does not provide is access control and privacy. This is important to distinguish from actual implementations, which may or may not choose to include additional mechanisms for access control and privacy. The reason why access control and privacy have been omitted is that the information contained with the original DNS protocol is designed to be public data. With the advent of firewalls, concerns about information leakage of system names and locations, and DoS attacks, there is an increasing desire to have both access control and privacy. This demand is being reflected in the implementation of DNS. For example, the BIND implementation provided access control to prevent unauthorized systems from performing zone transfers. This has also been extended to prevent certain systems from querying servers. To date, privacy has been partially achieved by using firewalls and having what is known as a split DNS configuration, in which the internal DNS information is difficult to access from the Internet.

DNSSEC has many security advantages. Any organization that relies on the Internet should consider DNSSEC a critical component of its security infrastructure because the DNS protocol is vulnerable and can be misused in spoofing attacks. Only DNSSEC, through its cryptographic techniques, can provide data integrity and authentication to all aspects of the DNS. Consequently, it is possible and very likely that DNSSEC will become more important in the future. DNSSEC, however, also has its disadvantages. For example, signing and verifying DNS data create some additional overhead that will negatively affect network and service performance. Furthermore, some operational aspects of DNSSEC are still being worked out, such as exactly how and who will sign public keys and how public key certificates can be revoked. An obvious solution is to establish a hierarchical PKI that matches the structure of the DNS tree. In this case, the PKI has a globally unique root CA.

16.1.6 Distributed File Systems

Another application that is often enhanced with security features is a distributed file system. Note that security professionals may feel well protected with strong encryption mechanisms, such as those provided by IPsec or SSL/TLS. But what good is transferring encrypted data over a network connection if the data are stored unencrypted on local machines that may get compromised? Thus, instead of having each application handle encryption and decryption, it is also possible to enhance a file system so that files are automatically encrypted after they leave applications but before they are sent to the file system. This approach has several advantages with regard to transparency and efficiency. There are at least two file systems that follow this approach:

Matt Blaze from AT&T Bell Laboratories developed a Cryptographic File System (CFS) that is heavily used on UNIX systems [24, 25].
Another example is the Andrew File System (AFS) that was originally designed and developed at Carnegie Mellon University and later marketed by the Transarc Corporation^[10] [26, 27]. More recently, Transarc Corporation was acquired by IBM and has become the IBM Transarc Lab. In principle, the AFS refers to a Kerberized file system; it uses Kerberos for authentication and optional encryption, and is designed to work across WANs. Consequently, AFS provides a considerably higher level of security than does the Network File System (NFS) more commonly used today. NFS is shipped as part of the operating system with most versions of UNIX, while AFS is a commercial third-party product. Because Kerberos and AFS require significant technical expertise to set up and maintain, AFS is not widely used outside of a relatively small number of sites. If WAN file systems must be secured, it may be worth investigating AFS. There have been security problems with some earlier versions of AFS, but those have since been fixed [28].

Given the fact that the CFS is restricted to UNIX systems and the AFS requires the use of the Kerberos authentication system (which is used mainly in UNIX environments), there is not much choice for Windows users to transparently encrypt and decrypt their file systems. In Windows 2000, however, there is at least a possibility to transparently encrypt and decrypt NTFS files. This possibility has been named Encrypting File System (EFS). If a user activates encryption for a file, the EFS selects a random file encryption key and encrypts the file with this key. Furthermore, the file encryption key is encrypted with the public key of the user and the encrypted file encryption key is stored together with the encrypted file on disk. Consequently, file decryption always requires the decryption of the file encryption key, and the decryption of the file encryption key always requires the user's private key. For key recovery purposes it is possible to encrypt the file encryption key with an additional public key.

^[1]Refer to footnote 6 for an explanation of why SSH is addressed in this chapter.

^[2]http://www.ssh.com

^[3]http://www.ietf.org/html.charters/secsh-charter.html

^[4]http://www.openssh.com

^[5]http://www.openbsd.org

^[6]Because of the existence of this generic transport layer security protocol, SSH was discussed in the chapter entitled "Transport Layer Security Protocols" in the first edition of this book. In fact, SSH provides similar functionality like SSL and TLS, and should therefore be covered in the same chapter. Mainly because SSH is primarily used as a replacement for Telnet and Rlogin, however, it is being discussed in this chapter.

^[7]The weak CRC-32 integrity check was exploited in an SSH insertion attack in 1998. You may refer to http://www.core-sdi.com/advisories/ssh-advisory.htm for an overview about the attack.

^[8]http://www.ietf.org/html.charters/wts-charter.html

^[9]Launched in 1994 as a nonprofit organization, CommerceNet is dedicated to advancing electronic commerce on the Internet. Its almost 600 member companies and organizations seek solutions to technology issues, sponsor industry pilots, and foster market and business development. The CommerceNet consortium is available on-line at http://www.commerce.net.

^[10]http://www.transarc.com

Team-Fly