7.16 Representing Public Keys and Certificates in Binary (DER Encoding)

7.16.1 Problem

You want to represent a digital certificate or some other cryptographic primitive in a standard binary format, either for signing or for storing to disk.

7.16.2 Solution

There is an industry-standard way to represent cryptographic objects in binary, but it isn't very pretty at all. (You need to use this standard if you want to programmatically sign an X.509 certificate in a portable way.) We strongly recommend sticking to standard APIs for encoding and decoding instead of writing your own encoding and decoding routines.

When storing data on disk, you may want to use a password to encrypt the DER-encoded representation, as discussed in Recipe 4.10.

7.16.3 Discussion

ASN.1 is a language for specifying the fields a data object must contain. It's similar in purpose to XML (which it predates). Cryptographers use ASN.1 extensively for defining precise descriptions of data. For example, the definition of X.509 certificates is specified in the language. If you look at that specification, you can clearly see which parts of the certificate are optional and which are required, and see important properties of all of the fields.

ASN.1 is supposed to be a high-level specification of data. By that, we mean that there could be a large number of ways to translate ASN.1 data objects into a binary representation. That is, data may be represented however you want it to be internal to your applications, but if you want to exchange data in a standard way, you need to be able to go back and forth from your internal representation to some sort of standard representation. An ASN.1 representation can be encoded in many ways, though!

The cryptographic community uses distinguished encoding rules (DER) to specify how to map an ASN.1 specification of a data object to a binary representation. That is, if you look at the ASN.1 specification of an X.509 certificate, and you have all the data ready to go into the certificate, you can use DER and the ASN.1 specification to encode the data into an interoperable binary representation.

ASN.1 specifications of data objects can be quite complex. In particular, the specification for X.509v3 is vast because X.509v3 is a highly versatile certificate format. If you plan on reading and writing DER-encoded data on your own instead of using a cryptographic library, we recommend using an ASN.1 "compiler" that can take an ASN.1 specification as input and produce C data structures and routines that encode and parse data in a DER-encoded format. The Enhanced SNACC ASN.1 compiler is available under the GNU GPL from http://www.getronicsgov.com/hot/snacc_lib.htm.

If you need to do sophisticated work with certificates, you may want to look at the freeware Certificate Management Library, available from http://www.getronicsgov.com/hot/cml_home.htm. It handles most operations you can perform on X.509 certificates, including retrieving certificates from LDAP databases.

Here, we'll show you the OpenSSL APIs for DER-encoding data objects and for converting binary data into OpenSSL data types. All of the functions in the OpenSSL API either convert OpenSSL's internal representation to a DER representation (the i2d functions) or convert DER into the internal representation (the d2i functions).

The basic i2d functions output to memory and take two arguments: the object to convert to DER and a buffer into which to write the result. The second argument is a pointer to a buffer of unsigned characters, represented as unsigned char **. That is, if you are outputting into an unsigned char *x, where x doesn't actually hold the string, but holds the address in memory where that string starts, you need to pass in the address of x.

OpenSSL requires you to pass in a pointer to a pointer because it takes your actual pointer and "advances" it. We don't like this feature and have never found it useful. In general, you should copy over the pointer to your buffer into a temporary variable, then send in the address of the temporary variable.

Note that you need to know how big a buffer to pass in as the second parameter. To figure that out, call the function with a NULL value as the second argument. That causes the function to calculate and return the size.

For example, here's how to DER-encode an RSA public key:

#include <openssl/rsa.h>     /* Returns the malloc'd buffer, and puts the size of the buffer into the integer  * pointed to by the second argument.  */ unsigned char *DER_encode_RSA_public(RSA *rsa, int *len) {   unsigned char *buf, *next;       *len = i2d_RSAPublicKey(rsa, 0);   if (!(buf = next = (unsigned char *)malloc(*len))) return 0;   i2d_RSAPublicKey(rsa, &next); /* If we use buf here, return buf; becomes wrong */   return buf; }

For each basic function in the i2d API, there are two additional functions implemented as macros that output to a FILE object or an OpenSSL BIO object, which is the library's generic IO abstraction.^[5] The name of the base function is suffixed with _fp or _bio as appropriate, and the second argument changes to a FILE or a BIO pointer as appropriate.

^[5] There are three exceptions to this rule, having to do with the OpenSSL EVP interface. We don't discuss (or even list) the functions here, because we don't cover the OpenSSL EVP interface (it's not a very good abstraction of anything in our opinion). If you do want to look at this interface, it's covered in the book Network Security with OpenSSL.

The d2i API converts DER-encoded data to an internal OpenSSL representation. The functions in this API take three arguments. The first is a pointer to a pointer to the appropriate OpenSSL object (for example, an RSA ** instead of the expected RSA *). The second is a pointer to a pointer to the buffer storing the representation (i.e., a char ** instead of a char *). The third is the input length of the buffer (a long int). The first two arguments are pointers to pointers because OpenSSL "advances" your pointer just as it does in the i2d API.

The return value is a pointer to the object written. However, if the object cannot be decoded successfully (i.e., if there's an error in the encoded data stream), a NULL value will be returned. The first argument may be a NULL value, in which case an object of the appropriate type is allocated and returned.

Here's an example of converting an RSA public key from DER format to OpenSSL's internal representation:

#include <openssl/rsa.h>     /* Note that the pointer to the buffer gets copied in.  Therefore, when  * d2i_... changes its value, those changes aren't reflected in the caller's copy  * of the pointer.  */ RSA *DER_decode_RSA_public(unsigned char *buf, long len) {   return d2i_RSAPublicKey(0, &buf, len); }

As with the i2d interface, all of the functions have macros that allow you to pass in a FILE or an OpenSSL BIO object, this time so that you may use one as the input source. Those macros take only two arguments, where the base function takes three. The first argument is the BIO or FILE pointer from which to read. The second argument is a pointer to a pointer to the output object (for example, an RSA **). Again, you can pass in a NULL value for this argument. The len argument is omitted; the library figures it out for itself. It could have figured it out for itself in the base API, but it requires you to pass in the length so that it may ensure that it doesn't read or write past the bounds of your buffer.

Table 7-3 lists the most prominent things you can convert to DER and back. The last two rows enumerate calls that are intended for people implementing actual infrastructure for a PKI, and they will not generally be of interest to the average developer applying cryptography.^[6]

^[6] However, PKCS #7 can be used to store multiple certificates in one data object, which may be appealing to some, instead of DER-encoding multiple X.509 objects separately.

Table 7-3. Objects that can be converted to and from DER format

Kind of object	OpenSSL object type	Base encoding function	Base decoding function	Header File
RSA public key	RSA	i2d_RSAPublicKey()	d2i_RSAPublicKey()	openssl/rsa.h
RSA private key	RSA	i2d_RSAPrivateKey()	d2i_RSAPrivateKey()	openssl/rsa.h
Diffie-Hellman parameters	DH	i2d_DHparams()	d2i_DHparams()	openssl/dh.h
DSA parameters	DSA	i2d_DSAparams()	d2i_DSAparams()	openssl/dsa.h
DSA public key	DSA	i2d_DSAPublicKey()	d2i_DSAPublicKey()	openssl/dsa.h
DSA private key	DSA	i2d_DSAPrivateKey()	d2i_DSAPrivateKey()	openssl/dsa.h
X.509 certificate	X509	i2d_X509()	d2i_X509()	openssl/x509.h
X.509 CRL	X509_CRL	i2d_X509_CRL()	d2i_X509_CRL()	openssl/x509.h
PKCS #10 certificate signing request	X509_REQ	i2d_X509_REQ()	d2i_X509_REQ()	openssl/x509.h
PKCS #7 container	PKCS7	i2d_PCKS7()	d2i_PKCS7()	openssl/x509.h

7.16.4 See Also

• Enhanced SNACC ASN.1 compiler: http://www.getronicsgov.com/hot/snacc_lib.htm

• Certificate Management Library: http://www.getronicsgov.com/hot/cml_home.htm

• Recipe 4.10