Section 3.1. The POSIX Standard

3.1. The POSIX Standard

POSIX was named (like many things in the Unix software world) by Richard Stallman. It stands for Portable Operating System Interface-X, meaning a portable definition of a Unix-like operating system API. The reason for the existence of the POSIX standard is interesting and lies in the history of the Unix family of operating systems.

As is commonly known, Unix was created in 1969 at AT&T Bell Labs by Ken Thompson and Dennis Richie. Not originally designed for commercialization, the source code was shipped to universities around the world, most notably Berkeley in California. One of the world's first truly portable operating systems, Unix soon splintered into many different versions as people modified the source code to meet their own requirements. Once companies like Sun Microsystems and the original, prelitigious SCO (Santa Cruz Organization) began to commercialize Unix, the original Unix system call API remained the core of the Unix system, but each company added proprietary extensions to differentiate their own version of Unix. Thus began the first of the "Unix wars" (I'm a veteran, but I don't get disability benefits for the scars they caused). For independent software vendors (ISVs), such proprietary variants were a nightmare. You couldn't assume that code that ran correctly on one Unix would even compile on another.

During the late 1980s, in an attempt to create a common API for all Unix systems, and fix this problem, the POSIX set of standards was born. Because no one trusted any of the Unix vendors, the Institute of Electrical and Electronics Engineers (IEEE) shepherded the standards process and created the 1003 series of standards, known as POSIX. The POSIX standards cover much more than the operating system APIs, going into detail on system commands, shell scripting, and many other parts of what it means to be a Unix system. I'm only going to discuss the programming API standard part of POSIX here because, as a programmer, that's really the only part of it I care about on a day-to-day basis.

Few people have actually seen an official POSIX standard document, as the IEEE charges money for copies. Back before the Web became really popular, I bought one just to take a look at the real thing. It wasn't cheap (a few hundred dollars, as I recall). Amusingly enough, I don't think Linus Torvalds ever read or referred to it when he was creating Linux; he used other vendors' references to it and manpage descriptions of what POSIX calls were supposed to do.

Reading the POSIX standard document, however, is very interesting. It reads like a legal document; every line of every section is numbered so that it can be referred to in other parts of the text. It's detailed. Really detailed. The reason for such detail is that it was designed to be a complete specification of how a Unix system has to behave when called from an application program. The secret is that it was meant to allow someone reading the specification to completely reimplement their own version of a Unix operating system starting from scratch, with nothing more than the POSIX spec. The goal is that if someone writes an application that conforms to the POSIX specification, the resulting application can be compiled with no changes on any system that is POSIX compliant. There is even a POSIX conformance suite, which allows a system passing the tests to be officially branded a POSIX-compliant system. This was created to reduce costs in government and business procurement procedures. The idea was that you specified "POSIX compliant" in your software purchasing requests, the cheapest system that had the branding could be selected, and it would satisfy the system requirement.

This ended up being less useful than it sounds, given that Microsoft Windows NT has been branded POSIX compliant and generic Linux has not.

Sounds wonderful, right? Unfortunately, reality intruded its ugly head somewhere along the way. Vendors didn't want to give up their proprietary advantages, so each pushed to get its particular implementation of a feature into POSIX. As all vendors don't have implementations of all parts of the standard, this means that many of the features in POSIX are optionalusually just the one you need for your application. How can you tell if an implementation of POSIX has the feature you need? If you're lucky, you can test for it at compile time.

The GNU project suffered from these "optional features" more than most proprietary software vendors because the GNU software is intended to be portable across as many systems as possible. To make their software portable across all the weird and wonderful POSIX variants, the wonderful suite of programs known as GNU autoconf was created. The GNU autoconf system allows you to test to see whether a feature exists or works correctly before you even compile the code, thus allowing an application programmer to degrade missing functionality gracefully (i.e., not fail at runtime).

Unfortunately, not all features can be tested this way, as sometimes a standard can give too much flexibility, thus causing massive runtime headaches. One of the most instructive examples is in the pathconf( ) call. The function prototype for pathconf( ) looks like this :

long pathconf(char *path, int name);

Here, char *path is a pathname on the system and int name is a defined constant giving a configuration option you want to query. The constants causing problems are:

_PC_NAME_MAX _PC_PATH_MAX

_PC_NAME_MAX queries for the maximum number of characters that can be used in a filename in a particular directory (specified by char *path) on the system. _PC_PATH_MAX queries for the maximum number of characters that can be used in a relative path from the particular directory. This seems fine until you consider how Unix filesystems are structured and put together. A typical Unix filesystem looks like Figure 3-1.

Figure 3-1. Typical Unix filesystem

Any of the directory nodes, such as /usr/bin or /mnt, could be a different filesystem type, not the standard Unix filesystem (maybe even network mounted). In Figure 3-1, the /mnt/msdos_dir path has been mounted from a partition containing an old MS-DOS-style FAT filesystem type. The maximum directory entry length on such a system is the old DOS 8.3 maximum of 11 characters. But below the Windows directory could be mounted a different filesystem type with different maximum name restrictions maybe an NFS mount from a different machine, for example, on the path /mnt/msdos_dir/nfs_dir. Now the pathconf() can accommodate these restrictions and tell your application about itif you remember to call it on every single possible path and path component your application might use! Hands up, all application programmers who actually do this....Yes, I thought so. (You at the back, put your hand down. I know how you do things in the U.S. Star Wars missile defense program, but no one programs in ADA anymore, plus your tests never work, OK?) This is an example of something that looks good on paper but in practical terms almost no one would use in an actual application. I know we don't in Samba, not even in the "rewritten from scratch with correctness in mind" Samba4 implementation.

Now let's look at an example of where POSIX gets it spectacularly wrong, and why this happens.