17.3 DOS Attributes, Extended File Attributes, Long Filenames, and Suchlike

These all present the same problem.

The CIFS protocol suite is designed, in its heart and soul, to work with DOS, OS/2, and Windows systems. As a result, the protocols that make up the CIFS suite have a tendency to reflect the behavior of those operating systems.

DOS, of course, is the oldest and simplest of the IBM/Microsoft family of PC OSes. The filesystem used with DOS is the venerable F ile A llocation T able (FAT) filesystem which, according to legend, was originally coded up by Bill Gates himself. The characteristics of the FAT filesystem should be familiar to anyone who has spent any time working with DOS. Consider, for example, the following FAT features:

Case-insensitive filenames

Case is ignored, though file and directory names are stored in uppercase.

8.3 filename format

A name is a maximum of eight bytes in length, optionally followed by a dot ('.') and an extension of at most three bytes. For example: FILENAME.EXT .

No users or groups

FAT does not understand ownership of files.

Six attribute bits

FAT supports six attribute bits, stored in a single byte. The best known are the Archive, Hidden, Read-only, and System bits but there are two more: Volume Label and Directory. These are used to identify the type and handling of a FAT table entry.

It's a fairly spartan system.

There are improvements and extensions that have appeared over the years . The FAT32 filesystem, for example, is a modified version of FAT that uses disk space more efficiently and also supports much larger disk sizes than the original FAT. There is also VFAT, which keeps track of both 8.3 format filenames and longer secondary filenames that may contain a wider variety of characters than the 8.3 format allows. VFAT long filenames are case- preserving (but not case sensitive) so, overall, VFAT allows a lot more creativity with file and directory names. ^[1]

^[1] Digging through the documentation, it appears that the FAT family consists of FAT12, FAT16, FAT32, and VFAT. There is documentation on the web that provides implementation details, if you are so inclined.

Even with these extensions, the semantics of the FAT filesystem are not sufficient to meet the needs of more powerful OSes such as OS/2 and Windows NT. These OSes have newer , more complex filesystems which they support in addition to FAT. Specifically, OS/2 has HPFS ( H igh P erformance F ile S ystem), and Windows NT and W2K can make use of NTFS ( N ew T echnology F ile S ystem). These newer filesystems have lots and lots of features which, in turn , have to be supported by CIFS.

Problems arise when the server semantics (made available via CIFS) do not match those expected by the client. Consider, for instance, Samba running on a Unix system. Unix filesystems typically have these general characteristics:

Case-sensitive filenames

Case is significant in most Unix filesystems. File and directory names are stored with case preserved.

Longer, more complex filenames

Unix filesystems allow for a great deal of creativity in naming files and directories.

Users and groups

Unix filesystems assign user and group ownership to each directory entry.

More, and different, attribute bits

There are three sets of three bits each used for basic file access (read, write, and execute permissions for user, group, and world). There are additional bits defined for more esoteric purposes.

Now consider a Windows application that requires the old 8.3 name format. (Such applications do exist. They make calls to older, 16-bit OS functions that assume 8.3 format.) Unlike VFAT, Unix filesystems do not normally keep track of both long and short names. That causes a problem, and Samba has to compensate by generating 8.3 format names on the fly. The process is called "Name Mangling."

There are other gotchas too. Indeed, name mangling is just the tip of the proverbial iceberg.

One solution that some CIFS vendors have been able to implement is to develop a whole new filesystem for their server platform, one that maintains all of the required attributes and maps between them as necessary. This is a pain, but it works in situations in which the server vendor has control over the deployment of their product. One such filesystem is Microsoft's NTFS, which can handle a very wide variety of attributes and map them to the semantics required by Apple Macintosh clients, Unix clients, DOS clients, OS/2 clients ...

You've got the basic idea. Let's run through some of the trouble spots to give you a sense of what you're up against.

Long filenames

Long filenames can be much more descriptive than the old 8.3 names. The problem, of course, is that CIFS must support both long and short (8.3) names to be fully compatible with all of the potential clients out there. Even if a server supports only the NT LM 0.12 dialect , there will still be instances when the 8.3 format is required. Sigh.

DOS attributes

These are the six attribute bits that are supported by the FAT filesystem. These do not map well to the file protections offered by other filesystems. Compare these, for example, against the attribute bits offered by Unix systems.

The timestamps stored in the FAT filesystem may also be different from those used by other systems.

Extended File Attributes

These are an extended set of attribute bits and flags available on systems using the NTFS filesystem. They are a 32-bit superset of the set offered by the FAT system. Extended File Attributes are described in Section 3.13 of the SNIA doc.

The term "Extended File Attributes" is also sometimes misused when discussing NTFS permissions. Permissions are different; they are associated with A ccess C ontrol E ntries (ACEs), and ACEs are gathered together into A ccess C ontrol L ists. There's a whole bigbunch of stuff there that could be explored and would be, if this were a book about implementing NTFS.

Extended Attributes

These should get special mention because, it seems, CIFS is sufficiently complex that terminology has to be recycled. E xtended A ttributes (EAs) are not the same as Extended File Attributes.

EAs are a feature of HPFS and, therefore, are supported by NTFS. Basically, they are a separate data space associated with a file into which applications may store additional data or metadata specific to the application (things like author name or a file comment). ^[2]

^[2] NTFS is a complex filesystem based on some simple concepts. One such concept is that each "file" is actually a set of "attributes" (records). Many of these attributes are predefined to contain such things as the short name, the long name, file creation and access times, etc. The actual content of the file is stored in a specific, predefined "stream," where a stream is a particular kind of attribute. NTFS supports OS/2-style Extended Attributes in another type of NTFS attribute... and it just gets more confusing from there. There is a lot of documentation on the web about the workings of NTFS, and there is a project aimed at implementing NTFS for Linux.

CIFS offers facilities to support all of these features and more. That's good news if you are writing client code, because you can pick and choose the sets of attributes you want to support. It's bad for server systems, which may need to offer various levels of compatibility in order to contend with client expectations.