9.8 Binary Files

Binary files are even more system-dependent than text files. Native I/O system calls for a given operating system read or write binary data in multiples of the physical block size of the storage device. The programmer may set aside buffers in memory of the same or multiple larger sizes.

Not all programming languages, operating systems, and processor architectures store binary data the same way. Conventions are somewhat arbitrary choices: Are arrays stored row-by-row or column-by-column? Does the system use big-endian or little-endian representations?

The data in a binary file may be unstructured (i.e., just a byte stream) or may have some internal organization. Examples of structured data include indexed files and "direct-access" files containing arrays of floating-point data ordered according to the conventions of a specific high-level language (e.g., FORTRAN).

Prior to the development of relational databases, some operating systems offered file types with a considerable measure of internal organization. IBM defined the ISAM file format (indexed sequential access method), where an index field allows virtual "sequential" access in the indexed order.

The major concept for many file types is that access is possible to individual records in the file i.e., to aggregate information units embodying the programmer's data structures typically through function calls. Highly specialized file structures, such as a relational database using several indexed lookup fields, are usually handled by middleware. Function calls from an application program go to such layers, which in turn depend on standard operating system capabilities only for providing access to a raw "container" file inside which the database stores both data and lookup index information.