Section 2.6. The Runtime Architecture

2.6. The Runtime Architecture

Given a loose enough definition of a runtime environment, one can say that modern operating systems often provide multiple runtime environments. For example, whereas the Java virtual machine on Mac OS X is a runtime environment for Java programs, the virtual machine implementation itself executes in another, "more native" runtime environment. Mac OS X has several runtime environments for applications, as we will see later in this chapter. However, an operating system typically has only a single lowest-level (or "native") runtime environment that we will refer to as the runtime environment. The foundation of the runtime environment is the runtime architecture, which has the following key aspects.

• It provides facilities for launching and executing programs.

• It specifies how code and data reside on diskthat is, it specifies the binary format. It also specifies how compilers and related tools must generate code and data.

• It specifies how code and data are loaded into memory.

• It specifies how references to external libraries are resolved.

Mac OS X has only one runtime architecture: Mach-O. The name refers to the Mach Object File Format, although the term "Mach" is somewhat of a misnomer in this case since Mach is not meant to understand any object file format. Neither is Mach aware of the runtime conventions of user-space programs. The Mac OS X kernel, however, does understand the Mach-O format. In fact, Mach-O is the only binary format that the kernel can load^[20]using the execve()^[21] system call, which is implemented in the BSD portion of the Mac OS X kernel.

^[20] Note that we explicitly say binary format: The kernel can arrange for scripts to run.

^[21] The execve() system call executes the specified program, which may be a binary executable or a script, in the address space of the calling process.

2.6.1. Mach-O Files

Mac OS X uses Mach-O files for implementing several types of system components, for example, the following:

• Bundles (programmatically loadable code)

• Dynamic shared libraries

• Frameworks

• Umbrella frameworks, which contain one or more other frameworks

• Kernel extensions

• Linkable object files

• Static archives

• Executables

We will discuss frameworks, umbrella frameworks, and bundles later in this chapter. Before we continue, let us enumerate some programs that are useful in creating, analyzing, or manipulating Mach-O files. Such programs include the following:

• as the GNU-based assembler front-end

• dyld the default dynamic link editor (or runtime linker)

• gcc, g++ GNU compiler front-ends

• ld the static link editor (or static linker^[22])

  ^[22] Note that "static" in static linker refers to the fact that the program operates at compile timeand not dynamically at runtime. The static linker supports both dynamic shared libraries and static archive libraries.

• libtool a program that creates dynamically linked shared libraries and statically linked libraries from Mach-O object files; called by the compiler driver during library creation

• nm a program that displays the object file symbol table

• otool a versatile program for displaying the internals of Mach-O files; has disassembling capabilities

A Mach-O file contains a fixed-size header (see Figure 23) at the very beginning, followed by typically several variable-sized load commands, followed by one or more segments. Each segment can contain one or more sections.

Figure 23. The structure of the Mach-O header (32-bit version)

struct mach_header {     uint32_t      magic;      /* mach magic number identifier */     cpu_type_t    cputype;    /* cpu specifier */     cpu_subtype_t cpusubtype; /* machine specifier */     uint32_t      filetype;   /* type of file */     uint32_t      ncmds;      /* number of load commands */     uint32_t      sizeofcmds; /* the size of all the load commands */     uint32_t      flags;      /* flags */ };

The Mach-O header describes the features, layout, and linking characteristics of the file. The filetype field in the Mach-O header indicates the type and, therefore, the purpose of the file. Mach-O file types include the following:

• MH_BUNDLE plug-in code that is programmatically loaded into applications at runtime

• MH_CORE a file that stores the address space of an aborted program, that is, a core file containing a "core dump"

• MH_DYLIB a dynamic shared library; conventionally a file with a .dylib suffix if it is a stand-alone library

• MH_DYLINKER a special shared library that is a dynamic linker

• MH_EXECUTE a standard demand-paged executable

• MH_OBJECT an intermediate, relocatable object file (conventionally with a .o suffix); also used for kernel extensions

For executable files, one of the load commands (LC_LOAD_DYLINKER) in the Mach-O header specifies the path to the linker to be used for loading the program. By default, this load command specifies the standard dynamic linker, dyld (/usr/lib/dyld), which itself is a Mach-O file of type MH_DYLINKER. The kernel and dyld (or in theory, another dynamic linker, if one is specified) together prepare a Mach-O binary for execution using the following sequence of operations, which has been simplified for brevity.^[23]

^[23] Further details of program execution by the kernel are discussed in Section 7.5.

• The kernel examines the executable's Mach-O header and determines its file type.

• The kernel interprets the load commands contained in the Mach-O header. For example, to handle LC_SEGMENT commands, it loads program segments into memory.

• The kernel handles the LC_LOAD_DYLINKER load command by loading the specified dynamic linker into memory.

• The kernel eventually executes the dynamic linker on the program file. Note that this is the first user-space code that runs in the program's address space. The arguments passed to the linker include the program file's Mach-O header, the argument count (argc), and the argument vector (argv).

• The dynamic linker interprets load commands from the Mach-O header. It loads the shared libraries that the program depends on, and it binds external references that are required to start executionthat is, it binds the Mach-O file's imported symbols to their definitions in a shared library or framework.

• The dynamic linker calls the entry point function specified by the LC_UNIXTHREAD (or LC_THREAD) load command, which contains the initial thread state of a program's main thread. This entry point is normally a language runtime environment function, which in turn calls the program's "main" function.

Let us look at the example of a trivial executable. Figure 24 shows a C program that is compiled to an executable called empty.

Figure 24. A trivial C program to be compiled to an "empty" executable

// empty.c int main(void) {     return 0; }

Figure 25 shows the use of the otool program to list the load commands contained in empty.

Figure 25. Displaying the load commands in an executable's Mach-O header

$ otool -l ./empty empty: Load command 0       cmd LC_SEGMENT   cmdsize 56   segname __PAGEZERO ... Load command 4           cmd LC_LOAD_DYLINKER       cmdsize 28          name /usr/lib/dyld (offset 12) Load command 5           cmd LC_LOAD_DYLIB ... Load command 10         cmd LC_UNIXTHREAD     cmdsize 176      flavor PPC_THREAD_STATE       count PPC_THREAD_STATE_COUNT     r0  0x00000000 r1  0x00000000 r2  0x00000000 r3  0x00000000 r4   0x00000000 ... ctr 0x00000000 mq  0x00000000 vrsave 0x00000000 srr0 0x000023cc srr1 0x00000000

The LC_UNIXTHREAD load command shown in Figure 25 contains the initial values of the program's registers. In particular, the srr0 PowerPC register^[24] contains the address of the entry point function0x23cc in this case. As we can verify by using the nm program, this address belongs to a function called start(). Consequently, empty begins execution in this function, which comes from the language runtime stub /usr/lib/crt1.o. The stub initializes the program's runtime environment state before calling the main() function. The compiler links in crt1.o during compilation.

^[24] Chapter 3 discusses the PowerPC architecture in detail.

Note that if the Mach-O file in Figure 25 were an x86 executable, its LC_UNIXTHREAD command would contain x86 register state. In particular, the eip register would contain the address of the start() function.

Depending on aspects such as the program being compiled, the programming language, the compiler, and the operating system, more than one such stub may be linked in during compilation. For example, bundles and dynamic shared libraries on Mac OS X are linked along with /usr/lib/bundle1.o and /usr/lib/dylib1.o, respectively.

2.6.2. Fat Binaries

We came across "fat" binaries in Chapter 1, when we looked at NEXTSTEP. Since NEXTSTEP ran on multiple platforms such as Motorola 68K, x86, HP PA-RISC, and SPARC, it was rather easy to come across multifat binaries.

Fat binaries first became useful on Mac OS X with the advent of 64-bit user address space support, since a fat binary could contain both 32-bit and 64-bit Mach-O executables of a program. Moreover, with Apple's transition to the x86 platform, fat binaries become still more important: Apple's Universal Binary format is simply another name for fat binaries. Figure 26 shows an example of creating a three-architecture fat binary on Mac OS X.^[25] The lipo command can be used to list the architecture types in a fat file. It is also possible to build a fat Darwin kernelone that contains the kernel executables for both the PowerPC and x86 architectures in a single file.

^[25] Apple's build of GCC 4.0.0 or higher is required to create fat binaries on Mac OS X.

Figure 26. Creating fat binaries

$ gcc -arch ppc -arch ppc64 -arch i386 -c hello.c $ file hello.o hello.o: Mach-O fat file with 3 architectures hello.o (for architecture ppc):   Mach-O object ppc hello.o (for architecture i386):  Mach-O object i386 hello.o (for architecture ppc64): Mach-O 64-bit object ppc64 $ lipo -detailed_info hello.o Fat header in: hello.o fat_magic 0xcafebabe nfat_arch 3 architecture ppc     cputype CPU_TYPE_POWERPC     cpusubtype CPU_SUBTYPE_POWERPC_ALL     offset 68     size 368     align 2^2 (4) architecture i386     cputype CPU_TYPE_I386     cpusubtype CPU_SUBTYPE_I386_ALL     offset 436     size 284     align 2^2 (4) architecture ppc64     cputype CPU_TYPE_POWERPC64     cpusubtype CPU_SUBTYPE_POWERPC_ALL     offset 720     size 416     align 2^3 (8)

Figure 27 shows the structure of a fat binary containing PowerPC and x86 executables. Note that a fat binary is essentially a wrappera simple archive that concatenates Mach-O files for multiple architectures. A fat binary begins with a fat header (struct fat_header) that contains a magic number followed by an integral value representing the number of architectures whose binaries reside in the fat binary. The fat header is followed by a sequence of fat architecture specifiers (struct fat_arch)one for each architecture contained in the fat binary. The fat_arch structure contains the offset into the fat binary at which the corresponding Mach-O file begins. It also includes the size of the Mach-O file, along with a power of 2 value that specifies the alignment of the offset. Given this information, it is straightforward for other programsincluding the kernelto locate the code for the desired architecture within a fat binary.

Figure 27. A Universal Binary containing PowerPC and x86 Mach-O executables

Note that although a platform's Mach-O file in a fat binary follows that architecture's byte ordering, the fat_header and fat_arch structures are always stored in the big-endian byte order.

2.6.3. Linking

Dynamic linking is the default on Mac OS Xall normal user-level executables are dynamically linked. In fact, Apple does not support static linking of user-space programs (Mac OS X does not come with a static C library). One reason for not supporting static linking is that the binary interface between the C library and the kernel is considered private. Consequently, system call trap instructions should not appear in normally compiled executables. Although you can statically link object files into a static archive library,^[26] the language runtime stub that would yield statically linked executables doesn't exist. Therefore, a statically linked user executable cannot be generated using the default tools.

^[26] Static archive libraries can be used for distributing code that is not desirable in a shared library but is otherwise usable while compiling multiple programs.

Mac OS X kernel extensions must be statically linked. However, kernel extensions are not Mach-O executables (MH_EXECUTE) but Mach-O object files (MH_OBJECT).

The otool command can be used to display the names and version numbers of the shared libraries used by an object file. For example, the following command determines the libraries that launchd depends on (launchd's library dependencies are interesting because it is the first program to execute in user spacetypically, such programs are statically linked on Unix systems):

$ otool -L /sbin/launchd # PowerPC version of Mac OS X /sbin/launchd:            /usr/lib/libbsm.dylib (...)            /usr/lib/libsm.dylib (...) $ otool -L /sbin/launchd # x86 version of Mac OS X /sbin/launchd:            /usr/lib/libsm.dylib (...)            /usr/lib/libgcc_s.1.dylib (...)            /usr/lib/libSystem.B.dylib (...)

Figure 28 shows examples of compiling a dynamic shared library, compiling a static archive library, and linking with the two libraries.

Figure 28. Compiling dynamic and static libraries

$ cat libhello.c #include <stdio.h> void hello(void) {     printf("Hello, World!\n"); } $ cat main.c extern void hello(void); int main(void) {     hello();     return 0; } $ gcc -c main.c libhello.c $ libtool -static -o libhello.a libhello.o $ gcc -o main.static main.o libhello.a $ otool -L main.static main.static:         /usr/lib/libmx.A.dylib (...)         /usr/lib/libSystem.B.dylib (...) $ gcc -dynamiclib -o libhello.dylib -install_name libhello.dylib libhello.o $ gcc -o main.dynamic main.o -L. -lhello $ otool -L main.dynamic main.dynamic:         libhello.dylib (...)         /usr/lib/libmx.A.dylib (...)         /usr/lib/libSystem.B.dylib (...)

When compiling a dynamic shared library, you can specify a custom initialization routine that will be called before any symbol is used from the library. Figure 29 shows an example.

Figure 29. Using a custom initialization routine in a dynamic shared library

$ cat libhello.c #include <stdio.h> void my_start(void) {     printf("my_start\n"); } void hello(void) {     printf("Hello, World!\n"); } $ cat main.c extern void hello(void); int main(void) {     hello();     return 0; } $ gcc -c main.c libhello.c $ gcc -dynamiclib -o libhello.dylib -install_name libhello.dylib \         -init _my_start libhello.o $ gcc -o main.dynamic main.o -L. -lhello $ ./main.dynamic my_start Hello, World!

Other notable aspects of the Mach-O runtime architecture include multiple binding styles, two-level namespaces, and weakly linked symbols.

2.6.3.1. Multiple Binding Styles

When binding imported references in a program, Mac OS X supports just-in-time (lazy) binding, load-time binding, and prebinding. With lazy binding, a shared library is loaded only when a symbol from that library is used for the first time. Once the library is loaded, not all of the program's unresolved references from that library are bound immediatelyreferences are bound upon first use. With load-time binding, the dynamic linker binds all undefined references in a program when it is launchedor, as in the case of a bundle, when it is loaded.

We will look at details of prebinding in Section 2.8.4.

2.6.3.2. Two-Level Namespaces

A two-level namespace implies that imported symbols are referenced both by the symbol's name and by the name of the library containing it. Mac OS X uses two-level namespaces by default.

When two-level namespaces are being used, it is not possible to use dyld's DYLD_INSERT_LIBRARIES environment variable^[27] to preload libraries before the ones specified in a program. It is possible to ignore two-level namespace bindings at runtime by using dyld's DYLD_FORCE_FLAT_NAMESPACE environment variable, which forces all images in the program to be linked as flat-namespace images. However, doing so may be problematic when the images have multiply defined symbols.

^[27] This variable is analogous to the LD_PRELOAD environment variable supported by the runtime linker on several other platforms.

2.6.3.3. Weakly Linked Symbols

A weakly linked symbol^[28] is one whose absence does not cause the dynamic linker to throw a runtime binding errora program referencing such a symbol will execute. However, the dynamic linker will explicitly set the address of a nonexistent weak symbol to NULL. It is the program's responsibility to ensure that a weak symbol exists (i.e., its address is not NULL) before using it. It is also possible to link to an entire framework weakly, which results in all of the framework's symbols being weakly linked. Figure 210 shows an example of using a weak symbol.

^[28] We use the term "symbol" interchangeably with the term "reference" in this discussion. A reference is to a symbol, which may represent code or data.

Figure 210. Using weak symbols on Mac OS X 10.2 and newer

$ cat libweakfunc.c #include <stdio.h> void weakfunc(void) {     puts("I am a weak function."); } $ cat main.c #include <stdio.h> extern void weakfunc(void) __attribute__((weak_import)); int main(void) {     if (weakfunc)         weakfunc();     else         puts("Weak function not found.");     return 0; } $ gcc -c libweakfunc.c $ gcc -dynamiclib -o libweakfunc.dylib \         -install_name libweakfunc.dylib libweakfunc.o $ MACOSX_DEPLOYMENT_TARGET=10.4 gcc -o main main.c -L. -lweakfunc $ ./main I am a weak function. $ rm libweakfunc.dylib $ ./main Weak function not found.

2.6.3.4. dyld Interposing

Beginning with Mac OS X 10.4, dyld does support programmatic interposing of library functions, although the corresponding source code is not included in the open source version of dyld. Suppose you wish to use this feature to intercept a C library function, say, open(), with your own function, say, my_open(). You can achieve the interposing by creating a dynamic shared library that implements my_open() and also contains a section called __interpose in its __DATA segment. The contents of the __interpose section are the "original" and "new" function-pointer tuplesin this case, { my_open, open }. Using the DYLD_INSERT_ LIBRARIES variable with such a library will enable the interposing. Note that my_open() can call open() normally, without having to first look up the address of the "open" symbol. Figure 211 shows an example of dyld interposingthe programmer-provided library "takes over" the open() and close() functions.

Figure 211. Interposing a library function through dyld

// libinterposers.c #include <stdio.h> #include <unistd.h> #include <fcntl.h> typedef struct interpose_s {     void *new_func;     void *orig_func; } interpose_t; int my_open(const char *, int, mode_t); int my_close(int); static const interpose_t interposers[] \     __attribute__ ((section("__DATA, __interpose"))) = {         { (void *)my_open,  (void *)open  },         { (void *)my_close, (void *)close },     }; int my_open(const char *path, int flags, mode_t mode) {     int ret = open(path, flags, mode);     printf("--> %d = open(%s, %x, %x)\n", ret, path, flags, mode);     return ret; } int my_close(int d) {     int ret = close(d);     printf("--> %d = close(%d)\n", ret, d);     return ret; } $ gcc -Wall -dynamiclib -o /tmp/libinterposers.dylib libinterposers.c $ DYLD_INSERT_LIBRARIES=/tmp/libinterposers.dylib cat /dev/null --> 9 = open(/dev/null, 0, 0) --> 0 = close(9)

As a word of caution, one must note that calls to certain library functions could lead to recursive invocations of the interposer functions. For example, the implementation of printf() may call malloc(). If you are interposing malloc() and calling printf() from within your version of malloc(), a recursive situation can arise.