Section 2.4. The xnu Kernel | Mac OS X Internals: A Systems Approach

2.4. The xnu Kernel

The Mac OS X kernel is called xnu. In the simplest sense, xnu could be viewed as having a Mach-based core, a BSD-based operating system personality, and an object-oriented runtime environment for drivers^[7] and other kernel extensions. The Mach component is based on Mach 3, whereas the BSD component is based on FreeBSD 5. A running kernel contains numerous drivers that do not reside in the xnu code base but have their own Darwin packages. In this sense, the Mac OS X kernel is "more than" xnu. However, we will usually not make distinctions based on packagingwe will use the term "xnu" to refer to the combination of the base kernel (as implemented in the xnu Darwin package) and all kernel extensions. With this understanding, we can divide the Mac OS X kernel into the following components:

^[7] A driver is a specific type of kernel extension.

Machthe services layer
BSDthe primary system programming interface provider
The I/O Kitthe runtime environment for drivers
libkernan in-kernel library
libsaan in-kernel library that is normally used only during early system startup
The Platform Expertthe hardware abstraction module^[8]
^[8] The Platform Expert consists of support code in the base kernel and a platform-specific kernel extension.
Kernel extensionsvarious I/O Kit families, the majority of loadable device drivers, and some non-I/O Kit extensions

The Darwin xnu package consists of roughly a million lines of code, of which about half could be categorized under BSD and a third under Mach. The various kernel extensions, not all of which are needed (or loaded) on a given system, together constitute another million lines of code.

The number of kernel extensions loaded at any time on a given system is significantly less than the total number of kernel extensions present on the system. The kextstat command can be used to list the currently loaded kernel extensions. The /System/Library/Extensions/ directory is the standard location of kernel extensions.

Chapter 6 discusses several details of xnu. Several other chapters discuss specific areas of kernel functionality. Let us now briefly discuss the important kernel components.

2.4.1. Mach

If the xnu kernel is the core of Mac OS X, then Mach may be considered the core of xnu. Mach provides critical low-level services that are transparent to applications. System aspects that Mach is responsible for include the following:

Hardware abstraction to some extent
Processor management, including symmetric multiprocessing and scheduling
Preemptive multitasking, including support for tasks and threads
Virtual memory management, including low-level paging, memory protection, sharing, and inheritance
Low-level IPC mechanisms that are the basis for all messaging in the kernel
Real-time support that allows time-sensitive applications (e.g., media applications such as GarageBand and iTunes) to have latency-bounded access to processor resources
Kernel debugging support^[9]
^[9] xnu's built-in low-level kernel debugger is called KDB (or DDB). It is implemented in the Mach portion of the kernel, and so is KDPa remote kernel debugging protocol used by the GNU debugger (GDB).
Console I/O

Prior to Mac OS X 10.4, xnu already supported more than 4GB of physical memory on 64-bit hardware, although process virtual address spaces were still 32-bit. Consequently, an individual process could not address more than 4GB of virtual memory. With Mac OS X 10.4, xnu added support for 64-bit processes on 64-bit hardware, with an upper limit of 18 exabytes^[10] on process virtual address spaces.

^[10] 10¹⁸ bytes.

Mach is often unequivocally equated to a microkernel, but as we saw in Chapter 1, it was not until version 3 of Mach that it was used as a true microkernel. Earlier versionsincluding Mach 2.5, which was the basis for the Open Software Foundation's OSF/1 operating systemhad monolithic implementations in which BSD and Mach resided in the same "kernel" address space. Even though Apple uses a Mach implementation that derives from Mach 3, xnu does not use Mach as a traditional microkernel. Various subsystems that would be implemented as user-space servers in a true microkernel system are part of the kernel proper in Mac OS X. In particular, the BSD portion of xnu, the I/O Kit, and Mach, all reside in the same address space. However, they have well-defined responsibilities that separate them in terms of function and implementation.

2.4.2. BSD

The xnu kernel contains a substantial amount of BSD-derived code, which is what we collectively refer to as BSD in the context of Mac OS X. However, it is not the case that a well-defined BSD kernel runs within xnu, whether as a single Mach task or otherwise. Whereas some BSD-derived portions in xnu are similar to their original forms, other portions are quite different, since they were made to coexist with non-BSD entities such as the I/O Kit and Mach. Consequently, one can find several instances of code of different origins intertwined in the xnu kernel. Some aspects that BSD (or BSD-style code) is responsible for include the following:

BSD-style process model
Signals
User IDs, permissions, and basic security policies
POSIX APIs
Asynchronous I/O APIs (AIO)
BSD-style system calls
TCP/IP stack, BSD sockets, and firewalling
Network Kernel Extensions (NKEs), a type of kernel extension for making the BSD networking architecture fit into xnu^[11]
^[11] Before Mac OS X 10.4, an NKE was a specially designated kernel extension. Beginning with 10.4, NKE functionality is accessible to a regular kernel extension through a set of kernel programming interfaces (KPIs).
The virtual file system (VFS) layer and numerous file systems, including a file-system-independent VFS-level journaling mechanism
System V and POSIX interprocess communication mechanisms
In-kernel cryptographic framework
A system notification mechanism based on FreeBSD's kqueue/kevent mechanism, which is a system-wide service enabling notifications between applications, and from the kernel to applications
The fsevents file system change notification mechanism that is used by the Spotlight search technology
Access control lists (ACLs) and the kauth authorization framework^[12]
^[12] Beginning with Mac OS X 10.4, the kauth framework is used for the evaluation of ACLs. It is a general-purpose, extensible authorization framework.
Various synchronization primitives

Certain kernel functionality has a lower-level implementation in one portion of the kernel with higher-level abstraction layers in another portion. For example, the traditional process structure (struct proc), which is the primary kernel data structure that represents a UNIX process, is contained in the BSD portion, as is the u-area.^[13] However, strictly speaking, in Mac OS X, a BSD process does not executeit corresponds to exactly one Mach task, which contains one or more Mach threads, and it is these threads that execute. Consider the example of the fork() system call, which, along with variants like vfork(), is the only way to create a new process on a UNIX system. In Mac OS X, Mach tasks and threads are created and manipulated using Mach calls, which user programs typically do not use directly. The BSD-style fork() implementation in the kernel uses these Mach calls to create a task and a thread. Additionally, it allocates and initializes a process structure that is associated with the task. From the standpoint of the caller of fork(), these operations occur atomically, with the Mach and BSD-style data structures remaining in sync. Therefore, the BSD process structure acts as Unix "glue"^[14] in Mac OS X.

^[13] Historically, the user area (or u-area) is the name for a data structure containing per-process or per-thread data that is swappable.

^[14] It simplifies the incorporation of BSD code that depends upon the process structure.

Similarly, BSD's unified buffer cache (UBC) has a back-end that hooks into Mach's virtual memory subsystem.

The UBC allows the file system and the virtual memory subsystem to share kernel memory buffers. Each process's virtual memory typically contains mappings from both physical memory and on-disk files. Unifying the buffer cache yields a single backing store for various entities, reducing disk accesses and the amount of "wired" memory used.

Funnels

An important synchronization abstraction in Mac OS X prior to version 10.4 is the funnel, which has the semantics of a mutex of large scope^[15] that automatically drops when the holding thread sleeps. Funnels are used to serialize access to the BSD portion of the kernel. Beginning with version 10.4, xnu uses finer-grained locking. However, funnels still exist for the benefit of old code or code that is not performance critical.

^[15] Such a mutex is sometimes referred to as a giant mutex.

In addition to BSD system calls, which include sysctl() and ioctl() calls, Mac OS X uses Mach system callsor Mach trapsas necessary. There are several ways to map memory, perform block-copy operations, and otherwise exchange information between the Mac OS X user and kernel spaces.

2.4.3. The I/O Kit

xnu has an object-oriented device driver framework called the I/O Kit, which uses a restricted subset^[16] of C++ as its programming language. C++ features that are not allowed in this subset include exceptions, multiple inheritance, templates, complicated constructors, initialization lists, and runtime type identification (RTTI). However, the I/O Kit does implement its own minimal RTTI system.

^[16] The subset is based on Embedded C++.

The I/O Kit's implementation consists of kernel-resident C++ libraries (libkern and IOKit) and a user-space framework (IOKit.framework). The kernel-resident libraries are available to loadable drivers (and for that matter, the kernel). Note that the Kernel framework (Kernel.framework) encapsulates the kernel-resident libraries in order to export their header filesthe executable code for these libraries is contained in the kernel. IOKit.framework is a conventional framework used for writing user-space programs that communicate with the I/O Kit.

The I/O Kit's runtime architecture is modular and layered. It provides an infrastructure for capturing, representing, and maintaining relationships between the various hardware and software components that are involved in I/O connections. In this manner, the I/O Kit presents abstractions of the underlying hardware to the rest of the system. For example, the abstraction of a disk partition involves dynamic relationships between numerous I/O Kit classes: the physical disk, the disk controller, the bus that the controller is attached to, and so on. The device driver model provided by the I/O Kit has several useful features, such as the following:

Extensive programming interfaces, including interfaces for applications and user-space drivers to communicate with the I/O Kit
Numerous device families such as ATA/ATAPI, FireWire, Graphics, HID, Network, PCI, and USB
Object-oriented abstractions of devices
Plug-and-play and dynamic device management ("hot-plugging")
Power management
Preemptive multitasking, threading, symmetric multiprocessing, memory protection, and data management
Dynamic matching and loading of drivers for multiple bus types
A database for tracking and maintaining detailed information on instantiated objects (the I/O Registry)
A database of all I/O Kit classes available on a system (the I/O Catalog)
Interfaces for applications and user-space drivers to communicate with the I/O Kit
Driver stacking

The I/O Registry can be browsed either programmatically or by using a system utility such as ioreg, IORegistryExplorer.app (part of Apple Developer Tools), and Mr. Registry.app (part of the FireWire SDK).

Standard devices that conform to well-defined and well-supported specifications typically do not require custom I/O Kit drivers. For example, devices such as mice and keyboards are likely to work out of the box. Moreover, even if a device requires a custom driver, it may need only a user-space driver, provided it uses a FireWire or USB connection to the computer.

2.4.4. The libkern Library

The libkern library implements the runtime system for the restricted subset of C++ used by the I/O Kit's programming model. Besides providing commonly needed services to drivers, libkern also contains classes that are generally useful for kernel software development. In particular, it defines the OSObject class, which is the root base class for the Mac OS X kernel. OSObject implements dynamic typing and allocation features for supporting loadable kernel modules. The following are examples of the functionality provided by libkern:

Dynamic allocation, construction, and destruction objects, with support for a variety of built-in object types such as Arrays, Booleans, and Dictionaries
Atomic operations and miscellaneous functions such as bcmp(), memcmp(), and strlen()
Functions for byte-swapping
Provisions for tracking the number of current instances for each class
Mechanisms that help alleviate the C++ fragile base-class problem

The Fragile Base-Class Problem

The fragile base-class problem occurs when modifications to a nonleaf class "break" a derived class. A nonleaf class is one that is a base class of at least one other class. The said breakage may occur because the derived class is relyingexplicitly or implicitlyon the knowledge of certain characteristics of the nonleaf class. Examples of such characteristics include the size of the base class's virtual table (vtable), offsets in the vtable, offsets to class-protected data, and offsets to public data.

libkern provides ways to create reserved slots for class data members and virtual functions to absorb future addition of these entitiesup to a limit.

2.4.5. The libsa Library

libsa is an in-kernel support libraryessentially an in-kernel linkerused during early system startup for loading kernel extensions. The "sa" in its name is a vestigial reference to its being a library that provides functions for use by stand-alone applicationsin this case, the kernel.

Stand-alone libraries exist on other operating systemsoften with the name libstandto provide minimal runtime environments.

Mac OS X kernel extensions are normally loaded on demand through the kexTD user-space daemon (/usr/libexec/kextd). During early stages of bootstrapping, kextd is not yet available. libsa provides a subset of kextd's capabilities to the kernel. Examples of specific functionality implemented by libsa for loading, linking, and recording kernel extension object files include the following:

Simple memory allocation
Binary searching
Sorting
Miscellaneous string-handling functions
Symbol remangling
A dependency graph package used while determining kernel extension dependencies
Decompression of compressed kernels and verification of checksums

Note that libsa is not a generally available kernel library. In a typical boot-strapping scenario, libsa's code is removed from the kernel once kextd becomes available. Even when libsa is present, its constituent functions are not available to the kernel as part of any programming interface.^[17]

^[17] The kernel accesses libsa's extension-loading functionality through a function pointer shared between libsa and the kernel. libsa's constructor function initializes this pointer to point to a libsa function.

2.4.6. The Platform Expert

The Platform Expert is an objectessentially a motherboard-specific driverthat knows the type of platform that the system is running on. The I/O Kit registers a nub for the Platform Expert at system initialization time. An instance of the IOPlatformExpertDevice class becomes the root of the device tree. The root nub then loads the correct platform-specific driver, which further discovers the busses present on the system, registering a nub for each bus found. The I/O Kit loads a matching driver for each bus nub, which in turn discovers the devices connected to the bus, and so on.

Nubs

In the context of the I/O Kit, a nub is an object that defines an access point and communication channel for a physical device or a logical service. A physical device could be a bus, a disk drive or partition, a graphics card, and so on. Examples of logical services include arbitration, driver matching, and power management.

The Platform Expert abstraction provides access to a wide variety of platform-specific functions and information, such as those related to:

Constructing device trees
Parsing certain boot arguments
Identifying the machine, which includes determining processor and bus clock speeds
Accessing power management information
Retrieving and setting system time
Retrieving and setting console information
Halting and restarting the machine
Accessing the interrupt controller
Creating the system serial number string
Saving kernel panic information
Initializing a "user interface" to be used in case of kernel panics
Reading and writing the nonvolatile memory (NVRAM)
Reading and writing the parameter memory (PRAM)

2.4.7. Kernel Extensions

Besides the core kernel, the Mac OS X kernel environment includes kernel extensions that are dynamically loaded as needed. Most standard kernel extensions are targeted for the I/O Kit, but there are exceptions such as certain networking-related and file-system-related kernel extensionsfor example, webdav_fs.kext and PPP.kext. On a typical Mac OS X installation, there may be close to a hundred kernel extensions loaded at any time. Many more reside in the /System/Library/Extensions/ directory.