Section 5.4. Mach Subsystem Initialization

5.4. Mach Subsystem Initialization

kernel_bootstrap() performs much of the higher-level kernel startup. In fact, it eventually launches BSD initialization, which in turn ends by initiating user-level system startup. Figure 511 shows the key steps performed by kernel_bootstrap().

Figure 511. Bootstrapping kernel subsystems

lck_mod_init() [osfmk/kern/locks.c] initializes data structures that are used by Mach's locking primitives.

5.4.1. Scheduler Initialization

sched_init() [osfmk/kern/sched_prim.c] initializes the processor scheduler. It sets the default preemption rate to DEFAULT_PREEMPTION_RATE (defined to be 100 per second in osfmk/kern/sched_prim.c), unless a boot argument was used to set it to some other value. sched_init() calculates the values of fundamental scheduling constants. For example, it calculates the standard timeslicing quantum in microseconds as the number 1,000,000 divided by the default preemption rate. It then prints a message advertising the value so calculated. In a verbose boot (PowerPC), this is the first kernel message seen by the user:

standard timeslicing quantum is 10000 us

sched_init() also performs the following specific operations.

• It calls wait_queues_init() [osfmk/kern/sched_prim.c] to initialize the event wait queues used by the scheduler APIs. The hash of wait queue structuresconsisting of NUMQUEUES (59) bucketsis statically allocated in osfmk/kern/sched_prim.c. Each bucket contains a queue of threads that have the same hash function value. All wait queues are initialized with a synchronizer wait ordering policy of SYNC_POLICY_FIFO.

• It calls load_shift_init() [osfmk/kern/sched_prim.c] to initialize the timeshare loading factors contained in the sched_load_shifts array. The array contains a per-run-queue factor, which is used as the dynamic, load-based component in the computation of the timeshare priority conversion factor.

• It calls pset_init() [osfmk/kern/processor.c] to initialize the default processor set. Mach scheduling uses processor sets,^[13] hence this is necessary for the scheduler to run. pset_init() initializes the components of the specified processor-set data structure, including its various queues.

  ^[13] We will look at Mach processor sets and scheduling in Chapter 7.

• Finally, sched_init() sets the scheduler tickthe sched_tick variableto 0.

5.4.2. High-Level Virtual Memory Subsystem Initialization

kernel_bootstrap() calls vm_mem_bootstrap() [osfmk/vm/vm_init.c] to initialize the platform-independent virtual memory subsystema major step in bootstrapping. Figure 512 shows the sequence of actions performed by vm_mem_bootstrap().

Figure 512. High-level virtual memory subsystem initialization

vm_page_bootstrap() [osfmk/vm/vm_resident.c] initializes the resident memory module.^[14] It allocates memory for VM management data structures, initializes the page queues, "steals" memory for Mach's map and zone subsystems, allocates and initializes the virtual-to-physical table hash buckets, allocates the resident page table by calling pmap_startup() [osfmk/vm/vm_resident.c], and computes the number of pages that must be marked as "wired" because they cannot be moved. pmap_startup() calculates the amount of free memory and allocates space for all page frames that would be needed. It then iterates over available physical pages, calling vm_page_init() [osfmk/vm/vm_resident.c] to initialize page frames. Figure 513 depicts this operation.

^[14] We will look at Mach VM details in Chapter 8.

Figure 513. Initializing page frames during VM subsystem initialization

// osfmk/vm/vm_resident.c struct vm_page vm_page_template; ... void vm_page_bootstrap(vm_offset_t *startp, vm_offset_t *endp) {     register vm_page_t m;     ...     // Initialize the vm_page_template     m = &vm_page_template;     m->object = VM_OBJECT_NULL;     m->offset = (vm_object_offset_t)-1;     ...     // Set various fields of m     ...     m->phys_page = 0;     ...     // "Steal" memory for Mach's map and zone subsystems     vm_map_steal_memory();     zone_steal_memory();     ...     pmap_startup(&virtual_space_start, &virtual_space_end);     ... } ... void pmap_startup(vm_offset_t *startp, vm_offset_t *endp) {     unsigned int i, npages, pages_initialized, fill, fillval;     vm_page_t pages;     ppnum_t   phys_page;     addr64_t  tmpaddr;     ...     // We calculate (in npages) how many page frames we will have, and then     // allocate the page structures in one chunk     // Get the amount of memory left     tmpaddr = (addr64_t)pmap_free_pages() * (addr64_t)PAGE_SIZE;     // Account for any slack     tmpaddr = tmpaddr + \         (addr64_t)(round_page_32(virtual_space_start) - (virtual_space_start);     npages = (unsigned int)(tmpaddr / (addr64_t)(PAGE_SIZE + sizeof(*pages)));     pages = (vm_page_t)pmap_steal_memory(npages * sizeof(*pages));     // Initialize the page frames     for (i = 0, pages_initialized = 0; i < npages; i++) {         // Allocate a physical page         if (!pmap_next_page(&phys_page))             break;         // Initialize the fields in a new page         vm_page_init(&pages[i], phys_page);         vm_page_pages++;         pages_initialized++;     }     ... } ... void vm_page_init(vm_page_t mem, ppnum_t phys_page) {     assert(phys_page);     *mem = vm_page_template;     mem->phys_page = phys_page; }

Once vm_page_bootstrap() returns, all physical memory is accounted for, and the kernel can explicitly use virtual addresses. As Figure 512 shows, vm_mem_bootstrap() then initializes various other components of the VM subsystem.

zone_bootstrap() [osfmk/kern/zalloc.c] initializes zone_zone, the "zone of zones,"^[15] which uses fixed memory allocated earlier during memory subsystem initialization.

^[15] We will look at Mach's zone-based memory allocator in Chapter 8.

vm_object_bootstrap() [osfmk/vm/vm_object.c] initializes Mach's VM objects module. This includes initializing zones for VM objects (vm_object_zone) and VM object hash entries (vm_object_hash_zone). The kernel object (kernel_object) and the submap object (vm_submap_object) are also initialized here. Mach's external page management hint technology,^[16] which maintains a (potentially incomplete) map of pages written to external storage for a range of virtual memory, is initialized via a call to vm_external_module_initialize() [osfmk/vm/vm_external.c].

^[16] This should not be confused with Mach's external (to the kernel) memory management, which is not available in Mac OS X.

vm_map_init() [osfmk/vm/vm_map.c] initializes a zone for allocating vm_map structures (vm_map_zone), a zone for allocating nonkernel vm_map_entry structures (vm_map_entry_zone), a special zone for allocating kernel-only vm_map_entry structures (vm_map_kentry_zone), and a zone for vm_map_copy structures (vm_map_copy_zone).

kmem_init() [osfmk/vm/vm_kern.c] initializes the kernel's virtual memory map (kernel_map). Moreover, any virtual memory that may have been allocated so faras determined by the difference between the constant VM_MIN_KERNEL_ADDRESS and the lower address bound passed to kmem_init()is reserved by entering it into the kernel's map. kmem_init() also sets the value of the vm_page_wire_count global variable, which, at this point, represents all kernel memory used.

vm_page_wire_count = (atop_64(max_mem)                        - (vm_page_free_count                        + vm_page_active_count                        + vm_page_inactive_count));

pmap_init() [osfmk/ppc/pmap.c] finishes the initialization of the physical map module by allocating the remaining data structures that the module needs to map virtual memory. It initializes a zone of pmap structures (pmap_zone) from which new physical maps are allocated, marks pmap as initialized, and sets the free pmap count to zero.

vm_mem_bootstrap() then checks for the presence of the zsize boot argument, which, if present, specifies the maximum amount of address space allocated for zonesthat is, the zone map size. By default, the kernel uses 25% of the physical memory size (sane_size). Regardless of whether zsize is specified or not, the kernel clamps the maximum and minimum values of the zone map size to 768MB and 12MB, respectively. vm_mem_bootstrap() now calls zone_init() [osfmk/kern/zalloc.c] to allocate address space for zones.

kalloc_init() [osfmk/kern/kalloc.c] initializes the kernel memory allocator, which uses multiple power-of-2-sized zones and a 16MB submap allocated from kernel_map. The latter is used in conjunction with kmem_alloc() [osfmk/vm/vm_kern.c] for allocations that are too large for a zone. kalloc_init() determines the value for kalloc_max, which represents the first power of 2 for which no zone exists. By default, kalloc_max is set to 16KB, unless the page size is more than 16KB, in which case it is set to the page size. kalloc_init() then iterates over supported allocation sizesstarting from 1 and going up to kalloc_max in powers of two. It initializes a zone for sizes KALLOC_MINSIZE (16 bytes) and higher by calling zinit(). The zones are named kalloc.16, kalloc.32, kalloc.64, and so on. The maximum number of elements in a zone depends on the size that the zone handles.

• kalloc.16 has 1024 elements.

• kalloc.32 tHRough kalloc.256 have 4096 elements.

• kalloc.512 through kalloc.4096 have 1024 elements.

• kalloc.8192 has 4096 elements.

vm_fault_init() [osfmk/vm/vm_fault.c] initializes any private data structures that the kernel might have in the page-fault-handling module.

vm_page_module_init() [osfmk/vm/vm_resident.c] initializes a zone for "fictitious" resident page structures that do not actually refer to any physical pages but are used for holding important page information. The physical page address for such pages is set to -1.

memory_manager_default_init() [osfmk/vm/memory_object.c] initializes a mutex that is used while getting or setting the default memory manager Mach port. memory_object_control_bootstrap() [osfmk/vm/memory_object.c] initializes a zone (mem_obj_control_zone) used for allocating pager^[17] request ports. device_pager_bootstrap() [osfmk/vm/device_vm.c] initializes a zone for device node pager structures (device_pager_zone).

^[17] As we will see in Chapter 8, a pager in Mach's VM subsystem represents a data source.

5.4.3. IPC Initialization

kernel_bootstrap() calls ipc_bootstrap() [osfmk/ipc/ipc_init.c] to set up the IPC subsystem^[18] enough for the kernel task to be created. ipc_bootstrap() performs the following actions.

^[18] Chapter 9 discusses the Mac OS X IPC subsystem.

• It initializes zones for IPC capability spaces (ipc_space_zone), IPC tree entries (ipc_tree_entry_zone), IPC ports and port sets (ipc_object_zones[IOT_PORT] and ipc_object_zones[IOT_PORT_SET], respectively), and IPC kernel messages (ipc_kmsg_zone).

• It calls mig_init() [osfmk/kern/ipc_kobject.c] to initialize the Mach Interface Generator (MIG). As part of this initialization, various standard MIG subsystems, such as those for tasks and threads, are examined as the MIG hash table is populated. When an IPC message is sent to the kernel, the message ID is used to search for a matching entry in the hash table. The entry specifies the size of the message's reply and contains a pointer to the routine that performs the corresponding kernel function.

• It calls ipc_table_init() [osfmk/ipc/ipc_table.c] to allocate and initialize a table of IPC capabilities (ipc_table_entries) and another for dead-name requests (ipc_table_dnrequests).

• It calls ipc_hash_init() [osfmk/ipc/ipc_hash.c] to allocate and initialize a reverse hash global table for IPC entries.

• It calls semaphore_init() [osfmk/kern/sync_sema.c] to initialize the zone from which semaphores are allocated (semaphore_zone).

• It calls lock_set_init() [osfmk/kern/sync_lock.c] to initialize the lock set subsystem.

• It calls mk_timer_init() [osfmk/kern/mk_timer.c] to initialize the zone from which Mach timers are allocated (mk_timer_zone).

• It calls host_notify_init() [osfmk/kern/host_notify.c] to initialize the zone from which host notification request entries are allocated (host_notify_zone).

5.4.4. Finishing VM and IPC Initialization

kernel_bootstrap() next calls vm_mem_init() [osfmk/vm/vm_init.c], which in turn calls vm_object_init() [osfmk/vm/vm_object.c] to finish initializing the kernel object.

ipc_init() [osfmk/ipc/ipc_init.c] performs the final initialization of the IPC subsystem. It allocates two pageable maps: the ipc_kernel_map map to manage memory allocations made during Mach IPC calls and the ipc_kernel_copy_map map in which space is allocated during Mach IPC for out-of-line data that is to be physically copied. ipc_init() finally calls ipc_host_init() [osfmk/kern/ipc_host.c], which performs the following actions.

• It allocates some of the special host ports, such as the HOST_PORT, the HOST_PRIV_PORT, and the HOST_SECURITY_PORT. Moreover, it sets the special ports by calling kernel_set_special_port() [osfmk/kern/host.c].

• It sets all the host-level exception ports to IP_NULL.

• It calls ipc_pset_init() [osfmk/kern/ipc_host.c] to allocate the control and name ports for the default processor set. Next, it calls ipc_pset_enable() [osfmk/kern/ipc_host.c] to set these ports, which in turn calls ipc_kobject_set() [osfmk/kern/ipc_kobject.c] to make the ports represent the processor set and its name, respectively.

• It calls ipc_processor_init() [osfmk/kern/ipc_host.c] to allocate the master processor's control port. Next, it calls ipc_processor_enable() [osfmk/kern/ipc_host.c], which calls ipc_kobject_set() to make the port represent the processor.

5.4.5. Initializing Miscellaneous Subsystems

machine_init() [osfmk/ppc/model_dep.c] calls clock_config() [osfmk/kern/clock.c] to configure the clock subsystem. clock_config() calls the configuration (but not initialization) functions of all available clock devices, such as the calendar and the system clocks. It also calls timer_call_initialize() [osfmk/kern/timer_call.c], which registers timer_call_interrupt() [osfmk/kern/timer_call.c] as the function to be called from the real-time clock device interrupt handler whenever the real-time clock timer expires (in other words, timer_call_interrupt() services the timer callout queue for a processor).

machine_init() also calls perfmon_init() [osfmk/ppc/hw_perfmon.c], which initializes a lock used by the performance-monitoring facility.

kmod_init() [osfmk/kern/kmod.c] initializes locks and a command queue used by the kernel module subsystem. The kernel enqueues data packets containing module load requests in this queue, which is serviced by the kextd user-space daemon.

clock_init() [osfmk/kern/clock.c] calls the initialization functions of all available clock devices. Note that unlike clock_config(), which is called only once on the master processor at boot time, clock_init() is called every time a processor is started.

ledger_init() [osfmk/kern/ledger.c] initializes Mach ledgers. A ledger is a kernel abstraction used for resource accounting. It can be used to limit consumption of other resources. Ledgers are not used in Mac OS X, and xnu's implementation of ledgers is not functional.

5.4.6. Tasks and Threads

task_init() [osfmk/kern/task.c] initializes a zone (task_zone) from which new task structures are allocated. The built-in limit on the number of tasks is defined as TASK_MAX in osfmk/kern/mach_param.h, with a typical value of 1024. task_init() calls task_create_internal() [osfmk/kern/task.c] to create the first taskthe kernel task (kernel_task). The kernel task's default address space map is deallocatedkernel_map is assigned as its address space instead. Note that since the kernel task's parent task is TASK_NULL, it does not inherit any memory.

thread_init() [osfmk/kern/thread.c] initializes a zone (thread_zone) from which new thread structures are allocated. The built-in limit on the number of threads is defined as THREAD_MAX (2560) in osfmk/kern/mach_param.h. tHRead_init() also calls stack_init() [osfmk/kern/stack.c], which allocates a map (stack_map) for kernel stacks. A kernel stack is 16KB in size and resides in nonpageable memory. thread_init() also calls machine_thread_init() [osfmk/ppc/pcb.c], which may perform machine-specific initializations.

5.4.7. Launching the Kernel Bootstrap Thread

kernel_bootstrap() now creates a kernel thread, with kernel_bootstrap_thread() [osfmk/kern/startup.c] as the continuation function,^[19] for completing the remaining kernel startup operations. This will be the first kernel thread to run on the processor. The thread's resources are deallocated before it is handed over to load_context() [osfmk/kern/startup.c] for execution. load_context() calls machine_set_current_thread() [osfmk/ppc/machine_routines_asm.s], which loads the SPRG1 register with the current thread pointer.^[20] It then calls processor_up() [osfmk/kern/machine.c], the machine-independent Mach-level CPU enabler routine that adds the specified processor to the default processor set. The processor's state is set as PROCESSOR_RUNNING. The global machine_info structure's avail_cpus field is atomically incremented by one. processor_up() also calls the ml_cpu_up() [osfmk/ppc/machine_routines.c] machine-dependent routine. ml_cpu_up() atomically increments the physical_cpu and logical_cpu fields of machine_info by one each. Finally, load_context() calls machine_load_context() [osfmk/ppc/cswtch.s] to load the thread's hardware context and set it running.

^[19] We will look at continuations in Chapter 7.

^[20] The Mac OS X kernel conventionally uses SPRG1 for this purpose.