Section 17.4. Introduction to Synchronization Objects

17.4. Introduction to Synchronization Objects

The Solaris kernel implements several types of synchronization objects. Locks provide mutual exclusion semantics for synchronized access to shared data. Locks come in several forms and are the primary focus of this chapter. The most commonly used lock in the Solaris kernel is the mutual exclusion, or mutex lock, which provides exclusive read and write access to data. Also implemented are reader/ writer (RW) locks, for situations in which multiple readers are allowable but only one writer is allowed at a time. Kernel semaphores are also employed in some areas of the kernel, where access to a finite number of resources must be managed. A special type of mutex lock, called a dispatcher lock, is used by the kernel dispatcher when synchronization requires access protection through a locking mechanism, as well as protection from interrupts.

Condition variables, which are not a type of lock, are used for thread synchronization and are an integral part of the kernel sleep/wakeup facility. Condition variables are introduced here and covered in detail in Chapter 3.

The actual number of locks that exist in a running system at any time is dynamic and scales with the size of the system. Several hundred locks are defined in the kernel source code, but a lock count based on static source code is not accurate because locks are created dynamically during normal system activitywhen kernel threads and processes are created, file systems are mounted, files are created and opened, network connections are made, etc. Many of the locks are embedded in the kernel data structures that provide the abstractions (processes, files) provided by the kernel, and thus the number of kernel locks will scale up linearly as resources are created dynamically.

This design speaks to one of the core strengths of the Solaris kernel: scalability and scaling synchronization primitives dynamically with the size of the kernel. Dynamic lock creation has several advantages over static allocations. First, the kernel is not wasting time and space managing a large pool of unused locks when running on a smaller system, such as a desktop or workgroup server. On a large system, a sufficient number of locks is available to sustain concurrency for scalable performance. It is possible to have literally thousands of locks in existence on a large, busy system.

17.4.1. Synchronization Process

When an executing kernel thread attempts to acquire a lock, it will encounter one of two possible lock states: free (available) or not free (owned, held). A requesting thread gets ownership of an available lock when the lock-specific get lock function is invoked. If the lock is not available, the thread most likely needs to block and wait for it to come available, although, as we will see shortly, the code does not always block (sleep), waiting for a lock. For those situations in which a thread will sleep while waiting for a lock, the kernel implements a sleep queue facility, known as turnstiles, for managing threads blocking on locks.

When a kernel thread has completed the operation on the shared data protected by the lock, it must release the lock. When a thread releases a lock, the code must deal with one of two possible conditions: threads are waiting for the lock (such threads are termed waiters), or there are no waiters. With no waiters, the lock can simply be released. With waiters, the code has several options. It can release the lock and wake up the blocking threads. In that case, the first thread to execute acquires the lock. Alternatively, the code could select a thread from the turnstile (sleep queue), based on priority or sleep time, and wake up only that thread. Finally, the code could select which thread should get the lock next, and the lock owner could hand the lock off to the selected thread. As we will see in the following sections, no one solution is suitable for all situations, and the Solaris kernel uses all three methods, depending on the lock type. Figure 17.4 provides the big picture.

Figure 17.4. Solaris LocksThe Big Picture

Figure 17.4 provides a generic representation of the execution flow. Later we will see the results of a considerable amount of engineering effort that has gone into the lock code: improved efficiency and speed with short code paths, optimizations for the hot path (frequently hit code path) with well-tuned assembly code, and the best algorithms for lock release as determined by extensive analysis.

17.4.2. Synchronization Object Operations Vector

Each of the synchronization objects discussed in this sectionmutex locks, reader/writer locks, and semaphoresdefines an operations vector that is linked to kernel threads that are blocking on the object. Specifically, the object's operations vector is a data structure that exports a subset of object functions required for kthreads sleeping on the lock. The generic structure is defined as follows:

 /*   * The following data structure is used to map   * synchronization object type numbers to the   * synchronization object's sleep queue number   * or the synch. object's owner function.   */ typedef struct _sobj_ops {         int             sobj_type;         kthread_t       *(*sobj_owner)();         void            (*sobj_unsleep)(kthread_t *);         void            (*sobj_change_pri)(kthread_t *, pri_t, pri_t *); } sobj_ops_t;                                                                       See sys/sobject.h 

The structure shown above provides for the object type declaration. For each synchronization object type, a type-specific structure is defined: mutex_sobj_ops for mutex locks, rw_sobj_ops for reader/writer locks, and sema_sobj_ops for semaphores.

The structure also provides three functions that may be called on behalf of a kthread sleeping on a synchronization object:

• An owner function, which returns the ID of the kernel thread that owns the object.

• An unsleep function, which transitions a kernel thread from a sleep state.

• A change_pri function, which changes the priority of a kernel thread, used for priority inheritance. (See Section 17.7.)

We will see how references to the lock's operations structure are implemented as we move through specifics on lock implementations in the following sections.

It is useful to note at this point that our examination of Solaris kernel locks offers a good example of some of the design trade-offs involved in kernel software engineering. Building the various software components that make up the Solaris kernel is a series of design decisions, when performance needs are measured against complexity. In areas of the kernel where optimal performance is a top priority, simplicity might be sacrificed in favor of performance. The locking facilities in the Solaris kernel are an area where such trade-offs are mademuch of the lock code is written in assembly language, for speed, rather than in the C language; the latter is easier to code with and maintain but is potentially slower. In some cases, when the code path is not performance critical, a simpler design will be favored over cryptic assembly code or complexity in the algorithms. The behavior of a particular design is examined through exhaustive testing, to ensure that the best possible design decisions were made.