15.6. Locking in UFS UFS uses two basic types of locks: kmutex_t and krwlock_t. The workings of these synchronization primitives is covered in Chapter 17. UFS locks can be divided into eight categories: Inode locks Queue locks ACL locks VNODE locks VFS locks VOP_RWLOCK ufs_iuniqtime_lock Logging locks 15.6.1. UFS Lock Descriptions Tables 15.2 through 15.9 describe the UFS locks in more detail. Table 15.2. Inode LocksName | Type | Description |
---|
i_rwlock | krwlock_t | Serializes write requests. Allows reads to proceed in parallel. Serializes directory reads and updates. Does not protect inode fields. Indirectly protects block lists since it serializes allocations/deallocations in UFS. Must be taken before starting UFS logging transactions if operating on a file; otherwise, taken after starting logging transaction.
| i_contents | krwlock_t | Protects most fields in the inode. When held as a writer, protects all the fields protected by the i_tlock.
| i_tlock | kmutex_t | When held with the i_contents reader lock, protects the following inode fields: i_utime, i_ctime, i_mtime, i_flag, i_delayoff, i_delaylen, i_nextrio, i_writes, i_writer, i_mapcnt. Also used as mutex for write throttling in UFS. i_contents and i_tlock held together allows parallelism in updates.
| i_hlock | kmutex_t | |
Table 15.3. Inode Queue LocksName | Type | Description |
---|
ufs_scan_lock | kmutex_t | Synchronizes ufs_scan_inodes threads ufs_update(), ufs_sync(), ufs_scan_inodes(). Needed because of global inode list. | ufs_q->uq_mutex | krwlock_t | | ufs_hlock | kmutex_t | | ih_lock | kmutex_t | |
Table 15.4. Quota Queue LocksName | Type | Description |
---|
dq_cachelock | kmutex_t | | dq_freelock | kmutex_t | | dq_rwlock | krwlock_t | Protects the entire quota subsystem. Taken as writer when the quota subsystem is initialized. Taken as reader when we do not want entire quota subsystem to be quiesced. As writer, allows updates to quota-related fields in the ufsvfs structure. Also protects the dquot file as writer to allow quota updates. As reader, allows reads from the quota-related fields in the ufsvfs structure.
| dqout.dq_lock | kmutex_t | |
Table 15.5. VNODE LocksName | Type | Description |
---|
v_lock | kmutex_t | |
Table 15.6. ACL LocksName | Type | Description |
---|
s_lock | krwlock_t | |
Table 15.7. VFS LocksName | Type | Description |
---|
vfs_lock | kmutex_t | | vfs_dqrwlock | krwlock_t | Manages quota subsystem quiescence. If held as writer, UFS quota subsystem may be experiencing changes in quotas, enabling/disabling of quotas, setting new quota limits. Protects d_quot structure. This structure keeps track of all the enabled quotas per file system. Important note: UFS shadow inodes that are used to hold ACL data and extended attribute directories are not counted against user quotas. Thus, this lock is not held for updates to these. Reader held for this lock indicates to quota subsystem that major changes should not be occurring during that time. Held when the i_contents writer lock is held, as described above, signifying that changes are occurring that affect user quotas. Since UFS quotas can be enabled/disabled on the fly, this lock must be taken in all appropriate situations. It is not sufficient to check if the UFS quota subsystem is enabled before taking the lock. | ufsvfs_mutex | kmutex_t | Protects access to the list that links all UFS file system instances. Updates lists as a part of the mount operation. Allows synchronization of all UFS file systems.
|
Table 15.8. VOP_RWLOCK or ufs_rwlockName | Type | Description |
---|
ufs_rwlock() | function | Prevents concurrent reads and writes to a file. Used by NFS when calling a VOP_READDIR, to prevent directory contents from changing. NFS uses this lock to get attributes before and after a read or write to disable another operation from modifying the file.
|
Table 15.9. Logging LocksName | Type | Description | mtm_lock | kmutex_t | | mtm_mutex | kmutex_t | | mtm_rwlock | krwlock_t | | un_log_mutex | kmutex_t | | un_state_mutex | kmutex_t | |
15.6.2. Inode Lock Ordering Now that we are all familiar with the several different types of locks available in UFS, let us put them in order as if we were to work on an inode. Lock ordering is critical, and any mistake will more than likely cause the system to deadlock, and may end up panicking it! Figure 15.16 give us a quick overview of lock ordering specific to the inode. Figure 15.16. Inode Lock Ordering Precedence 15.6.3. UFS Lockfs Protocol Along with basic inode locking, UFS also provides a mechanism to quiesce a file system for file system locking and for the forced unmounting of a file system. All VOPs (vnode operations) in UFS are required to follow the UFS lock protocol with ufs_lockfs_begin() and ufs_lockfs_end(), although the following functions purposely do not adhere to the tradition: ufs_close ufs_putpage ufs_inactive ufs_addmap ufs_delmap ufs_rwlock ufs_rwunlock ufs_poll The basic principle here is that UFS supports various file system lock states (see list below) and each vnode operation must initiate the protocol by calling ufs_lockfs_begin() with an appropriate lock mask (a lock that this operation might grab while it is being processed) and end the protocol by calling ufs_lockfs_end before it returns. This way, UFS knows exactly how many vnode operations are in progress for the given file system by incrementing and decrementing the ul_vnops_cnt variable in the file-system-dependent ulockfs structure. If the file system is hard-locked, the thread gets an EIO error. If the file system is error-locked, then the thread is blocked. Here are the file system locks and their actions. Write lock. Suspends writes that would modify the file system. Access times are not kept while a file system is write-locked. Name lock. Suspends accesses that could change or remove existing directories entries. Delete lock. Suspends access that could remove directory entries. Hard lock. Returns an error upon every access to the locked file system and cannot be unlocked. Hard-locked file systems can be unmounted. Hard lock supports forcible unmount. Error lock. Blocks all local access to the file system and returns EWOULDBLOCK on all remote access. File systems are error-locked by UFS upon detection of internal inconsistency. They can only be unlocked after successful repair by fsck, which is usually done automatically. Error-locked file systems can be unmounted. Once the file system becomes clean, it can be upgraded to a hard lock. Soft lock. Quiesces a file system. Unlock. Awakens suspended accesses, releases existing locks, and flushes the file system. While a vnode operation is being executed in UFS, a call can be made to another vnode function on the same UFS or a different UFS. This is called recursive VOP. The per-file system vnode operation counter is not incremented or decremented during recursive calls. Here is the basic ordering to initiate and complete the lock protocol when operating on an inode in UFS. 1) Acquire i_rwlock (from the vnode layer in most cases). 2) Begin the UFS lock protocol by calling ufs_lockfs_begin(). 3) Open UFS logging transactions if necessary now. 4) Acquire inode and quota locks (vfs_dqrwlock, i_contents, i_tlock, ...). 5) [work on inode] 6) Drop inode and quota locks (i_tlock, i_contents, vfs_dqrwlock, ...). 7) Close logging transactions. 8) End the UFS lock protocol by calling ufs_lockfs_end(). 9) Release i_rwlock. When working with directories, you need to make one minor change. i_rwlock is acquired after the logging transaction is initialized, and i_rwlock is released before the transaction is ended. Here are the steps. 1) Begin the UFS lock protocol by calling ufs_lockfs_begin(). 2) Open UFS logging transactions if necessary now. 3) Acquire i_rwlock. 4) Acquire inode and quota locks (vfs_dqrwlock, i_contents, i_tlock, ...). 5) [work on inode] 6) Drop inode and quota locks (i_tlock, i_contents, vfs_dqrwlock, ...). 7) Release i_rwlock. 8) Close logging transactions. 9) End the UFS lock protocol by calling ufs_lockfs_end(). |