Section 12.8. Examining HFS Features


12.8. Examining HFS+ Features

In this section, we will look at several standard HFS+ features, such as case sensitivity, journaling, hard links, symbolic links, and aliases.

12.8.1. Case Sensitivity

By default, HFS+ is a case-preserving, case-insensitive file system, whereas traditional Unix file systems are case-sensitive. The case insensitivity of HFS+ might be undesirable in certain situations. Suppose you have an archive containing files called Makefile and makefile in the same directory:

$ tar -tf archive.tar Makefile makefile


If we extract these files on an HFS+ volume, the second file to be extracted would overwrite the first:

$ tar -xvf archive.tar Makefile makefile $ ls *akefile makefile


The default case insensitivity of HFS+ applies only to file and folder names. Extended attribute names are always case-sensitive.


HFSX was introduced with Mac OS X 10.3 as an extension to HFS+ for supporting case-sensitive file and folder names. You can create a case-sensitive HFS+ file system by passing the -s option to newfs_hfs. HFSX disk images can be created using hdiutil.

$ hdiutil create -size 32m -fs HFSX -volname HFSX /tmp/hfsx.dmg ... created: /tmp/hfsx.dmg $ open /tmp/hfsx.dmg $ hfsdebug -V /Volumes/HFSX -v ... # HFS Plus Volume   Volume size          = 32728 KB/31.96 MB/0.03 GB # HFS Plus Volume Header   signature            = 0x4858 (HX)   version              = 0x5 ... $


Note the differences from a case-insensitive HFS+ volume: The volume signature is HX instead of H+, and the version is 5 instead of 4. A signature value of HX is still stored as H+ in memory and therefore is not flushed to disk.

The case sensitivity of a volume is also recorded in the keyCompareType field of the Catalog B-Tree. If this field's value is 0xbc, binary comparison (case-sensitive) is used to compare names. If the field's value is 0xcf, case folding is performed while comparing names. Note that keyCompareType is always 0xbc in the Attributes B-Tree and is irrelevant in the Extents Overflow B-Tree.

$ sudo hfsdebug -b catalog # root volume, should be case-insensitive by default ... keyCompareType       = 0xcf (kHFSCaseFolding, case-insensitive) ... $ hfsdebug -V /Volumes/HFSX -b catalog # case-sensitive volume ... keyCompareType       = 0xbc (kHFSBinaryCompare, case-sensitive) ...


12.8.2. Filename Encodings

HFS+ uses Unicode for encoding names of files, folders, and extended attributes. As we saw in Section 12.7.2, file and folder names are represented by the HFSUniStr255 structure, which consists of a 16-bit length followed by up to 255 double-byte Unicode characters. HFS+ stores Unicode characters fully decomposed, with the composing characters being in canonical order. When strings containing such characters are exchanged between HFS+ and user space, they are encoded by the kernel as ASCII-compatible UTF-8 bytes (see Figure 1221). hfsdebug can be used to view the Unicode characters stored on disk corresponding to a node name.

$ /bin/zsh $ cd /tmp $ touch `echo '\xe0\xa4\x85\xe0\xa4\xae\xe0\xa4\xbf\xe0\xa4\xa4'` # UTF-8 $ ls -wi # -w forces raw printing of non-ASCII characters ... 3364139  # Terminal.app can display the name $ sudo hfsdebug -c 3364139 # the UTF-8-encoded name can also be used here   <Catalog B-Tree node = 68829 (sector 0x11b588)>   path                 = Macintosh HD:/private/tmp/%0905%092e%093f%0924 ...


Figure 1221. Unicode filenames in Mac OS X


HFS+ uses the : character as a path separator, whereas the Unix APIs use the / character. Since : cannot appear in an HFS+ node name, HFS+ translates any : characters that may appear in a user-provided node name (through a Unix API function, say) to / when storing them to disk. Conversely, when encoding an HFS+ Unicode string as a UTF-8 string for a Unix function, any / characters are translated to : characters.


UTF-8

Unicode is an encoding scheme that maps characters to integers. It is meant to contain characters for all known languages and various categories of symbols. Such a huge character set requires multiple bytes to be represented. Traditionally, operating systems have used single-byte characters. A convenient representation of Unicode on such systems is UTF-8, which is an 8-bit, variable-length encoding scheme. UTF-8 encodes each 7-bit ASCII character as itself in a single byte, whereas non-ASCII characters are encoded as multibyte sequences, with the high-order bits of the first byte indicating the number of bytes that follow. Moreover, UTF-8 preserves the C convention of null-terminated strings.

UTF-8 was created by Ken Thompson and Rob Pike. It was first implemented on Plan 9.


Older Mac OS APIs use file system names consisting of characters encoded using localization-specific Apple-only text encodingsfor example, MacDevanagari, MacGreek, MacJapanese, and MacRoman. For the benefit of these APIs, file and folder records in the Catalog B-Tree contain a hint (the textEncoding field) for name conversions between Unicode and older text encodings. The various conversion tables are loadable, with the exception of tables for conversion between HFS MacRoman and Unicodethese are built into the kernel. The volume header contains a 64-bit encoding bitmap (the encodingsBitmap field) for recording encodings used on the volume. Based on this bitmap, the appropriate encoding tablesif availablemay be loaded by an implementation when a volume is mounted. The directory /System/Library/Filesystems/hfs.fs/Encodings/ contains loadable encodings.

12.8.3. Permissions

HFS+ provides Unix-style file system permissions. Both the HFSPlusCatalogFile and HFSPlusCatalogFolder structures include an HFSPlusBSDInfo structure that encapsulates information related to ownership, permissions, and file type.

struct HFSPlusBSDInfo {     // owner ID 99 ("unknown") is treated as the user ID of the calling     // process (substituted on the fly)     u_int32_t ownerID;     // group ID 99 ("unknown") is treated as the owner ID of the calling     // process (substituted on the fly)     u_int32_t groupID;     // superuser-changeable BSD flags, see chflags(2)     u_int8_t  adminFlags;     // owner-changeable BSD flags, see chflags(2)     u_int8_t ownerFlags;     // file type and permission bits     u_int16_t fileMode;     union {         // indirect inode number for hard links         u_int32_t iNodeNum;         // links that refer to this indirect node         u_int32_t linkCount;         // device number for block/character devices         u_int32_t rawDevice;     } special; };


12.8.3.1. Manipulating Volume-Level Ownership Rights

Although permissions are mandatory on a root volume, they can be deactivated on a nonroot HFS+ volume.

$ hdiutil create -size 32m -fs HFSJ -volname HFSPerms /tmp/hfsperms.dmg ... $ open /tmp/hfsperms.dmg $ touch /Volumes/HFSPerms/file.txt $ chmod 600 /Volumes/HFSPerms/file.txt $ sudo chown root:wheel /Volumes/HFSPerms/file.txt $ ls -l /Volumes/HFSPerms total 0 -rw-------   1 root  wheel  0 Oct 15 10:55 file.txt $ mount -u -o noperm /Volumes/HFSPerms $ ls -l /Volumes/HFSPerms total 0 -rw-------   1 amit  amit  0 Oct 11 10:55 file.txt


Disabling permissions essentially assigns the ownership of the volume's files and folders to a single user IDthe so-called replacement user ID. The replacement user ID can be explicitly specified; otherwise, the kernel will use UNKNOWNUID, the unknown user's ID (99). UNKNOWNUID has the special property that it matches any user ID when IDs are being compared for ownership rights determination.

The replacement is purely behavioral. Each file system object retains its original owner ID. The hfsmount structure holds the replacement ID in memory.

$ hfsdebug /Volumes/HFSPerms/file.txt ...   # BSD Info   ownerID              = 0 (root)   groupID              = 0 (wheel) ... $ sudo hfsdebug -V /Volumes/HFSPerms -m ...   HFS+ flags                              = 00000000000000000000000000001110                                             + HFS_UNKNOWN_PERMS                                             + HFS_WRITEABLE_MEDIA                                             + HFS_CLEANED_ORPHANS   default owner                           = { uid=99, gid=99 } ...


Note that the term permissions really means ownership in this contextthe file mode bits are still honored.

$ chmod 000 /Volumes/HFSPerms/file.txt $ cat /Volumes/HFSPerms/file.txt cat: /Volumes/HFSPerms/file.txt: Permission denied


Figure 1222 shows the algorithm that the Mac OS X HFS+ implementation uses to determine whether a given process has ownership rights to a file system object.

Figure 1222. Algorithm for determining ownership rights to a file system object


12.8.3.2. Repairing Permissions

Applications written using older APIs may disregard (and possibly even clobber) Unix-style permissions. Therefore, a permissions-unaware or misbehaving application can corrupt on-disk permissions if it is run with enough privileges. Mac OS X supports the concept of repairing permissions to address this problem.

Permissions are usually repaired only on a boot volume. The Mac OS X installer uses a bill of materials for each package it installs. A bill-of-materials (bom) file contains a listing of all files within a directory, along with metadata for each file. In particular, it contains each file's Unix permissions. Bom files for installed packages are located within the package metadata[20] found in /Library/Receipts/. Tools that repair permissions use these bom files to determine the original permissions.

[20] For a given package, this metadata is also known as its package receipt.

Let us create a disk image with some files, create a bom file for the disk image, corrupt a file's permissions, and then repair permissions on the volume. Note that we need to make the disk image look like a boot volume to the programs we will use in this experiment.

First we create a disk image, mount it, and ensure that permissions are enabled.

$ hdiutil create -size 32m -fs HFSJ -volname HFSPR /tmp/hfspr.dmg ... $ open /tmp/hfspr.dmg $ mount -u -o perm /Volumes/HFSPR


Next we add certain files to the volume so that the permissions repair tool will run on it.

$ mkdir -p /Volumes/HFSPR/System/Library/CoreServices $ mkdir -p /Volumes/HFSPR/Library/Receipts/BaseSystem.pkg/Contents $ cp /System/Library/CoreServices/SystemVersion.plist \         /Volumes/HFSPR/System/Library/CoreServices/


Then we create a file whose permissions we will repair. We also set the file's permissions to some initial value.

$ touch /Volumes/HFSPR/somefile.txt $ chmod 400 /Volumes/HFSPR/somefile.txt


Next we create a bom file for the disk image. Note that during creation of the bom file, the existing permissions on somefile.txt will be picked up as the correct ones.

$ cd /Volumes/HFSPR/Library/Receipts/BaseSystem.pkg/Contents/ $ sudo mkbom /Volumes/HFSPR Archive.bom


Finally we change the permissions on somefile.txt and run diskutil to repair the volume's permissions.

$ chmod 444 /Volumes/HFSPR/somefile.txt $ sudo diskutil repairPermissions /Volumes/HFSPR Started verify/repair permissions on disk disk10s2 HFSPR Determining correct file permissions. Permissions differ on ./somefile.txt, should be -r-------- , they are -r--r--r-- Owner and group corrected on ./somefile.txt Permissions corrected on ./somefile.txt The privileges have been verified or repaired on the selected volume Verify/repair finished permissions on disk disk10s2 HFSPR $ ls -l /Volumes/HFSPR/somefile.txt -r--------   1 amit  amit  0 Oct 16 12:27 /Volumes/HFSPR/somefile.txt


12.8.4. Journaling

HFS+ supports journaling of metadata, including volume data structures, wherein metadata-related file system changes are recorded to a log file (the journal) that is implemented as a circular on-disk buffer.[21] The primary purpose of a journal is to ensure file system consistency in the case of failure. Certain file system operations are semantically atomic but may result in considerable I/O internally. For example, creating a file, which involves adding the file's thread and file records to the Catalog B-Tree, will cause one or more disk blocks to be written. If the tree needs balancing, several more blocks will be written. If a failure occurs before all changes have been committed to physical storage, the file system will be in an inconsistent stateperhaps even irrecoverably so. Journaling allows related modifications to be grouped into transactions that are recorded in a journal file. Then related modifications can be committed to their final destinations in a transactional mannereither all of them or none at all. Journaling makes it easier and significantly faster to repair the volume after a crash, because only a small amount of informationthat contained in the journalneeds to be examined. Without a journal, the entire volume would typically need to be scanned by fsck_hfs for inconsistencies.

[21] Since the journaling mechanism first writes intended changes to the journal file and then to the actual destination blocks (typically in the buffer cache), it is said to perform write-ahead journaling.

Since writes to files occur independently of the journal, which strives only to keep the metadata consistent, journaling cannot guarantee consistency between a file's metadata and its user data.


Syncing Fully

The journal implementation uses the DKIOCSYNCHRONIZECACHE ioctl operation to flush media state to the drive. This ioctl is also used to implement the F_FULLFSYNC fcntl(2) command, which performs a similar flush operation. More precisely, the flush operation is attemptedit may or may not succeed, depending on whether the underlying device supports and honors the corresponding hardware command. Figure 1223 shows how an F_FULLFSYNC request on an HFS+ file is propagated from user space to an ATA device that supports the FLUSH CACHE command.

Figure 1223. Processing of the F_FULLFSYNC file control operation



Journaling was retrofitted into HFS+ by introducing a VFS-level journaling layer in the kernel [bsd/vfs/vfs_journal.c]. This layer exports an interface that can be used by any file system to incorporate a journal. Figure 1224 shows an overview of the journaling interface and its use by HFS+. Note that from the journal's standpoint, modifications are performed in units of journal block size, which must be specified when the journal is created. HFS+ uses the physical block size (typically 512 bytes for disks) as the journal block size. When HFS+ needs to modify one or more blocks as part of an operation, it starts a journal transaction to encapsulate related changes. Modification of each block is indicated separately to the journal. When all blocks in the transactions have been modified, the file system ends the transaction. When the volume is cleanly unmounted, the transactions recorded in the journal are committed by copying modified blocks from the journal file to their actual on-disk locations, which are also recorded in the journal.

Figure 1224. The VFS-layer journaling interface in Mac OS X


When a journaled volume is mounted, HFS+ checks the lastMountedVersion field of the volume header to determine whether the last mount was performed by a journaling-aware implementation, in which case the field would contain HFSJ, as we saw earlier. If that is the case, and the journal contains uncommitted transactions (because of an unclean shutdown, say), HFS+ will commit transactions recorded in the journal before the volume is availablethat is, the journal will be replayed.

The volume header on a journaled HFS+ volume contains the location of a data structure called the journal info block, which in turn contains the location and size of the journal proper. The latter consists of a header and a circular buffer. Both the info block and the journal are stored as files: .journal_info_block and .journal, respectively. Both these files are contiguous (occupying exactly one extent each). They are normally neither visible nor directly accessible through the file system APIs. The invisibility is implemented inside the kernel, as the file system's catalog-level lookup routine returns an ENOENT error if the file's CNID matches that of one of the journal files. The journal files may be seen, if they exist, through an EFI or Open Firmware shell, for example:

0 > dir hd:\ ...    8388608 10/ 7/ 3  2:11:34            .journal       4096 10/ 7/ 3  2:11:34            .journal_info_block ...


hfsdebug can also retrieve information about the journal files. We specify a journal file to it by providing its name and the CNID of its parent (the root folder).

$ sudo hfsdebug -F 2:.journal <Catalog B-Tree node = 9309 (sector 0x32c88)>   path                 = Macintosh HD:/.journal # Catalog File Record   type                 = file   file ID              = 16 ...   extents              =   startBlock   blockCount      % of file                                 0x4c7       0x1000       100.00 %                          4096 allocation blocks in 1 extents total.                          4096.00 allocation blocks per extent on an average. ...


Figure 1225 shows the structures of the journal files.

Figure 1225. An overview of the file-system-independent journal used by HFS+


We can use hfsdebug to view the contents of the .journal_info_block file and the journal header.

$ sudo hfsdebug -j # HFS+ Journal # Journal Info Block   flags                = 00000000000000000000000000000001                        . Journal resides on local volume itself.   device_signature     = ...   offset               = 5009408 bytes   size                 = 16777216 bytes   reserved             = ... # Journal Header   magic                = 0x4a4e4c78   endian               = 0x12345678   start                = 15369216 bytes   end                  = 677376 bytes   size                 = 16777216 bytes   blhdr_size           = 16384 bytes   checksum             = 0x8787407e   jhdr_size            = 512 bytes


With reference to Figure 1225, the structure of a transaction is as follows. Each transaction consists of one or more block lists. Each block list begins with a block_list_header structure, followed by two or more block_info structures, and finally followed by the actual block dataone chunk for each block_info structure except the first.

typedef struct block_info {     off_t       bnum;  // sector number where data in this block is to be written     size_t      bsize; // number of bytes to be copied from journal buffer to bnum     struct buf *bp;    // used as "next" when on disk } block_info;


The first block_info structure connects two consecutive block lists as part of the same transaction. If the first structure's bp field is 0, the current block list is the last in the current transaction. If the bp field is not 0, then the transaction continues on to the next block list.

12.8.4.1. Enabling or Disabling Journaling on a Volume

The diskutil program can be used to enable or disable journaling on a mounted HFS+ volume. The hfs.util program (/System/Library/Filesystems/hfs.fs/hfs.util) can also be used for this purpose and to display the size and location of the journal file.

$ /System/Library/Filesystems/hfs.fs/hfs.util -I "/Volumes/Macintosh HD" /Volumes/Macintosh HD : journal size 16384 k at offset 0x4c7000


The following sysctl operations, defined in bsd/hfs/hfs_mount.h, allow programmatic manipulation of the journal:

  • HFS_ENABLE_JOURNALING

  • HFS_DISABLE_JOURNALING

  • HFS_GET_JOURNAL_INFO

12.8.4.2. Observing the Journal's Operation

Let us now use hfsdebug to view the contents of the journal buffer and relate them to specific file system operations. We will create a fresh disk image for this purpose.

$ hdiutil create -size 32m -fs HFSJ -volname HFSJ /tmp/hfsj.dmg ... $ open /tmp/hfsj.dmg $ hfsdebug -V /Volumes/HFSJ -J # HFS+ Journal # Journal Buffer # begin transaction   # Block List Header   max_blocks           = 1023   num_blocks           = 5   bytes_used           = 29184   checksum             = 0xfdfd8386   pad                  = 0   binfo[0].bp          = 0      block_info[  1] { bnum 0x0000000000004218 bsize  4096 bytes bp 0x5208ed90 }      block_info[  2] { bnum 0x0000000000004210 bsize  4096 bytes bp 0x52147860 }      block_info[  3] { bnum 0x0000000000000002 bsize   512 bytes bp 0x5208d440 }      block_info[  4] { bnum 0x0000000000000008 bsize  4096 bytes bp 0x520ea6e0 } #end transaction Summary: 5 blocks using 29184 bytes in 1 block lists.


We see that the newly mounted volume has several modified blocks recorded in the journal. Recall that the journal is using disk sectors for blocks. block_info[3]'s target sector is 2, which is the volume header. As we saw earlier, the disk arbitration daemon would have created the .trashes folder when we mounted the newly created volume. The modifications to the Catalog B-Tree and the Allocation file must also be part of the journal records. Let us verify this.

$ hfsdebug -V /Volumes/HFSJ -v ...   blockSize            = 4096 bytes ... # Allocation Bitmap File   logicalSize          = 4096 bytes   totalBlocks          = 1   clumpSize            = 4096 bytes   extents              =   startBlock   blockCount      % of file                                   0x1          0x1       100.00 % ... # Catalog File   logicalSize          = 258048 bytes   totalBlocks          = 63   clumpSize            = 258048 bytes   extents              =   startBlock   blockCount      % of file                                 0x842         0x3f       100.00 % ...


The Allocation file is entirely contained within allocation block number 1that is, it starts at sector 8 and is 4096 bytes in size. Therefore, block_info[4] corresponds to the Allocation file.

block_info[1] and block_info[2] correspond to allocation block numbers 0x843 and 0x842, respectively. (We simply divide the sector numbers by 8, since a 4KB allocation block contains 8 512-byte sectors.) Both these allocation blocks belong to the Catalog file. Since allocation block 0x842 (sector 0x4210) is also the beginning of the Catalog file, it is the location of the tree's header node. hfsdebug displays the sector number where a given Catalog file record's tree node is located. Let us use it to display this information for the .trashes folder.

$ hfsdebug -V /Volumes/HFSJ/.Trashes <Catalog B-Tree node = 1 (sector 0x4218)>   path                 = HFSJ:/.Trashes # Catalog Folder Record ...


Thus, all records in the journal are accounted for.

12.8.5. Quotas

HFS+ supports volume-level quotas based on user and group IDs. It theoretically supports quotas based on other criteria, since the in-memory catalog-node structure (struct cnode [bsd/hfs/hfs_cnode.h]) contains an array of disk quota usage records (struct dquot [bsd/sys/quota.h]). The array contains two elements in Mac OS X 10.4, one for user quotas and one for group quotas. The corresponding quota filenames are .quota.user and .quota.group. These files reside in the file system's root directory. Each file contains a header followed by a hash table of structures specifying various quota limits and usage values for user or group IDs. These IDs are hashed to yield offsets into the quota hash tables.

We can enable user (group) quotas on a volume by creating an empty mount options file named .quota.ops.user (.quota.ops.group) in the volume's root directory. The presence of this file would cause user (group) quotas to be enabled at mount time, provided a .quota.user (.quota.group) file also exists. The latter file is created by running the quotacheck program.

Let us create an HFS+ disk image and enable quotas on it. By default, a Mac OS X client installation does not have quotas enabled. You can use the quota or repquota commands to view quota information for a file system.

$ sudo repquota -a $ hdiutil create -size 32m -fs HFSJ -volname HFSQ /tmp/hfsq.dmg $ open /tmp/hfsq.dmg $ mount -u -o perm,uid=99 /Volumes/HFSQ $ sudo touch /Volumes/HFSQ/.quota.ops.user /Volumes/HFSQ/.quota.ops.group $ sudo quotacheck -ug /Volumes/HFSQ quotacheck: creating quota file /Volumes/HFSQ/.quota.user quotacheck: creating quota file /Volumes/HFSQ/.quota.group


We can now turn quotas on using the quotaon command.

$ sudo quotaon -ug /Volumes/HFSQ $ sudo repquota -u /Volumes/HFSQ                     1K Block limits                      File limits User            used        soft      hard  grace    used  soft  hard  grace amit      --       8           0         0              4     0     0


We see that the user has already consumed a few inodes (because of .DS_Store and such). We can edit a user's quota values using the edquota command, which will launch the text editor specified by the EDITOR environment variable (or vi, if EDITOR is not set).

$ sudo edquota -u amit Quotas for user amit: /Volumes/HFSQ: 1K blocks in use: 8, limits (soft = 0, hard = 0)         inodes in use: 4, limits (soft = 4, hard = 4)


We change the soft and hard limits and save the file, after which there is a limit on the total number of files and folders the user can have on this volume.

$ sudo repquota -u /Volumes/HFSQ                     1K Block limits                      File limits User            used        soft      hard  grace    used  soft  hard  grace amit      -+       8           0         0              4     4     4


repquota reports the updated quota limits for user amit. Let us attempt to exceed this limit.

$ touch /Volumes/HFSQ/file.txt touch: /Volumes/HFSQ/file.txt: Disc quota exceeded


12.8.6. Hard Links

On typical Unix file systems, each file has an associated link count representing the number of physical references to it. Suppose a file foo has a link count of 1. If you make a hard link bar to it (using the ln command or the link() system call), the following statements will apply to the two files.

  • foo and bar are two different pathnameswith corresponding directory entriesthat refer to the same physical file on disk. The links are equivalent in all respects. The stat() system call will return the same inode numbers for foo and bar.

  • The link count of the file becomes 2. Since foo and bar are equivalent, the link counts of both files will be reported as 2.

  • If you remove either foo or bar, the link count will decrease by 1. As long as the link count is greater than 0, the physical file will not be deleted.

We can consider a hard link to be simply another directory entry for an existing file. Normally, only files are allowed to have hard links, as hard links to folders can create cycles in the folder hierarchywith highly undesirable results. Hard links may also not span across file systems.

Early versions of UNIX allowed the superuser to create a hard link to a directory.


Hard links to a file on HFS+ are conceptually similar to those on Unix systems: They represent multiple directory entries referring to common file content. The Mac OS X implementation of HFS+ hard links uses a special hard-link file for each directory entry. The common file content is stored in another special file: the indirect-node file.

The linkCount field in the HFSPlusBSDInfo structure, which we saw in Section 12.8.3, holds the link count for a file. A folder also has a link count that represents the number of its directory entries. However, a folder's HFSPlusBSDInfo structure does not hold the folder's link count in its linkCount fieldthe valence field of the HFSPlusCatalogFolder structure does. A folder's link count value, as reported by the Unix APIs, is two more than the on-disk value because of the . and .. directory entries, which are dummy entries on HFS+.


A hard-link file has a file type hlnk and a creator code hfs+. It is otherwise an ordinary file in the Catalog B-Tree. All indirect-node files are stored in the private metadata folder, which resides in the file system's root directory. When an HFS+ volume is mounted, the kernel checks for the existence of this folder, creating it if it doesn't exist. Several measures such as the following are taken to make the metadata folder tamper-resistant.

  • Its name is four null characters (NUL) followed by the string "HFS+ Private Data".

  • Its permissions are set to 000 by defaultthat is, no read, write, or execute access to anyone.

  • It is set to be invisible in the Finder through the kIsInvisible Finder flag.

  • Its kNameLocked Finder flag is set so that it cannot be renamed, nor can its icon be changed, from the Finder.

  • Its icon location is set to (22460, 22460) in its Finder information.

The folder may be seen[22] from Open Firmware, where it sorts last in the output of the dir command.

[22] However, the folder is not visible from an EFI shell.

0 > dir hd:\ ...    10/ 7/ 3  2: 7:21   %00%00%00%00HFS+%20Private%20Data


hfsdebug can be used to display the properties of this folder and those of its contents (Figure 1226 shows an example). Since the folder is created before user files are created, it will typically have a low CNID. Given that the first user-available CNID is 16, the metadata folder is likely to have the CNID 18 on a journaled volume, since CNIDs 16 and 17 would have been taken by the two journal files.

Figure 1226. Examining the HFS+ private metadata folder and hard-link creation

$ hdiutil create -size 32m -fs HFSJ -volname HFSLink /tmp/hfslink.dmg $ open /tmp/hfslink.dmg $ hfsdebug -V /Volumes/HFSLink -c 18   <Catalog B-Tree node = 1 (sector 0x4218)>   path                 = HFSLink:/%00%00%00%00HFS+ Private Data # Catalog Folder Record ...   # BSD Info   ownerID              = 0 (root)   groupID              = 0 (wheel)   adminFlags           = 00000000   ownerFlags           = 00000000   fileMode             = d--------- ...   frFlags              = 0101000000000000                        . kNameLocked                        . kIsInvisible   frLocation           = (v = 22460, h = 22460) ... $ cd /Volumes/HFSLink $ touch file.txt $ ls -i file.txt # note the inode number 22 file.txt $ ln file.txt link.txt $ sudo hfsdebug link.txt   <Catalog B-Tree node = 1 (sector 0x4218)>   path                 = HFSLink:/link.txt # Catalog File Record   type                 = file (hard link)   file ID              = 24   flags                = 0000000000000010                        . File has a thread record in the catalog. ...   # BSD Info   ownerID              = 0 (root)   groupID              = 0 (wheel)   adminFlags           = 00000000   ownerFlags           = 00000000   fileMode             = ----------   iNodeNum             = 22 (link reference number) ...   # Finder Info   fdType               = 0x686c6e6b (hlnk)   fdCreator            = 0x6866732b (hfs+) ... $ ls -l link.txt -rw-r--r--   2 amit  amit  0 Oct 12 05:12 link.txt

It is also possible for the superuser to change directory to the private metadata folder (from a shell, say), provided its pathname is passed to the cd command appropriately. The problem is that the folder's name begins with NUL characters, which terminate C-style strings. We can use the NUL character's UTF-8 representation, which is the following byte sequence: 0xe2, 0x90, 0x80.

$ sudo /bin/zsh # cd /Volumes/HFSLink # cd "`echo '\xE2\x90\x80\xE2\x90\x80\xE2\x90\x80\xE2\x90\x80HFS+ Private Data'`" # ls -l iNode22


We see that the metadata folder on our example volume contains a single file named iNode22. We also see that 22 is the link reference number reported for the hard-link file (link.txt) we created in Figure 1226iNode22 is the indirect-node file for the hard link in question. An interesting observation is that the ownership and file mode details for link.txt in Figure 1226 do not match between hfsdebug and the ls command. This is because link.txt is a kernel-owned reference to iNode22, which holds the original contents (along with the original ownership and file mode information) of the hard link's target file.txt. In fact, file.txt is also a hard-link file now and has properties similar to those of link.txt.

$ cd /Volumes/HFSLink $ hfsdebug file.txt ... # Catalog File Record   type                 = file (hard link)   indirect node file   = HFSLink:/%00%00%00%00HFS+ Private Data/iNode22   file ID              = 23 ...


HFS+ indirect-node files are always named iNode<LRN>, where <LRN> is a link reference number represented in decimal. Link reference numbers are randomly generated. They are unique on a given volume but are unrelated to CNIDs.


Note that the CNIDs of file.txt and link.txt are 23 and 24, respectively. Since hard-link semantics require that all hard links to a given file have the same inode numbers, in this case, HFS+ does not report the hard-link files' CNIDs as their respective inode numbers. Instead, it reports the indirect-node file's CNID, which is the same as the original CNID of file.txt, as the inode numbers of both hard-link files.

$ ls -i file.txt link.txt 22 file.txt    22 link.txt


We can now summarize the process of hard-link creation on HFS+ as follows. When the first hard link is created to a file, its link count goes from 1 to 2. Moreover, the file's content is moved to the private metadata folder as an indirect-node file, which retains the CNID and other properties of the original file. Two new entries are created in the catalog: one for the original pathname and the other for the newly created hard link. The "original" notwithstanding, both are brand-new entrieshard-link filesthat serve as references to the indirect-node file. Both have the file type hlnk and the creator code hfs+. Although they have their own CNIDs that are unrelated to the original file's CNID, the stat() system call still reports the latter as the inode number of each hard-link file. When a user accesses a hard-link file, HFS+ automatically follows it so that the user actually accesses the indirect-node file.

12.8.7. Unlinking Open Files

Whereas Carbon semantics prohibit deletion of open files, POSIX semantics do not. HFS+ supports both behaviors: The delete() and unlink() system calls provide Carbon and POSIX semantics, respectively. The private metadata folder is used to store files that are unlinked while they are still open, or busy. Such files are renamed and moved to the metadata folder, where they are stored temporarilyat least until they are closed. If the busy file being unlinked has multiple nonzero forks, any forks that are not busy are truncated.

We can observe this behavior by listing the contents of the private metadata folder after using the rm command to remove a busy filesay, one that we are viewing using the less command. Figure 1227 shows this experiment.

Figure 1227. Use of the private metadata folder for storing unlinked busy files

$ sudo /bin/zsh # cd /Volumes/HFSLink # cd "`echo '\xE2\x90\x80\xE2\x90\x80\xE2\x90\x80\xE2\x90\x80HFS+ Private Data'`" # echo hello > /Volumes/HFSLink/busyunlink.txt # hfsdebug /Volumes/HFSLink/busyunlink.txt ...   file ID              = 27 ...   extents              =   startBlock   blockCount      % of file                                 0xaf9          0x1       100.00 % ... # less /Volumes/HFSLink/busyunlink.txt hello /Volumes/HFSLink/busyunlink.txt lines 1-1/1 (END) ^z zsh: suspended  less /Volumes/HFSLink/busyunlink.txt # rm /Volumes/HFSUnlink/busyunlink.txt # ls iNode22 temp27 # cat temp27 hello # hfsdebug temp27 ...   file ID              = 27 ...   extents              =   startBlock   blockCount      % of file                                 0xaf9          0x1       100.00 % ...

As seen in Figure 1227, a temporary file appears in the private metadata folder after a busy file is unlinked. The file is a moved version of the original file, with any nonbusy forks truncated.

It is possible for the "temp" files in the private metadata folder to persist across a reboot if the volume was not cleanly unmounted. Such files are called orphaned files. They are removed when the volume is mounted next.


12.8.8. Symbolic Links

A symbolic link (or symlink) is a file system entity that refers to another file or folder by relative or absolute pathname. The following are some important properties of symbolic links.

  • Unlike the case of a hard link, a symbolic link's target may reside on a different file system or may not even exist.

  • Unlike HFS+ aliases (see Section 12.8.9), if a symbolic link's target is renamed or deleted, the symbolic link is not updated in any wayit is broken.

  • Most file operations on a symbolic link are forwarded to its target. Some system calls have special versions that operate on symbolic links themselves, rather than their targets.

  • The ownership and file mode of a symbolic link file are unrelated to its target. Although a symbolic link's ownership can be changed through the lchown() system call, there is no analogous call to change the file mode of a symbolic link.

  • Symbolic links can easily lead to cycles, for example:

    $ ln -s a b $ ln -s b a $ cat a cat: a: Too many levels of symbolic links

HFS+ implements symbolic links as normal files whose data forks contain the UTF-8-encoded pathnames of their targets. A symbolic link file's resource fork is empty. Moreover, the file's type and creator code are slnk and rhap, respectively.

$ cd /Volumes/HFSLink $ echo hello > target.txt $ ln -s target.txt symlink.txt $ hfsdebug symlink.txt ... # Catalog File Record   type                 = file (symbolic link)   linkTarget           = target.txt ...   # Finder Info   fdType               = 0x736c6e6b (slnk)   fdCreator            = 0x72686170 (rhap) ...


You can even synthesize your own symbolic links manually by simply setting the file type and creator code of a file. If the file contains a valid pathname, it will be a working symbolic link.

$ cd /tmp $ echo hello > target.txt $ echo -n /tmp/a > symlink.txt $ ls -l /tmp/symlink.txt -rw-r--r--   1 amit  wheel  15 Oct 12 07:25 /tmp/symlink.txt $ /Developer/Tools/SetFile -t slnk -c rhap /tmp/a $ ls -l /tmp/symlink.txt lrwxr-xr-x   1 amit  wheel  15 Oct 12 07:25 /tmp/symlink.txt -> /tmp/target.txt $ cat /tmp/symlink.txt hello


The SetFile program is part of Apple Developer Tools. Alternatively, you can set a file's type and creator code by setting the com.apple.FinderInfo pseudo extended attribute or by using the FSSetCatalogInfo() Carbon function.


12.8.9. Aliases

Aliases, which are supported by both HFS and HFS+, are lightweight references to files and folders. An alias has semantics similar to that of a symbolic link, except that it fares better when the link target is moved: It has the special property that moving its target on the volume does not break the alias, whereas a symbolic link would break if its target were moved.

The resource fork of an alias file is used to track the alias target by storing both the pathname and the CNID of the target. The CNID works as a unique, persistent identity that will not change when the target is moved. When an alias is accessed, it can withstand the staleness of one of the two references (pathname or unique identity). If one of the two is wrong in that the target cannot be found using it, the alias is updated with the correct one (using which the target could be found). This feature is the reason why it is possible to rename applications or move them to different places on a volume without breaking their Dock shortcuts.

An alias is described by an alias-record data structure. An alias's target may be a file, directory, or volume. Besides the target's location, an alias record contains some other information such as creation date, file type, creator code, and possibly volume mounting information.

To make use of aliases, an application must use either the Carbon API or the Cocoa APIaliases are not available through the Unix API. On the other hand, although the Finder presents aliases and symbolic links similarly to the user, it allows creation of aliases only through its user interface. Symbolic links must be created by using the ln command or programmatically through the Unix API.


Figure 1228 shows a Python program that resolves an alias and prints the pathname of its target.

Figure 1228. A Python program to resolve an alias

#! /usr/bin/python # ResolveAlias.py import sys import Carbon.File def main():     if len(sys.argv) != 2:         sys.stderr.write("usage: ResolveAlias <alias path>\n")         return 1     try:         fsspec, isfolder, aliased = \             Carbon.File.ResolveAliasFile(sys.argv[1], 0)     except:         raise "No such file or directory."     print fsspec.as_pathname()     return 0 if __name__ == "__main__":     sys.exit(main()) $ ResolveAlias.py "/User Guides And Information" /Library/Documentation/User Guides And Information.localized

12.8.10. Resource Forks

Historically, resource forks on HFS and HFS+ file systems have been used to hold resources. For an application, resources might include custom icons, menus, dialog boxes, the application's executable code and runtime memory requirements, license information, and arbitrary key-value pairs. For a document, resources might include fonts and icons used by the document, preview pictures, preferences, and window locations to use while opening the document. A resource fork is usually structured in that there is a map describing resources that follow it. There are practical limits on the number of resources you could put in a resource fork. In contrast, a data fork is unstructuredit simply contains the file's data bytes.

By default, the Unix API on Mac OS X accesses a file's data fork. It is, however, possible to access a resource fork through the Unix API by using the special suffix /..namedfork/rsrc after the file's pathname.

$ cd /System/Library/CoreServices/ $ ls -l System -rw-r--r--   1 root  wheel  0 Mar 20  2005 System $ ls -l System/..namedfork/rsrc 504 -rw-r--r--   1 root  wheel  256031 Mar 20  2005 System/rsrc


The shortened suffix /rsrc can also be used to access the resource fork, although it is deemed a legacy suffix and is deprecated in Mac OS X 10.4.


An HFS+ file with multiple nonzero forks is not a single stream of bytes and is therefore incompatible with most other file systems. Care must be taken while transferring HFS+ files to other file systems. Before Mac OS X 10.4, most standard Unix utilities on Mac OS X either didn't handle multiple forks at all or handled them poorly. Mac OS X 10.4 has better command-line support for multiple forksstandard tools such as cp, mv, and tar handle multiple forks and extended attributes, including when the destination file system does not support these features. These programs rely on the copyfile() function, whose purpose is to create faithful copies of HFS+ file system objects. copyfile() simulates multiple forks on certain file systems that do not support them. It does so by using two files for each file: one containing the data fork and the other containing the resource fork and attributes, flattened and concatenated. The second file's name is the prefix ._ followed by the first file's name. This scheme of storing multiple forks is known as the AppleDouble format.

The SplitFork command can be used to convert a two-fork file into AppleDouble format. Conversely, the FixupResourceForks command can be used to combine AppleDouble files into two-fork resource files.


/usr/bin/ditto can be used to copy files and directories while preserving resource forks and other metadata. If the destination file system does not have native support for multiple forks, ditto will store this data in additional files. ditto can also be used to create PKZip archives with flattened resource forks, in which case it will keep resource forks and other metadata in a directory called __MACOSX within the PKZip archive.

$ cd /tmp $ touch file $ echo 1234 > file/..namedfork/rsrc $ ls -l file -rw-r--r--  1 amit  wheel  0 24 Apr 15:56 file $ ls -l file/..namedfork/rsrc -rw-r--r--  1 amit  wheel  5 24 Apr 15:56 file/..namedfork/rsrc $ ditto -c -k -sequesterRsrc file file.zip $ unzip file.zip Archive:  file.zip  extracting: file    creating: __MACOSX/   inflating: __MACOSX/._file $ cat __MACOSX/._file         2 R1234


The original file can be recreated from the PKZip archive using ditto.

% rm -rf file __MACOSX % ditto -x -k -sequesterRsrc file.zip . % ls -l file -rw-r--r--  1 amit  wheel  0 24 Apr 15:56 file % ls -l file/rsrc -rw-r--r--  1 amit  wheel  5 24 Apr 15:56 file/rsrc





Mac OS X Internals. A Systems Approach
Mac OS X Internals: A Systems Approach
ISBN: 0321278542
EAN: 2147483647
Year: 2006
Pages: 161
Authors: Amit Singh

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net