13.7 Direct IO

13.7 Direct I/O

Direct I/O was first introduced in V5.0A^[5]. When a file is opened with the Direct I/O flag, all I/O performed on the file will bypass the Unified Buffer Cache (UBC). To enable Direct I/O on a file, the file must be opened with the open(2) system call using the O_DIRECTIO flag as the mode.

Notes

• Direct I/O can be enabled only for files on an AdvFS file system.

• Direct I/O can be enabled only on a file and not on an entire file system.

• You cannot mmap(2) a file opened with Direct I/O.

Figure 13-2 illustrates how I/O flows with Direct I/O enabled. Notice that although the I/O is "direct", the term does not imply that the I/O goes directly to disk, only that it goes directly to the CFS server (although if the CFS server is the member performing Direct I/O then the I/O will go directly to disk, bypassing the UBC).

Figure 13-2: Direct I/O in V5.0A

While Direct I/O can be a performance enhancement for an application that already uses its own caching mechanism (like a database), this is generally not considered a performance enhancement and can in fact cause performance degradation because writing to a device is much slower than writing to memory.

For Direct I/O that bypasses the UBC and goes directly to disk, you want Concurrent Direct I/O.

13.7.1 Concurrent Direct I/O (CDIO)

Concurrent Direct I/O was introduced in V5.1^[6] and enables members with direct access to the device where the file lives not only to bypass the UBC but also bypass the CFS server for I/O not related to log updates and metadata operations (file size changes, etc.). Those writes that allocate storage will continue to be sent to the CFS server.

It is also worth noting that CDIO is truly concurrent – all members with direct physical connectivity to the shared storage devices can read and write to the file systems simultaneously.

Figure 13-3 illustrates the I/O flow using CDIO.

Figure 13-3: Concurrent Direct I/O

The cluster keeps track of Concurrent Direct I/O operations performed on the cluster, and these statistics can be retrieved using the cfsstat command with the "directio" option.

For additional information on Direct I/O, see the TruCluster Server Cluster Administration Guide and the open(2) reference page.

^[5]For the TruCluster Server product. Tru64 UNIX added support for Direct I/O in the base operating system in V5.0.

^[6]For the TruCluster Server product. Tru64 UNIX added support for CDIO in the base operating system in V5.0.