Basics of Asynchronous Device IO

[Previous] [Next]

Compared to most other operations carried out by a computer, device I/O is one of the slowest and most unpredictable. The CPU performs arithmetic operations and even paints the screen much faster than it reads data from or writes data to a file or across a network. However, using asynchronous device I/O enables you to better utilize resources and thus create more efficient applications.

Consider a thread that issues an asynchronous I/O request to a device. This I/O request is passed to a device driver, which assumes the responsibility of actually performing the I/O. While the device driver waits for the device to respond, the application's thread is not suspended as it waits for the I/O request to complete. Instead, this thread continues executing and performs other useful tasks.

At some point, the device driver finishes processing the queued I/O request and must notify the application that data has been sent, data has been received, or an error has occurred. You'll learn how the device driver notifies you of I/O completions in the next section, "Receiving Completed I/O Request Notifications." For now, let's concentrate on how to queue asynchronous I/O requests. Queuing asynchronous I/O requests is the essence of designing a high-performance, scalable application, and it is what the remainder of this chapter is all about.

To access a device asynchronously, you must first open the device by calling CreateFile, specifying the FILE_FLAG_OVERLAPPED flag in the dwFlagsAndAttrs parameter. This flag notifies the system that you intend to access the device asynchronously.

To queue an I/O request for a device driver, you use the ReadFile and WriteFile functions that you already learned about in the section "Performing Synchronous Device I/O." For convenience, I'll list the function prototypes again:

BOOL ReadFile(    HANDLE      hfile,     PVOID       pvBuffer,    DWORD       nNumBytesToRead,     PDWORD      pdwNumBytes,    OVERLAPPED* pOverlapped); BOOL WriteFile(    HANDLE      hfile,     CONST VOID  *pvBuffer,    DWORD       nNumBytesToWrite,     PDWORD      pdwNumBytes,    OVERLAPPED* pOverlapped); 

When either of these functions is called, the function checks to see if the device, identified by the hfile parameter, was opened with the FILE_ FLAG_OVERLAPPED flag. If this flag is specified, the function performs asynchronous device I/O. By the way, when calling either function for asynchronous I/O, you can (and usually do) pass NULL for the pdwNumBytes parameter. After all, you expect these functions to return before the I/O request has completed, so examining the number of bytes transferred is meaningless at this time.

The OVERLAPPED Structure

When performing asynchronous device I/O, you must pass the address to an initialized OVERLAPPED structure via the pOverlapped parameter. The word "overlapped" in this context means that the time spent performing the I/O request overlaps the time your thread spends performing other tasks. Here's what an OVERLAPPED structure looks like:

typedef struct _OVERLAPPED {    DWORD  Internal;     // [out] Error code    DWORD  InternalHigh; // [out] Number of bytes transferred    DWORD  Offset;       // [in]  Low  32-bit file offset    DWORD  OffsetHigh;   // [in]  High 32-bit file offset    HANDLE hEvent;       // [in]  Event handle or data } OVERLAPPED, *LPOVERLAPPED; 

This structure contains five members. Three of these members, Offset, OffsetHigh, and hEvent, must be initialized prior to calling ReadFile or WriteFile. The other two members, Internal and InternalHigh, are set by the device driver and can be examined when the I/O operation completes. Here is a more detailed explanation of these member variables:

• Offset and OffsetHigh When a file is being accessed, these members indicate the 64-bit offset in the file where you want the I/O operation to begin. Recall that each file kernel object has a file pointer associated with it. When issuing a synchronous I/O request, the system knows to start accessing the file at the location identified by the file pointer. After the operation is complete, the system updates the file pointer automatically so that the next operation can pick up where the last operation left off.

When performing asynchronous I/O, this file pointer is ignored by the system. Imagine what would happen if your code placed two asynchronous calls to ReadFile (for the same file kernel object). In this scenario, the system wouldn't know where to start reading for the second call to ReadFile. You probably wouldn't want to start reading the file at the same location used by the first call to ReadFile. You might want to start the second read at the byte in the file that followed the last byte that was read by the first call to ReadFile. To avoid the confusion of multiple asynchronous calls to the same object, all asynchronous I/O requests must specify the starting file offset in the OVERLAPPED structure.

Note that the Offset and OffsetHigh members are not ignored for nonfile devices—you must initialize both members to 0 or the I/O request will fail and GetLastError will return ERROR_INVALID_PARAMETER.

• hEvent This member is used by one of the four methods available for receiving I/O completion notifications. When using the alertable I/O notification method, this member can be used for your own purposes. I know many developers who store the address of a C++ object in hEvent. (This member will be discussed more in the section "Signaling an Event Kernel Object.")
• Internal This member holds the processed I/O's error code. As soon as you issue an asynchronous I/O request, the device driver sets Internal to STATUS_PENDING, indicating that no error has occurred because the operation has not started. In fact, the macro HasOverlappedIoCompleted, which is defined in WinBase.h, allows you to check whether an asynchronous I/O operation has completed. If the request is still pending, FALSE is returned; if the I/O request is completed, TRUE is returned. Here is the macro's definition:

#define HasOverlappedIoCompleted(pOverlapped) \    ((pOverlapped)->Internal != STATUS_PENDING)

• InternalHigh When an asynchronous I/O request completes, this member holds the number of bytes transferred.

When first designing the OVERLAPPED structure, Microsoft decided not to document the Internal and InternalHigh members (which explains their names). As time went on, Microsoft realized that the information contained in these members would be useful to developers and so documented them. However, Microsoft didn't change the names of the members because the operating system source code referenced them frequently, and Microsoft didn't want to modify the code.

NOTE

---

When an asynchronous I/O request completes, you will receive the address of the OVERLAPPED structure that was used when the request was initiated. Having more contextual information passed around with an OVERLAPPED structure is frequently useful—for example, if you wanted to store the handle of the device used to initiate the I/O request inside the OVERLAPPED structure. The OVERLAPPED structure doesn't offer a device handle member or other potentially useful members for storing context, but you can solve this problem quite easily.

I frequently create a C++ class that is derived from an OVERLAPPED structure. This C++ class can have any additional information in it that I want. When my application receives the address of an OVERLAPPED structure, I simply cast the address to a pointer of my C++ class. Now I have access to the OVERLAPPED members and any additional context information my application needs. The FileCopy sample application at the end of this chapter demonstrates this technique. See my CIOReq C++ class in the FileCopy sample application for the details.

Asynchronous Device I/O Caveats

You should be aware of a couple of issues when performing asynchronous I/O. First, the device driver doesn't have to process queued I/O requests in a first-in first-out (FIFO) fashion. For example, if a thread executes the following code, the device driver will quite possibly write to the file and then read from the file:

OVERLAPPED o1 = { 0 }; OVERLAPPED o2 = { 0 }; BYTE bBuffer[100]; ReadFile (hfile, bBuffer, 100, NULL, &o1); WriteFile(hfile, bBuffer, 100, NULL, &o2); 

A device driver typically executes I/O requests out of order if doing so helps performance. For example, to reduce head movement and seek times, a file system driver might scan the queued I/O request list looking for requests that are near the same physical location on the hard drive.

The second issue you should be aware of is the proper way to perform error checking. Most Windows functions return FALSE to indicate failure or nonzero to indicate success. However, the ReadFile and WriteFile functions behave a little differently. An example might help to explain.

When attempting to queue an asynchronous I/O request, the device driver might choose to process the request synchronously. This can occur if you're reading from a file and the system checks whether the data you want is already in the system's cache. If the data is available, your I/O request is not queued to the device driver; instead, the system copies the data from the cache to your buffer, and the I/O operation is complete.

ReadFile and WriteFile return a nonzero value if the requested I/O was performed synchronously. If the requested I/O is executing asynchronously, or if an error occurred while calling ReadFile or WriteFile, FALSE is returned. When FALSE is returned, you must call GetLastError to determine specifically what happened. If GetLastError returns ERROR_IO_PENDING, the I/O request was successfully queued and will complete later.

If GetLastError returns a value other than ERROR_IO_PENDING, the I/O request could not be queued to the device driver. Here are the most common error codes returned from GetLastError when an I/O request can't be queued to the device driver:

• ERROR_INVALID_USER_BUFFER or ERROR_NOT_ENOUGH_MEMORY Each device driver maintains a fixed-size list (in a nonpaged pool) of outstanding I/O requests. If this list is full, the system can't queue your request, ReadFile and WriteFile return FALSE, and GetLastError reports one of these two error codes (depending on the driver).
• ERROR_NOT_ENOUGH_QUOTA Certain devices require that your data buffer's storage be page locked so that the data cannot be swapped out of RAM while the I/O is pending. This page-locked storage requirement is certainly true of file I/O when using the FILE_FLAG_NO_BUFFERING flag. However, the system restricts the amount of storage that a single process can page lock. If ReadFile and WriteFile cannot page lock your buffer's storage, the functions return FALSE, and GetLastError reports ERROR_NOT_ENOUGH_QUOTA. You can increase a process's quota by calling SetProcessWorkingSetSize.

How should you handle these errors? Basically, these errors occur because a number of outstanding I/O requests have not yet completed, so you need to allow some pending I/O requests to complete and then reissue the calls to ReadFile and WriteFile.

The third issue you should be aware of is that the data buffer and OVERLAPPED structure used to issue the asynchronous I/O request must not be moved or destroyed until the I/O request has completed. When queuing an I/O request to a device driver, the driver is passed the address of the data buffer and the address of the OVERLAPPED structure. Notice that just the address is passed, not the actual block. The reason for this should be quite obvious: memory copies are very expensive and waste a lot of CPU time.

When the device driver is ready to process your queued request, it transfers the data referenced by the pvBuffer address, and it accesses the file's offset member and other members contained within the OVERLAPPED structure pointed to by the pOverlapped parameter. Specifically, the device driver updates the Internal member with the I/O's error code and the InternalHigh member with the number of bytes transferred.

NOTE

---

It is absolutely essential that these buffers not be moved or destroyed until the I/O request has completed, or memory will be corrupted. Also, you must allocate and initialize a unique OVERLAPPED structure for each I/O request.

The preceding note is very important and is one of the most common bugs developers introduce when implementing an asynchronous device I/O architecture. Here's an example of what not to do:

VOID ReadData(HANDLE hfile) {    OVERLAPPED o = { 0 };    BYTE b[100];    ReadFile(hfile, b, 100, NULL, &o); } 

This code looks fairly harmless, and the call to ReadFile is perfect. The only problem is that the function returns after queuing the asynchronous I/O request. Returning from the function essentially frees the buffer and the OVERLAPPED structure from the thread's stack, but the device driver is not aware that ReadData returned. The device driver still has two memory addresses that point to the thread's stack. When the I/O completes, the device driver is going to modify memory on the thread's stack, corrupting whatever happens to be occupying that spot in memory at the time. This bug is particularly difficult to find because the memory modification occurs asynchronously. Sometimes the device driver might perform I/O synchronously, in which case you won't see the bug. Sometimes the I/O might complete right after the function returns, or it might complete over an hour later, and who knows what the stack is being used for then?

Canceling Queued Device I/O Requests

Sometimes you might want to cancel a queued device I/O request before the device driver has processed it. Windows offers a few ways to do this:

• You can call CancelIo to cancel all I/O requests queued by the calling thread for the specified handle:

BOOL CancelIo(HANDLE hfile);

• You can cancel all queued I/O requests, regardless of which thread queued the request, by closing the handle to a device itself.
• When a thread dies, the system automatically cancels all I/O requests issued by the thread.

As you can see, there is no way to cancel a single, specific I/O request.

NOTE

---

Canceled I/O requests complete with an error code of ERROR_OPERATION_ABORTED.