Synchronization Scope and IO Callback Serialization


Synchronization Scope and I/O Callback Serialization

Concurrently active callbacks on the same object might require access to shared object-specific data or might share such data with helper functions. For example, a driver's cleanup and cancellation callbacks often use the same data as its read, write, and device I/O control callbacks. A driver can set synchronization scope to manage concurrency among these functions.

Important 

Neither UMDF nor KMDF serializes calls to I/O completion callbacks. Such callbacks can run concurrently with any other driver callbacks. If an I/O completion callback shares data with another I/O event callback and requires synchronized access to that data, the driver must explicitly acquire the appropriate locks.

Synchronization scope determines the level in the object hierarchy at which the framework acquires an object presentation lock. This, in turn, controls whether the framework calls certain I/O event callback functions concurrently or serially. The I/O event callbacks that are serialized depend on the synchronization scope and the framework (that is, UMDF or KMDF).

WDF defines the following three synchronization scopes:

  • No scope

    No scope means that WDF does not acquire an object presentation lock and therefore can call any I/O event callback concurrently with any other event callback. No scope is the default for KMDF. A KMDF driver can "opt in" to serialization for a device or queue object by setting device scope or queue scope when it creates the object.

  • Device scope

    Device scope means that WDF acquires the device object presentation lock before calling certain I/O event callbacks. Therefore, device scope results in serialized calls to the I/O event callbacks for an individual device object or for any file objects or queue objects that are its children. UMDF uses device scope by default.

  • Queue scope

    Queue scope means that WDF acquires a queue object presentation lock, and thus the I/O event callbacks are serialized on a per-queue basis. A KMDF driver can specify queue scope for a device object so that the I/O event callbacks for each queue are serialized but the callbacks for different queues can run concurrently.

    The initial UMDF release does not support queue scope.

Important 

UMDF and KMDF have different defaults for synchronization scope. For UMDF drivers, device scope is the default. For KMDF drivers, no scope is the default.

Device Scope and Queue Dispatch Methods

The combination of synchronization scope and queue dispatch method provides great flexibility for the control of I/O requests through your driver.

Figure 10-4 shows the objects that are affected if the driver sets device scope for a device object.

image from book
Figure 10-4: Device synchronization scope

In the figure, a driver object has a single device object and the device object has two child queues and a child file. The read/write queue has a child request object. If the driver sets device synchronization scope on the device object, by default certain callbacks of the device object, the queue objects, the request, and the file would be serialized.

To examine how the synchronization scope and queue dispatch method interact, consider a hypothetical I/O sequence. Assume that the driver in Figure 10-4 configures the read/write queue for parallel dispatching of read and write requests and device I/O control queue for sequential dispatching of device I/O control requests. The following list shows the framework's actions as I/O requests arrive for the driver.

Event

Framework action

Read request 1 arrives.

Acquire lock and call I/O read callback with read request 1.

Device I/O control request 1 arrives.

None

Device I/O control request 2 arrives.

None

I/O read callback returns.

Release lock. Acquire lock and call device I/O control callback with device I/O control request 1.

Read request 2 arrives.

None

Write request arrives.

None

Device I/O control callback sends device I/O control request 1 to I/O target and then returns.

Release lock. Acquire lock and call I/O read callback with read request 2.

I/O read callback returns.

Release lock. Acquire lock and call I/O write callback with write request.

When the first read request arrives, the framework acquires the device presentation lock and dispatches the request to the queue's read callback function. Meanwhile, two device I/O control requests arrive. The framework adds them to the device I/O control queue but does not invoke a callback-even though no other requests from this queue are active-because the device object presentation lock prevents additional callbacks until the read callback returns.

When the read callback returns, the framework calls the device I/O control callback with the first device I/O control request. The device I/O control callback cannot complete the request, but instead sends it to the default I/O target and then returns. However, the framework does not dispatch the next device I/O control request, because the queue is sequential and the driver has neither completed nor requeued the first device I/O control request.

While the driver handles the device I/O control request, another read request and a write request arrive. The framework dispatches the first of those requests to an I/O event callback on the read/write queue. When that callback returns-assuming that the device I/O control request on the other queue still has not completed-the framework dispatches the next request from the read/write queue. Even though the read/write queue is configured for parallel processing, the device-level lock prevents the framework from calling the queue's read and write event callbacks concurrently.

Although using synchronization scope can greatly simplify a driver, it can also introduce deadlocks. For example, assume that the I/O target in this example cannot complete the device I/O control request without getting additional information or requesting an action from the driver's device stack. If the I/O target sends a request to the device stack while the original request is still pending, the driver deadlocks. The framework cannot dispatch the new request to the driver until the original request completes.

image from book
The Golden Rule of Synchronization

Never call outside your driver when holding a lock.

The bottom line is that synchronization scope is in general useful and safe in monolithic drivers. If your driver is going to be in a stack of drivers, unless you know and can control the behavior of the other drivers in the stack, it's problematic to opt for synchronization scope.
-Eliyas Yakub, Windows Driver Foundation Team, Microsoft

image from book

Synchronization Scope in UMDF Drivers

UMDF drivers can configure synchronization scope for its I/O queue and file callbacks. To set synchronization scope, a driver calls the SetLockingConstraint method of the IWDFDeviceInitialize interface before it creates the device object. Possible values are the following constants of the WDF_CALLBACK_CONSTRAINT type:

  • WdfDeviceLevel

    Sets device scope. This is the UMDF default.

  • None

    Sets no synchronization.

Table 10-1 lists the UMDF callback interfaces and methods that a driver can implement on device, queue, and file callback objects and shows whether the framework acquires the device presentation lock before it calls each method.

Table 10-1: UMDF Callbacks Serialized Using Device Scope
Open table as spreadsheet

Callback Method

Serialized in device scope

IFileCallbackCleanup::OnCleanupFile

Yes

IFileCallbackClose::OnCloseFile

Yes

IImpersonateCallback::OnImpersonate

No

IQueueCallbackCreate::OnCreateFile

Yes

IQueueCallbackDefaultIoHandler::OnDefaultIoHandler

Yes

IQueueCallbackDeviceIoControl::OnDeviceIoControl

Yes

IQueueCallbackIoResume::OnIoResume

Yes

IQueueCallbackIoStop::OnIoStop

Yes

IQueueCallbackRead::OnRead

Yes

IQueueCallbackStateChange::OnStateChange

Yes

IQueueCallbackWrite::OnWrite

Yes

IRequestCallbackCancel::OnCancel

Yes

IRequestCallbackRequestCompletion::OnCompletion

No

Callbacks for different device objects are not serialized. Thus, if a UMDF driver manages two devices and creates a device object and one or more I/O queues for each device object, the framework can concurrently invoke the callbacks for the two device objects. Each device object is part of a different device stack and thus in a different host process that has its own runtime environment and thus no risk of a race condition.

If the driver sets the synchronization scope to None, UMDF can concurrently call any event callback with any other event callback and the driver must create and acquire all its own locks. A critical section is often a good choice for such a lock because it prevents thread preemption. However, a multithreaded driver might instead require a mutex, which can be acquired recursively.

The USB Filter sample driver sets synchronization scope to None in the Device.cpp file, as follows:

 FxDeviceInit->SetLockingConstraint(None); 

The USB filter driver uses no synchronization scope because it does not maintain any state information from one request to the next and does not require locks.

Important 

UMDF does not support a method by which a driver can determine whether a particular I/O request has been canceled. If your driver sets device-level locking and handles all I/O requests synchronously, the framework cannot cancel an I/O request until the driver releases the lock and the driver might become nonresponsive as a result.

Consider this scenario: Your driver supports a device I/O control request that consists of many commands to the device hardware and has a significant amount of state information to maintain between commands, so the driver's OnDeviceControl callback performs the operation synchronously. However, if the application exits unexpectedly while the request is pending, the framework cannot cancel the request because the lock will not be released until the OnDeviceControl method returns. You can avoid such issues either by choosing no synchronization scope if your driver performs synchronous I/O or by setting a time-out on every synchronous I/O request.

Synchronization Scope in KMDF Drivers

A KMDF driver can configure synchronization scope for I/O event, file, and queue callbacks by setting the SynchronizationScope field in the object attributes structure when it creates a device object, I/O queue object, or file object. The framework can also apply synchronization scope to DPC, timer, and work item callbacks, as described in "Automatic Serialization for KMDF DPC, Timer, and Work Item Callbacks" later in this chapter.

Possible values for SynchronizationScope are the following WDF_SYNCHRONIZATION_ SCOPE constants:

  • WdfSynchronizationScopeDevice

    Sets device scope.

  • WdfSynchronizationScopeQueue

    Sets queue scope.

  • WdfSynchronizationScopeNone

    Sets no synchronization scope.

  • WdfSynchronizationScopeInheritFromParent

    Sets the scope for an object to the same value as that of its parent object.

If the driver sets synchronization scope for a device or queue object, the framework serializes callbacks to a driver's I/O request cancellation callbacks but not to the completion callbacks. A driver cannot set synchronization scope for an I/O request object and should accept the default setting for this field.

Table 10-2 lists the KMDF callbacks that are subject to synchronization scope. The table indicates which callbacks are serialized when the driver sets device scope and which are serialized when the driver sets queue scope on the device object, along with the execution levels at which each callback can run.

Table 10-2: Summary of KMDF Callback Serialization
Open table as spreadsheet

Callback function

Serialized in device scope

Serialized in queue scope

Execution level

CompletionRoutine

No

No

<=DISPATCH_LEVEL

EvtDeviceFileCreate

Yes

No

PASSIVE_LEVEL

EvtDpcFunc

Yes [1]

Yes [2]

DISPATCH_LEVEL

EvtFileCleanup

Yes

No

PASSIVE_LEVEL

EvtFileClose

Yes

No

PASSIVE_LEVEL

EvtInterruptDpc

Yes[1]

No

DISPATCH_LEVEL

EvtIoCanceledOnQueue

Yes

Yes

<=DISPATCH_LEVEL

EvtIoDefault

Yes

Yes

<=DISPATCH_LEVEL

EvtIoDeviceControl

Yes

Yes

<=DISPATCH_LEVEL

EvtIoInternalDeviceControl

Yes

Yes

<=DISPATCH_LEVEL

EvtIoQueueState

Yes

Yes

<=DISPATCH_LEVEL

EvtIoRead

Yes

Yes

<=DISPATCH_LEVEL

EvtIoResume

Yes

Yes

<=DISPATCH_LEVEL

EvtIoStop

Yes

Yes

<=DISPATCH_LEVEL

EvtIoWrite

Yes

Yes

<=DISPATCH_LEVEL

EvtRequestCancel

Yes

Yes

<=DISPATCH_LEVEL

EvtTimerFunc

Yes[1]

Yes[2]

DISPATCH_LEVEL

EvtWorkItem

Yes[1]

Yes[2]

PASSIVE_LEVEL

[1]Yes, if the driver sets AutomaticSerialization, which is the default. "Automatic Serialization for KMDF DPC, Timer, and Work Item Callbacks" later in this chapter describes automatic serialization.

[2]Yes, if the queue is the parent of the object and if the driver sets AutomaticSerialization, which is the default.

If the driver accepts the defaults for synchronization scope and automatic serialization, the framework does not serialize any of the callbacks in the table.

Synchronization Scope Defaults

The default scope for the driver object is WdfSynchronizationScopeNone, and the default scope for all other objects-which are children of the driver object-is WdfSynchronizationScopeInheritFromParent. Thus, by default, the framework does not acquire any locks or serialize any of the I/O event, file, and queue callbacks. To use framework synchronization, a KMDF driver must explicitly set the synchronization scope on the device or queue objects.

 Chapter 5 covers the object hierchy  Because scope is inherited by default, a driver can easily set synchronization scope for its I/O queues by setting the scope for the device object, which is the parent of the I/O queues.

Device Scope

If the driver sets device scope, the framework acquires the device object presentation lock before it invokes the callbacks for an individual device object or for any file objects or queues that are children of the device object. However, callbacks for different device objects are not serialized and therefore can run concurrently.

To use device scope, the driver sets WdfSynchronizationScopeDevice for the device object.

Queue Scope

 KMDF  If a KMDF driver specifies queue scope for a device object, the framework serializes calls to the I/O event callbacks for the queue object but not for the file object. Consequently, the EvtDeviceFileXxx and EvtFileXxx callbacks for the device object and its queues can run concurrently.

To use queue scope, the driver sets WdfSynchronizationScopeQueue for the device object and WdfSynchronizationScopeInheritFromParent for the queue object. Queue scope means that only one of the listed callback functions can be active for each queue at any one time. A driver cannot set synchronization scope separately for each queue.

Figure 10-5 shows the objects that are affected if a KMDF driver sets queue scope for the device object that was shown in Figure 10-4 earlier in the chapter.

image from book
Figure 10-5: Queue synchronization scope for a KMDF driver

If the driver sets queue scope, the framework acquires the presentation lock at queue level. Consequently, certain callbacks for the read/write queue and the request object are serialized with respect to each other. Callbacks for the device I/O control queue are similarly serialized. However, callbacks for the read/write queue and the request object can run concurrently with callbacks for device I/O control queue.

 Queue scope and file objects  Queue scope does not apply to file objects. However, by default, a file object inherits its synchronization scope from its parent. Therefore, if the driver sets queue scope for the parent device object and registers one or more file object callbacks, the driver must explicitly set either device scope or no scope for the file object. If the driver fails to do so, the framework generates an error.

If a driver sets device scope for a file object, it must also set the execution level for the object to WdfExecutionLevelPassive, as explained in "Execution Level in KMDF Drivers" later in this chapter.

The best practice for file objects is to use no scope and to acquire the appropriate locks in the event callback functions when synchronization is required.

KMDF Example: Synchronization Scope

Listing 10-1 shows how the Serial sample driver-which manages serial ports-sets WdfSynchronizationScopeNone for its file objects. This code appears in the Serial\Pnp.c file.

Listing 10-1: Setting synchronization scope in a KMDF driver

image from book
 WDF_OBJECT_ATTRIBUTES_INIT(&attributes); attributes.SynchronizationScope = WdfSynchronizationScopeNone; WDF_FILEOBJECT_CONFIG_INIT( &fileobjectConfig,                             SerialEvtDeviceFileCreate,                             SerialEvtFileClose,                             WDF_NO_EVENT_CALLBACK // Cleanup                            ); WdfDeviceInitSetFileObjectConfig( DeviceInit,                                   &fileobjectConfig,                                   &attributes                                   ); 
image from book

The code in the listing appears in the driver's EvtDriverDeviceAdd callback before the driver creates the device object. To set no synchronization scope for the file objects, the driver initializes an object attributes structure and sets the SynchronizationScope field to WdfSynchronizationScopeNone. It next initializes a WDF_FILEOBJECT_CONFIG structure with pointers to the driver's file object event callback functions. Finally, the driver calls WdfDeviceInitSetFileObjectConfig to record the file object configuration information and the object attributes in the WDFDEVICE_INIT structure that it will eventually pass when it creates the device object.

In the Serial driver, the SerialEvtDeviceFileCreate and SerialEvtFileClose file object callback functions do not share data with other device object callback functions and therefore do not require locks. "Serial" in the function names indicates that the functions are part of the Serial sample driver; it does not mean that the callbacks are serialized.

Automatic Serialization for KMDF DPC, Timer, and Work Item Callbacks

Every DPC, timer, or work item object is the child of a device object or of a queue object. To simplify a driver's implementation of callbacks for DPCs, timers, and work items, KMDF by default automatically serializes the execution of those callbacks with the execution of the callbacks for the parent object. A driver can override the default by setting AutomaticSerialization to FALSE in the configuration structure during creation of the DPC, timer, or work item object.

When automatic serialization is enabled, the framework applies the SynchronizationScope setting for the device or queue that is the parent of the object. Thus, if the parent of a timer is a device object for which the driver set device scope, the framework acquires the device object presentation lock before it invokes the EvtTimerFunc callback.

You should use automatic serialization only for callback functions that can run at the same IRQL. For example, if a single device object has a child DPC object and a child work item object, you should not enable automatic serialization for both of the children because a DPC callback always runs at DISPATCH_LEVEL and a work item callback always runs at PASSIVE_LEVEL. The framework cannot use the same lock to serialize both of these functions. The following topic, "Execution Level in KMDF Drivers," provides more information about how IRQL affects serialization.

The Serial sample creates DPCs that run when its timers expire and when it completes a read or write operation. The driver enables automatic synchronization for all of these DPCs so that they are serialized with the other callbacks for the I/O queue or device object. Consequently, the DPCs can access the device and queue context areas without using locks. Listing 10-2 shows how the Serial driver sets automatic serialization for one of these DPCs. This code appears in the Kmdf\Serial\Utils.c source file.

Listing 10-2: Setting automatic serialization for a DPC object

image from book
 WDF_DPC_CONFIG_INIT(&dpcConfig, SerialCompleteWrite); dpcConfig.AutomaticSerialization = TRUE; WDF_OBJECT_ATTRIBUTES_INIT(&dpcAttributes); dpcAttributes.ParentObject = pDevExt->WdfDevice; status = WdfDpcCreate(&dpcConfig,                       &dpcAttributes,                       &pDevExt->CompleteWriteDpc); 
image from book

In the listing, the sample driver initializes a WDF_DPC_CONFIG structure with the name of the DPC function and then sets the AutomaticSerialization field of the structure to TRUE.

This DPC is used only with a particular device object, so the driver initializes an object attributes structure and assigns the device object as the parent. The driver then calls WdfDpcCreate to create the DPC, passing a pointer to the DPC configuration structure, the attributes structure, and the DPC function.

Execution Level in KMDF Drivers

Every locking and synchronization primitive has an associated IRQL. In some situations the framework uses a PASSIVE_LEVEL synchronization primitive to serialize callbacks, and in other situations it uses a primitive that runs at DISPATCH_LEVEL or-for interrupts-at DIRQL. For a KMDF driver, you can force the framework to use a PASSIVE_LEVEL lock when it serializes certain callbacks.

 Chapter 15 describes IRQL guidelines  By setting the execution level, KMDF drivers can specify the maximum IRQL at which the callbacks for driver, device, file, and general objects are invoked. As with synchronization scope, execution level is an object attribute. The driver sets the ExecutionLevel field of the attributes structure when it creates a driver, device, file, or general object.

KMDF supports the following execution levels:

  • WdfExecutionLevelPassive

    Passive execution level means that the framework calls all event callbacks for the object at PASSIVE_LEVEL. If necessary, KMDF invokes the callback from a system worker thread.

    Drivers can set this level only for device and file objects. Typically, a driver should set passive execution level only if the callbacks access pageable code or data or call other functions that must be called at PASSIVE_LEVEL.

    Callbacks for events on file objects are always called at PASSIVE_LEVEL because these functions must be able to access pageable code and data. The EvtDeviceFileCreate and EvtFileCleanup callbacks run in the thread context of the application that opened the file handle. EvtFileClose can be called in an arbitrary thread context.

  • WdfExecutionLevelDispatch

    Dispatch execution level means that the framework can invoke the callbacks from any IRQL up to and including DISPATCH_LEVEL.

    This setting does not force all callbacks to occur at DISPATCH_LEVEL. However, if a callback requires synchronization, KMDF uses a spin lock, which raises IRQL to DISPATCH_LEVEL. You should therefore design the callback to run at DISPATCH_LEVEL and to use a work item to perform tasks that require execution at PASSIVE_LEVEL. If the work item shares data with the other callbacks, the work item can use the device or queue presentation lock to synchronize access. The driver acquires this lock by using the WdfObjectAcquireLock method.

  • WdfExecutionLevelInheritFromParent

    Inherited execution level means that the framework uses the same execution level value that is set for the parent of the object.

    This setting is the default for all objects except the driver object. The default execution level for the driver object is WdfExecutionLevelDispatch.

The execution level determines the locking mechanism behind the framework's object presentation locks, as follows:

  • If the driver sets passive execution level, the framework serializes event callbacks by using a fast mutex, which is a PASSIVE_LEVEL lock.

    Therefore the framework calls the serialized callback functions at PASSIVE_LEVEL regardless of the synchronization scope.

  • If the driver sets dispatch execution level and either device or queue synchronization scope, the framework serializes event callbacks by using a spin lock, which raises IRQL to DISPATCH_LEVEL.

    If the synchronization scope is set to none, the framework does not acquire a presentation lock and can invoke the callbacks at either DISPATCH_LEVEL or at PASSIVE_LEVEL.

 Exception for file callbacks  The EvtDeviceFileCreate, EvtFileCleanup, and EvtFileClose callbacks run in the caller's thread context and use pageable data, so they must be called at PASSIVE_LEVEL. If a driver sets device synchronization scope for its file objects, it must set the execution level for the file objects to WdfExecutionLevelPassive to force the framework to use a PASSIVE_LEVEL lock.

The following example, from the Toaster\Func\Featured\Toaster.c file, shows how the Featured Toaster sample sets synchronization scope and execution level:

 WDF_OBJECT_ATTRIBUTES_INIT_CONTEXT_TYPE(&fdoAttributes, FDO_DATA); fdoAttributes.SynchronizationScope = WdfSynchronizationScopeDevice; fdoAttributes.ExecutionLevel = WdfExecutionLevelPassive; . . . //Code omitted for brevity status = WdfDeviceCreate(&DeviceInit, &fdoAttributes, &device); 

As the example shows, a driver sets execution level in the object attributes structure by assigning a value to the ExecutionLevel field. The Featured Toaster sample sets device synchronization scope and PASSIVE_LEVEL execution for the device object, so that the framework serializes calls to I/O event and file object callbacks and invokes all such callbacks at IRQL PASSIVE_LEVEL.

Although forcing all callbacks to run at PASSIVE_LEVEL initially seems to make driver programming much easier, this setting is not appropriate for most drivers. In a busy driver, I/O processing might be delayed because the driver's I/O callbacks cannot run until all other DISPATCH_LEVEL processing is complete.




Developing Drivers with the Microsoft Windows Driver Foundation
Developing Drivers with the Windows Driver Foundation (Pro Developer)
ISBN: 0735623740
EAN: 2147483647
Year: 2007
Pages: 224

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net