Code Example: A Thread-Based Driver

< BACK  NEXT >

[oR]

This section presents a modified version for the packet-based slave DMA driver introduced in Chapter 12. What is different about this driver is that it uses a system thread to do most of the I/O processing. As a result, it spends very little time at DISPATCH_LEVEL IRQL and does not interfere as much with other system components. Code examples can be found on the CD that accompanies this book or on the book's Web site: http://www.W2KDriverBook.com.

How the Driver Works

Figure 14.6 gives a high-level view of the sample driver architecture. One of the first things to notice is that the driver has no Start I/O routine. When a user-mode I/O request arrives, one of the driver's Dispatch routines simply adds the IRP to a work queue associated with the Device object. Then the Dispatch routine calls KeReleaseSemaphore to increment a Semaphore object that keeps track of the number of IRPs in the work queue. A nonzero Semaphore count indicates the number of IRPs within the work queue yet to be processed.

Figure 14.6. Architecture of a thread-based DMA driver.

Each Device object has its own system thread that processes these I/O requests. This thread is in an endless loop that begins with a call to KeWaitForSingleObject on the Semaphore. If the Semaphore object has a nonzero count, the thread removes an IRP from the work queue and performs the I/O operation. On the other hand, if the count is zero the thread goes into a wait state until the Dispatch routine inserts another IRP in the queue.

When the thread needs to perform a data transfer, it starts the device and then uses KeWaitForSingleObject to wait for an Event object. The driver's DpcForIsr routine sets this Event into the signaled state after an interrupt arrives. The Event object effectively synchronizes the interrupt service code (actually the DPC) with the worker thread that dequeues IRPs.

When the driver's RemoveDevice routine needs to kill the system thread, it sets a flag in the Device Extension and increments the Semaphore object. If the thread was asleep waiting for the Semaphore object, it wakes up, sees the flag, and terminates itself. If it is in the middle of an I/O operation, it won't see the flag until it completes the current IRP.

The DEVICE_EXTENSION Structure

This file contains all the usual driver-defined data structures. The following excerpt shows only additional fields that the driver needs in order to manage the system thread and its work queue. Other fields are identical to those in the packet-based slave DMA example of chapter 12.

 typedef struct _DEVICE_EXTENSION {      ...      // Pointer to worker thread object      PETHREAD pThreadObj;            // Flag set to TRUE when worker thread should quit      BOOLEAN bThreadShouldStop;            // Event object signaling Adapter is now owned      KEVENT evAdapterObjectIsAcquired;      // Event signaling last operation now completed      KEVENT evDeviceOperationComplete;            // The work queue of IRPs is managed by this      //  semaphore and spin lock      KSEMAPHORE semIrpQueue;      KSPIN_LOCK lkIrpQueue;      LIST_ENTRY IrpQueueListHead; } DEVICE_EXTENSION, *PDEVICE_EXTENSION; 

The AddDevice Function

This portion of the example shows the initialization code for the Thread object, the work queue, and the various synchronization objects used to process an I/O request. Remember that AddDevice is called once for each Device object.

 NTSTATUS AddDevice( IN PDRIVER_OBJECT pDriverObj,                     IN PDEVICE_OBJECT pdo ) {      ...      // Initialize the work queue lock      KeInitializeSpinLock( &pDevExt->lkIrpQueue );            // Initialize the work queue itself      InitializeListHead( &pDevExt->IrpQueueListHead );            // Initialize the work queue semaphore      KeInitializeSemaphore( &pDevExt->semIrpQueue,                               0, MAXLONG);                                     // Initialize the event for the Adapter object      KeInitializeEvent(                &pDevExt-> evAdapterObjectIsAcquired,                SynchronizationEvent, FALSE );                      // Intialize the event for the operation complete      KeInitializeEvent(                &pDevExt->evDeviceOperationComplete,                SynchronizationEvent, FALSE );                      // Initially the worker thread runs      pDevExt->bThreadShouldStop = FALSE;            // Start the worker thread      HANDLE hThread = NULL;            status =                PsCreateSystemThread( &hThread,                                         (ACCESS_MASK)0,                                        NULL,                                         (HANDLE)0,                                        NULL,                                        WorkerThreadMain,                                        pDevExt );  // arg      if (!NT_SUCCESS(status)) {               IoDeleteSymbolicLink( &linkName );               IoDeleteDevice( pfdo );               return status;      }            // Obtain real pointer to Thread object      ObReferenceObjectByHandle(               hThread,               THREAD_ALL_ACCESS,               NULL,               KernelMode,               (PVOID*)&pDevExt->pThreadObj,               NULL );      ZwClose( hThread );    // don't need handle at all            ... } 

The DispatchReadWrite Function

This routine responds to user requests to read or write the device. After checking for a zero-length transfer, it puts the IRP into the pending state and inserts it into the work queue attached to the target Device object. It then increments the count in the work queue's Semaphore object. Notice that there are no calls to IoStartPakcet because there is no Start I/O routine.

 NTSTATUS DispatchReadWrite( IN PDEVICE_OBJECT pDO,                             IN PIRP pIrp ) {      PIO_STACK_LOCATION pIrpStack =           IoGetCurrentIrpStackLocation( pIrp );                 PDEVICE_EXTENSION pDE = pDO->DeviceExtension;            // Check for zero-length transfers      if( pIrpStack->Parameters.Read.Length == 0 )      {          pIrp->IoStatus.Status = STATUS_SUCCESS;          pIrp->IoStatus.Information = 0;          IoCompleteRequest( pIrp, IO_NO_INCREMENT );          return STATUS_SUCCESS;      }       // Start device operation       IoMarkIrpPending( pIrp );              // Add the IRP to the thread's work queue       ExInterlockedInsertTailList(          &pDE->IrpQueueListHead,          &pIrp->Tail.Overlay.ListEntry,          &pDE->lkIrpQueue );                 KeReleaseSemaphore(          &pDE->semIrpQueue,          0,                 // No priority boost          1,                 // Increment semaphore by 1          FALSE );     // No WaitForXxx after this call                 return STATUS_PENDING; } 

Thread.cpp

This module contains the main thread function and the routines needed to manage the thread.

WorkerThreadMain

This is the IRP-processing engine itself. Its job is to pull I/O requests from the work queue in the Device Extension and perform the data transfer operation. This function continues to wait for new IRPs until the RemoveDevice routine tells it to shut down.

 VOID WorkerThreadMain( IN PVOID pContext ) {      PDEVICE_EXTENSION pDevExt = (PDEVICE_EXTENSION)           pContext;                 PDEVICE_OBJECT pDeviceObj =           pDevExt->pDevice;      PLIST_ENTRY ListEntry;      PIRP pIrp;      CCHAR PriorityBoost;            // Worker thread runs at higher priority than      //    user threads - it sets its own priority      KeSetPriorityThread(           KeGetCurrentThread(),           LOW_REALTIME_PRIORITY );                 // Now enter the main IRP-processing loop      while( TRUE )      {           // Wait indefinitely for an IRP to appear in           // the work queue or for the RemoveDevice           // routine to stop the thread.           KeWaitForSingleObject(                &pDevExt->semIrpQueue,                Executive,                KernelMode,                FALSE,                NULL );                      // See if thread was awakened because      // device is being removed      if( pDevExt->bThreadShouldStop )           PsTerminateSystemThread(STATUS_SUCCESS);                 // It must be a real request. Get an IRP      ListEntry =           ExInterlockedRemoveHeadList(                &pDevExt->IrpQueueListHead,                &pDevExt->lkIrpQueue);                      pIrp = CONTAINING_RECORD(                ListEntry,                IRP,                Tail.Overlay.ListEntry );                      // Process the IRP. This is a synchronous      // operation, so this function doesn't return      // until it's time to get rid of the IRP.      PriorityBoost =           PerformDataTransfer(                pDeviceObj,                pIrp );                      // Release the IRP and go back to the      // top of the loop to see if there's      // another request waiting.      IoCompleteRequest( pIrp, PriorityBoost );            } // end of while-loop } 

KillThread

This function notifies the thread associated with a particular Device object that it's time to quit. For simplicity, this function stops and waits until the target thread is gone. Consequently, it can be called only from PASSIVE_LEVEL IRQL.

 VOID KillThread( IN PDEVICE_EXTENSION pDE ) {      // Set the Stop flag      pDE->bThreadShouldStop = TRUE;            // Make sure the thread wakes up      KeReleaseSemaphore(           &pDE->semIrpQueue,           0,          // No priority boost           1,          // Increment semaphore by 1           TRUE );     // WaitForXxx after this call                 // Wait for the thread to terminate      KeWaitForSingleObject(           &pDE->pThreadObj,           Executive,           KernelMode,           FALSE,           NULL );                 ObDereferenceObject( &pDE->pThreadObj ); } 

Transfer.C

This portion of the example contains the support routines that perform I/O operations. This code is largely derived from the packet-based slave DMA driver in chapter 12. Consequently, only those features that differ significantly are described in detail.

The most notable detail is that very little actual work occurs within the Adapter Control or DpcForIsr routines. Instead of doing their usual jobs, these functions just set Event objects to signal the thread's data transfer routines that they can proceed.

PerformDataTransfer

This function moves an entire buffer of data to or from the device. This may include splitting the transfer over several device operations if there aren't enough mapping registers to handle it all at once. This routine runs at PASSIVE_LEVEL IRQL and doesn't return to the caller until everything is done.

 CCHAR PerformDataTransfer(      IN PDEVICE_OBJECT pDevObj,      IN PIRP pIrp      ) {      PIO_STACK_LOCATION pIrpStack =           IoGetCurrentIrpStackLocation( pIrp );                 PDEVICE_EXTENSION pDE = (PDEVICE_EXTENSION)           pDevObj->DeviceExtension;                 PMDL pMdl = pIrp->MdlAddress;      ULONG MapRegsNeeded;      NTSTATUS status;            // Set the I/O direction flag      if( pIrpStack->MajorFunction == IRP_MJ_WRITE )           pDE->bWriteToDevice = TRUE;      else           pDE->bWriteToDevice = FALSE;                 // Set up bookkeeping values      pDE->bytesRequested =                MmGetMdlByteCount( pMdl );                     pDE->bytesRemaining =               pDE->bytesRequested;     pDE->transferVA = (PCHAR)               MmGetMdlVirtualAddress( pMdl );                    // Flush CPU cache if necessary     KeFlushIoBuffers(          pIrp->MdlAddress,          !pDE->bWriteToDevice,          TRUE );               // Calculate size of first partial transfer     pDE->transferSize = pDE->bytesRemaining;          MapRegsNeeded =          ADDRESS_AND_SIZE_TO_SPAN_PAGES(               pDE->transferVA,               pDE->transferSize );                    if( MapRegsNeeded > pDE->mapRegisterCount )     {      MapRegsNeeded =           pDE->mapRegisterCount;                      pDE->transferSize =                MapRegsNeeded * PAGE_SIZE -                MmGetMdlByteOffset( pMdl );     }          // Acquire the adapter object.     status = AcquireAdapterObject(                pDE,                MapRegsNeeded );      if( !NT_SUCCESS( status )) {           pIrp->IoStatus.Status = status;           pIrp->IoStatus.Information = 0;           return IO_NO_INCREMENT;      }            // Try to perform the first partial transfer      status =           PerformSynchronousTransfer(                pDevObj,                pIrp );                      if( !NT_SUCCESS( status )) {           pDE->pDmaAdapter->DmaOperations->                 FreeAdapterChannel ( pDE->pDmaAdapter );           pIrp->IoStatus.Status = status;           pIrp->IoStatus.Information = 0;           return IO_NO_INCREMENT;      }            // It worked. Update the bookkeeping information      pDE->transferVA += pDE->transferSize;      pDE->bytesRemaining -= pDE->transferSize;            // Loop through all the partial transfer      // operations for this request.      while( pDE->bytesRemaining >0 )      {           // Try to do all of it in one operation           pDE->transferSize = pDE->bytesRemaining;                      MapRegsNeeded =                ADDRESS_AND_SIZE_TO_SPAN_PAGES(                          pDE->transferVA,                          pDE->transferSize );                                     // If the remainder of the buffer is more           // than we can handle in one I/O. Reduce           // our expectations.           if (MapRegsNeeded > pDE->mapRegisterCount) {                MapRegsNeeded = pDE->mapRegisterCount;                                pDE->transferSize =                     MapRegsNeeded * PAGE_SIZE -                          BYTE_OFFSET(pDE->TransferVA);              }                            // Try to perform a device operation.              status =                   PerformSynchronousTransfer(                        pDevObj,                        pIrp );                                      if( !NT_SUCCESS( status )) break;                            // It worked. Update the bookkeeping              // information for the next cycle.              pDE->transferVA += pDE->transferSize;              pDE->bytesRemaining -= pDE->transferSize;      }      // After the last partial transfer is done,      // release the DMA Adapter object .      pDE->pDmaAdapter->DmaOperations->                  FreeAdapterChannel ( pDE->pDmaAdapter );                        // Send the IRP back to the caller. Its final      // status is the status of the last transfer      // operation.      pIrp->IoStatus.Status = status;      pIrp->IoStatus.Information =                     pDE->bytesRequested -                          pDE->bytesRemaining;      // Since there has been at least one I/O      // operation, give the IRP a priority boost.      //      return IO_DISK_INCREMENT; } 

AcquireAdapterObject and AdapterControl

These two functions work together to give a thread a synchronous mechanism for acquiring ownership of the adapter object. AcquireAdapterObject runs in the context of a system thread so it can stop and wait for a nonzero time interval.

 static NTSTATUS AcquireAdapterObject(      IN PDEVICE_EXTENSION pDE,      IN ULONG MapRegsNeeded      ) {      KIRQL OldIrql;      NTSTATUS status;            // We must be at DISPATCH_LEVEL in order      // to request the Adapter object      KeRaiseIrql( DISPATCH_LEVEL, &OldIrql );            pDE->pDmaAdapter->DmaOperations->            AllocateAdapterChannel (                  pDE->pDmaAdapter,                  pDE->pDevice,                  MapRegsNeeded,                  AdapterControl,                  pDE );                        KeLowerIrql( OldIrql );            // If the call failed, it's because there      // weren't enough mapping registers.      if( !NT_SUCCESS( status ))           return status;                 // Stop and wait for the Adapter Control      // routine to set the Event object. This is      // our signal that the Adapter object is ours.      KeWaitForSingleObject(           &pDE->evAdapterObjectIsAcquired,           Executive,           KernelMode,           FALSE,           NULL );                 return STATUS_SUCCESS; } IO_ALLOCATION_ACTION AdapterControl(      IN PDEVICE_OBJECT pDevObj,      IN PIRP pIrp,      IN PVOID MapRegisterBase,      IN PVOID pContext      ) {      PDEVICE_EXTENSION pDE = (PDEVICE_EXTENSION)            pContext;                  // Save the handle to the mapping      // registers. The thread will need it      // to set up data transfers.      //      pDE->mapRegisterBase = MapRegisterBase;           // Let the thread know that its Device     // object the Adapter object     KeSetEvent(          &pDE->evAdapterObjectIsAcquired,          0,          FALSE );                    return KeepObject; } 

PerformSynchronousTransfer

Running in the context of the system thread, this function performs a single data transfer operation. It doesn't return to the caller until the transfer finishes. Notably, the function uses an Event object to wait for the arrival of a device interrupt.

 NTSTATUS PerformSynchronousTransfer(      IN PDEVICE_OBJECT pDevObj,      IN PIRP pIrp      ) {      PDEVICE_EXTENSION pDE = (PDEVICE_EXTENSION)           pDevObj->DeviceExtension;      // Set up the system DMA controller      // attached to this device.      pDE->pDmaAdapter->DmaOperations->           MapTransfer(                 pDE->pDmaAdapter,                 pIrp->MdlAddress,                 pDE->mapRegisterBase,                 pDE->transferVA,                 &pDE->transferSize,                 pDE->bWriteToDevice );                       // Start the device      WriteControl(           pDE,           CTL_INTENB | CTL_DMA_GO );                 // The DPC routine will set an Event      // object when the I/O operation is      // done. Stop here and wait for it.      KeWaitForSingleObject(           &pDE->evDeviceOperationComplete,           Executive,           KernelMode,           FALSE,           NULL );                 // Flush data out of the Adapater      // object cache.      pDE->pDmaAdapter->DmaOperations->           FlushAdapterBuffers(                 pDE->pDmaAdapter,                 pIrp->MdlAddress,                 pDE->mapRegisterBase,                 pDE->transferVA,                 pDE->transferSize,                 pDE->bWriteToDevice );                       // Check for device errors      if( !STS_OK( pDE->DeviceStatus ))           return STATUS_DEVICE_DATA_ERROR;      else           return STATUS_SUCCESS; } 

DpcForIsr

When the device generates an interrupt, the Interrupt Service Routine (not shown here) saves the status of the hardware and requests a DPC. Eventually, DpcForIsr executes and just sets an Event object into the signaled state.

PerformSynchronousTransfer (which has been waiting for this Event object) wakes up and continues processing the current IRP.

 VOID DpcForIsr(      IN PKDPC pDpc,      IN PDEVICE_OBJECT pDevObj,      IN PIRP pIrp,      IN PVOID pContext      ) {      PDEVICE_EXTENSION pDE = (PDEVICE_EXTENSION)                pContext;                      KeSetEvent(           &pDE->evDeviceOperationComplete,           0,           FALSE );                 return; } 

< BACK  NEXT >