Thread Interruption Scenarios

A thread's scheduling priority and the processor's current IRQL determine whether a running thread can be preempted or interrupted. In thread preemption, the operating system replaces the running thread with another thread, usually of higher thread priority, on the same processor. The effect of preemption on an individual thread is to make the processor unavailable for a while. In thread interruption, the operating system forces the current thread to temporarily run code at a higher IRQL.

Some simple examples can show what happens when a thread is preempted or interrupted. This section presents a single-processor example and a multiprocessor example.

Thread Interruption on a Single-Processor System

Figure 15-1 shows a hypothetical example of thread interruption on a single-processor system. For the sake of simplicity, the example omits the thread scheduler, clock interrupts, and so forth.

Figure 15-1: Thread interruption on a single-processor system

The figure shows how thread execution proceeds over time, as follows:

1. Thread A is running at IRQL PASSIVE_LEVEL.

2. Device 1 interrupts at DIRQL. Thread A is interrupted, even though its quantum has not yet expired. The system suspends Thread A and runs the EvtInterruptIsr callback for Device 1. The EvtInterruptIsr function stops Device 1 from interrupting, saves any data it requires for further processing, registers an EvtInterruptDpc function, and exits.

3. No additional interrupts are pending at DIRQL for any device. Because DPCs run at IRQL DISPATCH_LEVEL, any entries in the system's queue of DPC functions run before Thread A can resume.

   In this case, the queued DPC runs the EvtInterruptDpc function that the Device 1 EvtInterruptIsr function specified. Because no further interrupts occur at IRQL>DISPATCH_LEVEL, the EvtInterruptDpc function runs to completion.

4. After the EvtInterruptDpc function exits, the DPC queue is empty and no other higher-priority threads are ready to run. Therefore, the system resumes running Thread A, which continues until one of the following occurs:

   • Its quantum expires.

   • It enters a wait state.

   • It exits.

   • A higher-priority thread becomes ready to run.

   • A hardware interrupt occurs.

   • The thread queues a DPC or an APC.

There is no guarantee that Thread A will exhaust its quantum. A thread can be interrupted or preempted any number of times during its quantum.

Thread Interruption on a Multiprocessor System

Figure 15-2 shows a hypothetical example of thread interruption on a multiprocessor system. This example omits clock interrupts and so forth, but it shows the system's thread scheduler.

Figure 15-2: Thread interruption on a multiprocessor system

Figure 15-2 shows thread execution on two processors, starting at the same time, as follows:

1. Processor 0 is running Thread A, whereas Processor 1 is running Thread B. Both threads run at PASSIVE_LEVEL.

2. Device 1 interrupts on Processor 0, so the system raises IRQL on Processor 0 to DIRQL for Device 1 and runs the Device 1 EvtInterruptIsr function. The Device 1 EvtInterruptIsr function queues a EvtInterruptDpc function. By default, the EvtInterruptDpc function is added to the queue for the same processor (Processor 0) on which the EvtInterruptIsr function is running.

3. Because the EvtInterruptDpc function runs on the same processor as the EvtInterruptIsr function, but at a lower IRQL, it does not start until after the EvtInterruptIsr function exits.

4. Device 1 interrupts again-this time on Processor 1-so the system raises IRQL on Processor 1 to DIRQL for Device 1 and runs the Device 1 EvtInterruptIsr function on Processor 1. The Device 1 EvtInterruptIsr function queues a EvtInterruptDpc function to Processor 1 (the default behavior). The EvtInterruptDpc function starts on Processor 1 after the EvtInterruptIsr function exits.

5. The same EvtInterruptDpc function is now running on two processors at the same time, in response to two different interrupts from the same device. The driver must use spin locks to protect shared memory that the functions might access. In this case, the EvtInterruptDpc function on Processor 0 acquires the lock first, so Processor 1 spins while waiting for the lock.

6. The EvtInterruptDpc function on Processor 0 releases the lock, completes its work, and exits. The system now runs the next ready thread on Processor 0-in this case, Thread A.

7. After the EvtInterruptDpc function on Processor 0 releases the lock, the EvtInterruptDpc function on Processor 1 acquires the lock and performs its tasks. After it exits, the system's thread scheduling code runs to determine which thread to run next.

8. When the thread scheduler exits, no additional DPCs have been queued, so the highest-priority ready thread (Thread C) runs on Processor 1.

Testing for IRQL Problems

Errors related to IRQL are common and often result in a system crash with the IRQL_NOT_LESS_OR_EQUAL bug check code. KMDF performs some internal IRQL checking. In addition, the WDK includes several features that can help you to determine the IRQL at which driver code runs and to find IRQL-related problems. This section describes the following features:

• Functions and debugger commands that return the current IRQL.

• The PAGED_CODE and PAGED_CODE_LOCKED macros.

• Forced IRQL checking in Driver Verifier.

PREfast for Drivers and SDV also include features that can help find IRQL problems. Chapter 23, "PREfast for Drivers," and Chapter 24, "Static Driver Verifier," contain more information about using these tools with KMDF drivers.

Techniques for Finding the Current IRQL

You can get the IRQL at which a processor is currently operating in one of two ways:

• Call the KeGetCurrentIrql function.

• Use the !irql kernel-mode debugger extension command.

A driver can call KeGetCurrentIrql to get the IRQL at which the current processor is operating. This function can be called at any IRQL. A driver can also determine whether it is operating in a critical region by calling KeAreApcsDisabled, which is available on Windows XP and later releases.

During debugging, you can use the !irql debugger extension command to get the IRQL at which a processor is operating. This command returns the IRQL at which the processor was operating immediately before the debugger became active. By default, the command returns the IRQL for the current processor, but you can also specify a processor number as a parameter. This command works on target systems that are running Windows Server 2003 and later versions of Windows.

PAGED_CODE and PAGED_CODE_LOCKED Macros

The PAGED_CODE and PAGED_CODE_LOCKED macros can help you find IRQL problems that are related to page faults.

If the processor is running at or above DISPATCH_LEVEL when the PAGED_CODE macro is called, the system breaks into the debugger. By placing the macro at the beginning of each driver function that contains or calls pageable code, you can determine whether the function performs actions that are invalid at the IRQL at which it is called.

When used together with the static verification tools, the PAGED_CODE macro can help you find IRQL problems in code that subsequently raises IRQL. For example, if the processor is running at PASSIVE_LEVEL when the PAGED_CODE macro runs but the function later calls WdfSpinLockAcquire-which raises IRQL to DISPATCH_LEVEL-you receive a warning.

 Tip  See "PAGED_CODE" in the WDK for more information-online at http://go.microsoft.com/fwlink/?LinkId=82722.

Some functions are pageable but must be locked into memory to run properly. By placing the PAGED_CODE_LOCKED macro at the beginning of such a function, you can ensure that the function is locked down upon entry. The macro causes the system to break into the debugger if the code that calls it is not locked into memory.

 Tip  See "PAGED_CODE_LOCKED" in the WDK for more information-online at http://go.microsoft.com/fwlink/?LinkId=82723.

Driver Verifier Options

Driver Verifier (Verifier.exe) performs numerous checks related to IRQL. By default, Driver Verifier checks for certain errors related to raising and lowering IRQL, allocating memory at an invalid IRQL, and acquiring or releasing spin locks at an invalid IRQL.

In addition, Driver Verifier also can perform forced IRQL checking. When you choose this option, Driver Verifier marks all pageable code and data when the driver or framework requests a spin lock, runs an EvtInterruptSynchronize callback, or raises the IRQL to DISPATCH_LEVEL or higher. If the driver tries to access any of the pageable memory, Driver Verifier issues a bug check.

When forced IRQL checking is enabled, Driver Verifier gathers IRQL-related statistics, including the number of times the driver raised IRQL, acquired a spin lock, or called KeSynchronizeExecution, which is the kernel-mode function underlying WdfInterruptSynchronize. Driver Verifier also counts the number of times that the operating system paged the contents of memory to disk. Driver Verifier stores these statistics in global counters. You can display their values by using the Driver Verifier command line or graphical user interface or by using the !verifier extension in the debugger. If your driver performs DMA, you should also use Driver Verifier's DMA verification option. This option checks for calls made to DMA functions at the wrong IRQL.

Tip 

The Driver Verifier command syntax depends on the version of Windows that is installed. Chapter 21, "Tools for Testing WDF Drivers," provides more information about using Driver Verifier.