Chapter 11 -- Thread Pooling

[Previous] [Next]

Chapter 11

In Chapter 8, we discussed how to synchronize threads using mechanisms that allow your threads to remain in user mode. The wonderful thing about user-mode synchronization is its speed. If you are concerned about your thread's performance, you should always start by seeing if a user-mode thread synchronization mechanism will work for you.

By now, you know that creating multithreaded applications is difficult. You face two big issues: managing the creation and destruction of threads and synchronizing the threads' access to resources. For synchronizing resource access, Windows offers many primitives to help you: events, semaphores, mutexes, critical sections, and so on. These are all fairly easy to use. The only thing that would make things easier is if the system could automatically protect shared resources. Unfortunately, we have a ways to go before Windows can offer this protection in a way that makes everybody happy.

Everybody has opinions on how to manage the creation and destruction of threads. I've created several different implementations of thread pools myself over the past years, each one fine-tuned for a particular scenario. Microsoft Windows 2000 offers some new thread pooling functions to make thread creation, destruction, and general management easier. This new general-purpose thread pool is definitely not right for every situation, but it often fits the bill and can save you countless hours of development time.

The new thread pooling functions let you do the following:

• Call functions asynchronously
• Call functions at timed intervals
• Call functions when single kernel objects become signaled
• Call functions when asynchronous I/O requests complete

To accomplish these tasks, the thread pool consists of four separate components. Table 11-1 shows the components and describes the rules that govern their behavior.

Table 11-1. Thread pool components and their behavior

Component
	Timer	Wait	I/O	Non-I/O
Initial Number of Threads	Always 1	1	0	0
When a Thread Is Created	When first thread pool timer function is called	One thread for every 63 registered objects	The system uses heuristics, but here are some factors that affect the creation of a thread: Some time (in seconds) has passed since the thread was added The WT_EXECUTELONGFUNCTION flag is used The number of queued work items exceeds a certain threshold
When a Thread is Destroyed	When a process terminates	When the number of registered wait objects is 0	When the thread has no pending I/O requests and has been idle for a threshold period (about a minute)	When the thread is idle for a threshold period (about a minute)
How a Thread Waits	Alertable	WaitForMultipleObjectsEx	Alertable	GetQueuedCompletionStatus
What Wakes Up a Thread	Waitable timer is signaled queuing a user APC	Kernal object becomes signaled	Queued user APC or completed I/O request	Posted completion status or completed I/O request (The completion port allows at most 2 * number of CPUs threads to run concurrently)

When a process initializes, it doesn't have any of the overhead associated with these components. However, as soon as one of the new thread pooling functions is called, some of the components are created for the process and some stay around until the process terminates. As you can see, the overhead of using the thread pool is not trivial: quite a few threads and internal data structures become part of your process. So you must carefully consider what the thread pool will and won't do for you: don't just blindly use these functions.

OK, enough with the disclaimers. Let's see what this stuff does.