Chapter 24: Threading

One of the results of the move from 16-bit to 32-bit computing was the ability to write code that made use of threads, but although Visual C++ developers have been able to use threads for some time, Visual Basic developers haven’t had a truly reliable way to do so, until now. Previous techniques involved accessing the threading functionality available to Visual C++ developers. Although this worked, actually developing multithreaded code without adequate debugger support in the Visual Basic environment was nothing short of a nightmare.

For most developers, the primary motivation for multithreading is the ability to perform long-running tasks in the background, while still providing the user with an interactive interface. Another common scenario is when building server-side code that can perform multiple long-running tasks at the same time. In that case, each task can be run on a separate thread, enabling all the tasks to run in parallel.

This chapter introduces you to the various objects in the .NET Framework that enable any .NET language to be used to develop multithreaded applications.

What Is a Thread?

The term thread is short for thread of execution. When your program is running, the CPU is actually running a sequence of processor instructions, one after another. You can think of these sequential instructions as forming a thread that is being executed by the CPU. A thread is, in effect, a pointer to the currently executing instruction in the sequence of instructions that make up the application. This pointer starts at the top of the program and moves through each line, branching and looping when it comes across decisions and loops. When the program is no longer needed, the pointer steps outside of the program code and the program is effectively stopped.

Most applications have only one thread, so they are only executing one sequence of instructions. Some applications have more than one thread, so they can simultaneously execute more than one sequence of instructions. It is important to realize that each CPU in your computer can only execute one thread at a time, with the exception of hyperthreaded processors that essentially contain multiple CPUs inside a single CPU. If you only have one CPU, then your computer can execute only one thread at a time. Even when an application has several threads, only one can run at a time in this case. If your computer has two or more CPUs, then each CPU will run a different thread at the same time. In this case, more than one thread in your application may run at the same time, each on a different CPU.

Of course, when you have a computer with only one CPU, on which several programs can be actively running at the same time, the statements in the previous paragraph fly in the face of visual evidence. Yet it is true that only one thread can execute at a time on a single-CPU machine. What you perceive to be simultaneously running applications is really an illusion created by the Windows operating system through a technique called preemptive multithreading, which is discussed later in the chapter.

All applications have at least one thread - otherwise, they couldn’t do any work, as there would be no pointer to the thread of execution. The principle of a thread is that it enables your program to perform multiple actions, potentially at the same time. Each sequence of instructions is executed independently of other threads.

The classic example of multithreaded functionality is Microsoft Word’s spell checker. When the program starts, the execution pointer begins at the top of the program and eventually gets itself into a position where you’re able to start writing code. However, at some point Word starts another thread and creates another execution pointer. As you type, this new thread examines the text and flags any spelling errors as you go, underlining them with a red oval (see Figure 24-1).

Figure 24-1

Every application has one primary thread. This thread serves as the main process thread through the application. Imagine you have an application that starts up, loads a file from disk, performs some processing on the data in the file, writes a new file, and then quits. Functionally, it might look like Figure 24-2.

Figure 24-2

This simple application needs only a single thread. When the program is told to run, Windows creates a new process and creates the primary thread. To understand more about exactly what it is that a thread does, you need to understand how Windows and the computer’s processor deal with different processes.

Processes, AppDomains, and Threads

Windows is capable of keeping many programs in memory at once and allowing the user to switch between them. Windows can also run programs in the background, possibly under different user identities. The capability to run many programs at once is called multitasking.

Each of these programs that your computer keeps in memory runs in a single process. A process is an isolated region of memory that contains a program’s code and data. All programs run within a process, and code running in one process cannot access the memory within any other process. This prevents one program from interfering with any other program.

The process is started when the program starts and exists for as long as the program is running. When a process is started, Windows sets up an isolated memory area for the program and loads the program’s code into that area of memory. It then starts the main thread for the process, pointing it at the first instruction in the program. From that point, the thread runs the sequence of instructions defined by the program.

Windows supports multithreading, so the main thread might execute instructions that create more threads within the same process. These other threads run within the same memory space as the main thread - all sharing the same memory. Threads within a process are not isolated from each other. One thread in a process can tamper with data being used by other threads in that same process. However, a thread in one process cannot tamper with data being used by threads in any other processes on the computer.

At this point, you should understand that Windows loads program code into a process and executes that code on one or more threads. The .NET Framework adds another concept to the mix: the AppDomain. An AppDomain is very much like a process in concept. Each AppDomain is an isolated region of memory, and code running in one AppDomain cannot access the memory of another AppDomain.

The .NET Framework introduced the AppDomain to make it possible to run multiple, isolated programs within the same Windows process. It turns out to be relatively expensive to create a Windows process in terms of time and memory. It is much cheaper to create a new AppDomain within an existing process.

Remember that Windows has no concept of an AppDomain; it only understands the concept of a process. The only way to get any code to run under Windows is to load it into a process. This means that each .NET AppDomain exists within a process. The result is that all .NET code runs within an AppDomain and within a Windows process (see Figure 24-3).

Figure 24-3

In most cases, a Windows process contains one AppDomain, which contains your program’s code. The main thread of the process executes your program’s instructions. The result is that the existence of the AppDomain is largely invisible to your program.

In some cases, most notably ASP.NET, a Windows process will contain multiple AppDomains, each with a separate program loaded (see Figure 24-4).

Figure 24-4

ASP.NET uses this technique to isolate Web applications from each other without having to start an expensive new Windows process for each virtual root on the server.

Note that AppDomains do not change the relationship between a process and threads. Each process has a main thread and may have other threads. Therefore, even in the ASP.NET process, with multiple AppDomains, there is only one main thread. Of course, ASP.NET creates other threads, so multiple Web applications can execute simultaneously, but there’s only a single main thread in the entire process.

Thread Scheduling

Earlier in the chapter it was noted that visual evidence suggests that multiple programs, and thus multiple threads, execute simultaneously, even on a single-CPU computer. Again, this is an illusion created by the operating system, through the use of a concept called time slicing or time sharing.

In reality, only one thread runs on each CPU at a time, with the exception of hyperthreaded processors, which are essentially multiple CPUs in one. In a single-CPU machine, this means that only one thread is ever executing at any one time. To provide the illusion that many things are happening at the same time, the operating system never lets any one thread run for very long, giving other threads a chance to get a bit of work done as well. As a result, it appears that the computer is executing several threads at the same time.

The length of time each thread is allowed to run is called a quantum. Although a quantum can vary, it is typically around 20 milliseconds. Once a thread has run for its quantum, the OS stops the thread and allows another thread to run. When that thread reaches its quantum, yet another thread is allowed to run, and so on. A thread can also give up the CPU before it reaches its quantum. This happens frequently, as most I/O operations and numerous other interactions with the Windows operating system cause a thread to give up the CPU.

Because the length of time each thread can run is so short, it’s never noticeable that threads are being started and stopped. This is the same concept animators use when creating cartoons or other animated media. As long as the changes happen faster than you can perceive them, you have the illusion of motion, or, in this case, simultaneous execution of code.

The technology used by Windows is called preemptive multitasking. It is preemptive because no thread is ever allowed to run beyond its quantum. The operating system always intervenes and allows other threads to run. This ensures that no single thread can consume all the processing power on the machine to the detriment of other threads.

It also means that you can never be sure when your thread will be interrupted and another thread allowed to run. This is the primary source of the complexity of multithreading, as it can cause race conditions when two threads access the same memory. If you attempt to solve a race condition with a lock, it can cause deadlock conditions when two threads attempt to access the same lock. You’ll learn more about these concepts later. For now, understand that writing multithreaded code can be exceedingly difficult.

The entity that executes code in Windows is the thread. This means that the operating system is primarily focused on scheduling threads to keep the CPU or CPUs busy at all times. The operating system does not schedule either processes or AppDomains. Processes and AppDomains are merely regions of memory that contain your code - threads are what execute the code.

Threads have priorities, and Windows always allows higher priority threads to run before lower priority threads. In fact, if a higher priority thread is ready to run, Windows will cut short a lower priority thread’s quantum to allow the higher priority thread to execute sooner. The result is that Windows has a bias towards threads of higher priority.

Setting thread priorities can be useful in situations where you have a process that requires a lot of processor muscle, but it doesn’t matter how long the process takes to do its work. Setting a program’s thread to a low priority allows that program to run continuously with little impact on other programs, so if you need to use Word or Outlook or another application, Windows gives more processor time to these applications and less time to the low-priority program. This enables the computer to work smoothly and efficiently for the user, letting the low-priority program only use otherwise wasted CPU power.

Threads may also voluntarily suspend themselves before their quantum is complete. This happens frequently - for example, when a thread attempts to read data from a file. It takes significant time for the I/O subsystem to locate the file and start retrieving the data. You can’t have the CPU sitting idle during that time, especially when other threads could be running. Instead, the thread enters a wait state to indicate that it is waiting for an external event. The Windows scheduler immediately locates and runs the next ready thread, keeping the CPU busy while the first thread waits for its data.

Windows also automatically suspends and resumes threads depending on its perceived processing needs, the various priority settings, and so on. Suppose you’re running one AppDomain containing two threads. If you can somehow mark the second thread as dormant (in other words, tell Windows that it has nothing to do), there’s no need for Windows to allocate time to it. Effectively, the first thread receive 100 percent of the processor horsepower available to that process. When a thread is marked as dormant, it’s said to be in a wait state.

Windows is particularly good at managing processes and threads. It’s a core part of Windows’ functionality, so its developers have spent a lot of time ensuring that it’s super-efficient and as bug-free as software can be. This means that creating and spinning up threads is very easy to do and happens very quickly. In addition, threads only take up a small amount of system resources. However, there is a caveat you should be aware of.

The activity of stopping one thread and starting another is called context switching. This switching happens relatively quickly, but only if you’re careful with the number of threads you create. Remember that this happens for each active thread at the end of each quantum (if not before) - so after at most 20 milliseconds. If you spin up too many threads, the operating system spends all its time switching between different threads, perhaps even getting to a point where the code in the thread doesn’t get a chance to run because as soon as you’ve started the thread it’s time for it to stop again.

Creating thousands of threads is not the right solution. What you need is a balance between the number of threads that your application requires and the number of threads that Windows can handle. There’s no magic number or right answer to the question of how many threads you should create. Just be aware of context switching and experiment a little.

Consider the Microsoft Word spell check example. The thread that performs the spell check is around all the time. Imagine you have a blank document containing no text. At this point, the spell check thread is in a wait state. If you type a single word into the document and then pause, Word will pass that word over to the thread and signal it to start working. The thread uses its own slice of the processor power to examine the word. If it finds something wrong with it, then it tells the primary thread that a spelling problem was found and that the user needs to be alerted. At this point, the spell check thread drops back into a wait state until more text is entered into the document. Word doesn’t spin up the thread whenever it needs to perform a check - rather, the thread runs all the time, but if it has nothing to do, it drops into this efficient wait state. (You’ll learn about how the thread starts again later.)

Again, this is an oversimplification. Word actually “wakes up” the thread at various times, but the principle is sound - the thread is given work to do, it reports the results, and then it starts waiting for the next chunk of work to do. So why is all this important? If you plan to author multithreaded applications, you need to understand how the operating system will be scheduling your threads, as well as the threads of all other processes on the system. Most important, you need to recognize that your thread can be interrupted at any time so that another thread can run.

Thread Safety and Thread Affinity

Most of the .NET Framework base class library is not thread safe. Thread-safe code is code that can be called by multiple threads at the same time without negative side effects. If code is not thread-safe, then calling that code from multiple threads at the same time can result in unpredictable and undesirable side effects, potentially even crashing your application. When dealing with objects that are not thread safe, you must ensure that multiple threads never simultaneously interact with the same object.

For example, suppose you have a ListBox control (or any other control) on a Windows Form and you start updating that control with data from multiple threads. You’ll find that your results are undependable. Sometimes you’ll see all your data in order, but other times it will be out of order, and other times some data will be missing. This is because Windows Forms controls are not thread safe and don’t behave properly when used by multiple threads at the same time.

To determine whether any specific method in the .NET Base Class Library is thread safe, refer to the online help. If no mention of threading appears in association with the method, then the method is not thread safe.

The Windows Forms subset of the .NET Framework is not only not thread safe, it also has thread affinity. Thread affinity means that objects created by a thread can only be used by that thread. Other threads should never interact with those objects. In the case of Windows Forms, this means that you must ensure that multiple threads never interact with Windows Forms objects (such as forms and controls). This is important because when you’re creating interactive multithreaded applications, you must ensure that only the thread that created a form interacts directly with that form.

As you’ll see, Windows Forms includes technology by which a background thread can safely make method calls on forms and controls by transferring the method call to the thread that owns the form.

When to Use Threads

If we regard computer programs as being either application software or service software, we find there are different motivators for each one. Application software uses threads primarily to deliver a better user experience. Common examples are as follows:

• Microsoft Word - Background spell checker

• Microsoft Word - Background printing

• Microsoft Outlook - Background sending and receiving of e-mail

• Microsoft Excel - Background recalculation

You can see that in all of these cases, threads are used to do “something in the background.” This provides a better user experience. For example, you can still edit a Word document while Word is spooling another document to the printer. Similarly, you can still read e-mails while Outlook is sending your new e-mail. As an application developer, you should use threads to enhance the user experience. At some point during the application startup, code running in the primary thread will have spun up another thread to be used for spell checking. As part of the “allow user to edit the document” process, you give the spell checker thread some words to check. This thread separation means that the user can continue to type, even though spell checking is still taking place.

Service software uses threads to both deliver scalability and improve the service offered. For example, imagine you have a Web server that receives six incoming connections simultaneously. That server needs to service each of the requests in parallel; otherwise, the sixth thread would have to wait for you to finish threads one through five before it even got started. Figure 24-5 shows how IIS might handle incoming requests.

Figure 24-5

The primary motivation for multiple threads in a service like this is to keep the CPU busy servicing user requests even when other user requests are blocked waiting for data or other events. If you have six user requests, the odds are high that some or all of them will read from files or databases and thus will spend many milliseconds in wait states. While some of the user requests are in wait states, other user requests need CPU time and can be scheduled to run. The result is higher scalability because the CPU, I/O, and other subsystems of the computer are kept as busy as possible at all times.

Designing a Background Task

The specific goals and requirements for background processing in an interactive application are quite different from a server application. By interactive application, I am talking about Windows Forms or Console applications. While a Web application might be somewhat interactive, in fact all your code runs on the server, and so Web applications are server applications when it comes to threading.

Interactive Applications

In the case of interactive applications (typically Windows Forms applications), your design must center on having the background thread do useful work, but also interact appropriately (and safely) with the thread managing the UI. After all, you typically want to let the user know when the background process starts, stops, and does interesting things over its life. The following list summarizes the basic requirements for the background thread:

• Indicate that the background task has started

• Provide periodic status or progress information

• Indicate that the background task has completed

• Enable the user to request that the background task be canceled

While every application is different, these four requirements are typical for background threads in an interactive application.

As noted earlier, most of the .NET Framework is not thread safe, and Windows Forms is even more restrictive by having thread affinity. You want your background task to be able to notify the user when it starts, stops, and provides progress information. The fact that Windows Forms has thread affinity complicates this, because your background thread can never directly interact with Windows Forms objects. Fortunately, Windows Forms provides a formalized mechanism by which code in a background thread can send messages to the UI thread so that the UI thread can update the display for the user.

This is done through the use of the BackgroundWorker control, which is found in the Components tab of the Toolbox. The purpose of the BackgroundWorker control is to start, monitor, and control the execution of background tasks. The control makes it easy for code on the application’s primary thread to start a task on a background thread. It also makes it easy for the code running on the background thread to notify the primary thread of progress and completion. Finally, it provides a mechanism by which the primary thread can request that the background task be canceled, and for the background thread to notify the primary thread when it has completed the cancellation.

All this is done in a way that safely transfers control between the primary thread (which can update the UI) and the background thread (which cannot update the UI).

Server Applications

In the case of server programs, your design should center on the background thread being as efficient as possible. Server resources are precious, so the quicker the task can complete, the fewer resources you’ll consume over time. Interactivity with a UI isn’t a concern, as your code is running on a server, detached from any UI. The key to success in server coding is to avoid or minimize locking, thus maximizing throughput because your code is never stopped by a lock.

For example, Microsoft went to great pains to design and refine ASP.NET to minimize the number of locks required from the time a user request hits the server to the time an ASPX page’s code is running. Once the page code is running, no locking occurs, so the page code can just run, top to bottom, as fast and efficiently as possible.

Avoiding locking means avoiding shared resources or data. This is the dominant design goal for server code - to design programs to avoid scenarios in which multiple threads need access to the same variables or other resources. Anytime multiple threads may access the same resource, you need to implement locking to prevent the threads from colliding with each other. We’ll discuss locking later in the chapter, as sometimes it is simply unavoidable.