Chapter 9: Threads

Ye mind that word? Clear as our gongs—again, an’ once again,
When rippin’ down through coral-trash ran out our moorin’ chain;
An’ by Thy Grace I had the Light to see my duty plain.
Light on the engine-room—no more—bright as our carbons burn.
I’ve lost it since a thousand times, but never past return.

—Rudyard Kipling, writing on dedication to a profession, “McAndrew’s
Hymn,” 1894.

Preliminary Note

Don’t panic! More programmers, especially of the Visual Basic persuasion, have run for the hills over the issue of threading than anything else I know of. Threading to them was just this mysterious thing that screwed them up because Visual Basic didn’t do it, nor did it work well with other code that did. That’s all in the past now, as threading is a full-fledged part of the .NET common language runtime, available to programmers of any language. Even if you’re not currently planning on writing multithreaded code yourself, the Windows environment in which your code runs uses threads extensively, and the success of your project will depend in no small measure on how well you manage to live with them. I’m writing this chapter as an introduction to threading for programmers who haven’t worked with it before. The goal is to crack the topic open easily so you can get your first taste of what’s inside. This chapter won’t make you a complete thread guru, but it will allow you to read the Microsoft documentation and other books, actually understand what they are talking about, and participate intelligently in design discussions on the topic. If it seems overwhelming at first, I’ll say to you what I used to say to my students when I taught rock climbing years ago: “When you start getting into trouble, don’t look down, look at me.” (To which the usual reply was some variation on “Aieeeeeeeeee [thunk]”.)

Problem Background

In the beginning, personal computers did only one thing at a time. You ran your word processor until you were finished, then you closed it and opened your spreadsheet. After working with that for a while, you closed it and treated yourself to a rousing text-based adventure game like Zork. MS-DOS made that possible, and if we sneer at it today, it was a whole lot better than what we had before it, which was nothing. As the next step, 16-bit Windows allowed you to keep several programs open and cut and paste between them if you had enough RAM. You could create a misleading chart in Excel and import it into a pointless Word report that no one ever read. Again you snicker, but it was a big advance at the time. If you’ve forgotten how much fun you had as a kid on your first tricycle, before you learned about fast cars and the opposite sex and throughput scalability, well, too bad for you.

Computers originally performed only one task at a time.

Users loved keeping several applications open at once, but it made the programmer’s job harder. Most PCs even today contain only a single CPU chip to run program code. By definition, while the CPU runs one application’s code, it isn’t running any of the others’, so we need a way to share the CPU among all the programs running at any given time, without the programs needing to know about each other. Sixteen-bit Windows accomplished this through cooperative multitasking. The operating system would give the CPU to a program by sending it a message, usually triggered by a user event such as a keystroke or a mouse click. The program would process the message and keep the CPU until the program finished with that message, at which time it voluntarily yielded the CPU back to Windows by calling a particular function. While one program had the CPU, no other programs could do anything at all, so you had to write your programs for short, “bursty” operations to keep from hanging everything else. That architecture made Windows itself easy to write, and it worked well for simple programs such as Notepad. But it didn’t work well when a program needed to run a background operation, such as recalculating a spreadsheet. Users love to tell stories about the bad old days when they had time to brew and drink a cup of coffee (and go to the bathroom, and maybe play a few quick hands of pinochle with paper cards, and finish War and Peace) before the recalculation finished, because of the fact that the spreadsheet processing took up the entire CPU and wouldn’t let any other programs run. As dedicated Windows users, they naturally wanted to play Solitaire or Minesweeper during the recalculation, even if this delayed its completion somewhat.

16-bit Windows featured co-operative multitasking, which had severe limitations.

That was a rotten user experience, so programmers naturally began working on ways to fix it. The basic approach required you to divide your background processing logic into small chunks. The spreadsheet would do a chunk of recalculation, write some internal state data to remember where it was in the recalculation process, then yield the CPU to any other app that wanted it. When the other apps had finished with the CPU (the user finished moving the card in Solitaire), Windows would give the CPU back to the spreadsheet. The spreadsheet would then read its previously saved state to remember where it had left off, do the next chunk, and yield again. This process would continue until the task was finished. Writing all this chunk- dividing and state-remembering and CPU-yielding code required a great deal of expensive programmer effort that could have been better spent on business logic.

Writing code for background operations, such as spreadsheet recalculations, was difficult.

That’s where matters sat with 16-bit Windows. I remember gripe sessions at the local brewpub after teaching my Harvard class on the subject: Every application needs this background processing capability, so why doesn’t Microsoft build it into the operating system instead of making us roll our own every time? We wanted some part of the operating system to deal with scheduling the various tasks that wanted the CPU. We wanted to be able to set priorities, so that low-priority tasks, such as a screensaver, would run only when no higher priority tasks, such as a recalculation, had work to do, and the recalculation would automatically get out of the way when a command from the user came in. We wanted to be able to make tasks wait for external events (say, I/O completion) to occur before starting to run so that we wouldn’t waste CPU cycles on them before they were ready to go. And we wanted the new architecture to contain its own ready solutions to the new problems it would inevitably generate, such as the conflict that would occur if two tasks wanted to access the same piece of data at the same time.

We want the operating system to handle multi- tasking for us.