Section 11.8. GUIs, Threads, and Queues
11.8. GUIs, Threads, and QueuesIn Chapter 5, we learned about threads and the queue mechanism that threads typically use to communicate with each other. We also described the application of those ideas to GUIs in the abstract. Now that we've become fully functional GUI programmers, we can finally see what these ideas translate to in terms of code. If you skipped the related material in Chapter 5, you should probably go back and take a look first; we won't be repeating the thread or queue background material here. The application to GUIs, however, is straightforward. Recall that long-running operations must generally be run in parallel threads, to avoid blocking the GUI from updating itself. In our packing and unpacking examples earlier in this chapter, for instance, we noted that the calls to run the actual file processing should generally run in threads so that the main GUI thread is not blocked until they finish. In the general case, if a GUI waits for anything to finish, it will be completely unresponsive during the waitit can't be resized, it can't be minimized, and it won't even redraw itself if it is covered and uncovered by other windows. To avoid being blocked this way, the GUI must run long-running tasks in parallel, usually with threads. That way, the main GUI thread is freed up to update the display while threads do other work. Because only the main thread should generally update a GUI's display, though, threads you start to handle long-running tasks or to avoid blocking input/output calls cannot update the display with results themselves. Rather, they must place data on a queue (or other mechanism), to be picked up and displayed by the main GUI thread. To make this work, the main thread typically runs a counter loop that periodically checks the thread for new results to be displayed. Spawned threads produce data but know nothing about the GUI; the main GUI thread consumes and displays results but does not generate them. As a more concrete example, suppose your GUI needs to display telemetry data sent in real time from a satellite over sockets (a network interface we'll meet later). Your program has to be responsive enough to not lose incoming data, but it also cannot get stuck waiting for or processing that data. To achieve both goals, spawn threads that fetch the incoming data and throw it on a queue, to be picked up and displayed periodically by the main GUI thread. With such a separation of labor, the GUI isn't blocked by the satellite, nor vice versathe GUI itself will run independently of the data streams, but because the data stream threads can run at full speed, they'll be able to pick up incoming data as fast as it's sent. GUI event loops are not generally responsive enough to handle real-time inputs. Without the data stream threads, we might lose incoming telemetry; with them, we'll receive data as it is sent and display it as soon as the GUI's event loop gets around to picking it up off the queueplenty fast for the real human user to see. If no data is sent, only the spawned threads wait, not the GUI itself. In other scenarios, threads are required just so that the GUI remains active during long-running tasks. While downloading a reply from a web server, for example, your GUI must be able to redraw itself if covered or resized. Because of that, the download call cannot be a simple function call; it must run in parallel with the rest of your programtypically, as a thread. When the result is fetched, the thread must notify the GUI that data is ready to be displayed; by placing the result on a queue, the notification is simplethe main GUI thread will find it the next time it checks the queue. For example, we'll use threads and queues this way in the PyMailGUI program in Chapter 15, to allow multiple overlapping mail transfers to occur without blocking the GUI itself. Whether your GUIs interface with satellites, web sites, or something else, this thread-based model turns out to be fairly simple in terms of code. Example 11-15 is the GUI equivalent of the queue-based threaded program we met earlier in Chapter 5. In the context of a GUI, the consumer thread becomes the GUI itself, and producer threads add data to be displayed to the shared queue as it is produced. The main GUI thread uses the Tkinter after method to check the queue for results. Example 11-15. PP3E\Gui\Tools\xd5 ueuetest-gui.py
When this script is run, the main GUI thread displays the data it grabs off the queue in the ScrolledText window captured in Figure 11-11. A new batch of four producer threads is started each time you left-click in the window, and threads issue "get" and "put" messages to the standard output stream (which isn't synchronized in this examplemessages might overlap occasionally). The producer threads issue sleep calls to simulate long-running tasks such as downloading mail, fetching a query result, or waiting for input to show up on a socket (more on sockets later in this chapter). Figure 11-11. Display updated by GUI thread Example 11-16 takes the model one small step further and migrates it to a class to allow for future customization and reuse. Its operation and output are the same as the prior non-object-oriented version, but the queue is checked more often, and there are no standard output prints. Example 11-16. PP3E\Gui\Tools\xd5 ueuetest-gui-class.py
We'll revisit this technique in a more realistic scenario later in this chapter, as a way to avoid blocking a GUI that must read an input streamthe output of another program. 11.8.1. Placing Callbacks on QueuesNotice that in the prior section's examples, the data placed on the queue is always a string. That's sufficient for simple applications where there is just one type of producer. If you may have many different kinds of threads producing many different types of results running at once, though, this can become difficult to manage. You'll probably have to insert and parse out some sort of type information in the string so that the GUI knows how to process it. Imagine an email client, for instance, where multiple sends and receives may overlap in time; if all threads share the same single queue, the information they place on it must somehow designate the sort of event it representsa downloaded message to display, a successful send completion, and so on. Luckily, queues support much more than just stringsany type of Python object can be placed on a queue. Perhaps the most general of these is a callable object: by placing a callback function on the queue, a producer thread can tell the GUI how to handle the message in a very direct way. The GUI simply calls the objects it pulls off the queue. Because Python makes it easy to handle functions and their argument lists in generic fashion, this turns out to be easier than it might sound. Example 11-17, for instance, shows one way to throw callbacks on a queue that we'll be using in Chapter 15 for PyMailGUI. The THReadCounter class in this module can be used as a shared counter and Boolean flag. The real meat here, though, is the queue interface functions. This example is mostly just a variation on those of the prior section; we still run a counter loop here to pull items off the queue in the main thread. Here, though, we call the object pulled off the queue, and the producer threads have been generalized to place a success or failure callback on the objects in response to exceptions. Moreover, the actions that run in producer threads receive a progress status function that, when called, simply adds a progress indicator callback to the queue to be dispatched by the main thread. We can use this, for example, to show progress during network downloads. Example 11-17. PP3E\Gui\Tools\threadtools.py
This module's self-test code demonstrates how this interface is used. On each button click in a ScrolledTest, it starts up six threads, all running the threadaction function. As this threaded function runs, calls to the passed-in progress function place a callback on the queue, which invokes threadprogress in the main thread. When the threaded function exits, the interface layer will place a callback on the queue that will invoke either threadexit or threadfail in the main thread, depending upon whether the threaded function raised an exception. Because all the callbacks placed on the queue are pulled off and run in the main thread's timer loop, this guarantees that GUI updates occur in the main thread only. Figure 11-12 shows part of the output generated after clicking the example's window once. Its exit, failure, and progress messages are produced by callbacks added to the queue by spawned threads and invoked from the timer loop running in the main thread. Figure 11-12. Messages from queued callbacks To use this module, you will essentially break a modal operation into thread and post-thread steps, with an optional progress call. Study this code for more details and try to trace through the self-test code. This is a bit complex, and you may have to make more than one pass over this code. Once you get the hang of this paradigm, though, it provides a general scheme for handling heterogeneous overlapping threads in a uniform way. PyMailGUI, for example, will do very much the same as mainaction in the self-test code here, whenever it needs to start a mail transfer. |
