9.6. Thread Performance
The way that applications use threads and the associated costs and benefits have greatly impacted the design of many Java APIs. We will discuss some of the issues in detail in other chapters. But it is worth briefly mentioning some aspects of thread performance and how the use of threads has dictated the form and functionality of several recent Java packages.
9.6.1. The Cost of Synchronization
The act of acquiring locks to synchronize threads, even when there is no contention, takes time. In older implementations of Java, this time could be significant. With newer VMs, it is almost negligible.[*] However, unnecessary low-level synchronization can still slow applications by blocking threads where legitimate concurrent access otherwise could be allowed. Because of this, two important APIs, the Java Collections API and the Swing GUI API, were specifically crafted to avoid unnecessary synchronization, by placing it under the developer's control.
The java.util Collections API replaces earlier, simple Java aggregate typesnamely Vector and Hashtablewith more fully featured and, notably, unsynchronized types (List and Map). The Collections API instead defers to application code to synchronize access to collections when necessary and provides special "fail fast" functionality to help detect concurrent access and throw an exception. It also provides synchronization "wrappers" that can provide safe access in the old style. In Java 5.0, special implementations of the Map and new Queue collections were added as part of the java.util.concurrent package. These implementations go even further in that they are written to allow a high degree of concurrent access without any user synchronization. We'll talk about these in Chapter 11.
The Java Swing GUI, which grew out of AWT, has taken a different approach to providing speed and safety. Swing dictates that modification of its components (with notable exceptions) must all be done by a single thread: the main event queue. Swing solves performance problems as well as nasty issues of determinism in event ordering by forcing a single super-thread to control the GUI. The application may access the event queue thread indirectly by pushing commands onto a queue through a simple interface.
9.6.2. Thread Resource Consumption
A fundamental pattern in Java, which will be illustrated in Chapters 12 and 13, is to start many threads to handle asynchronous external resources, such as socket connections. For maximum efficiency, a web server might be tempted to create a thread for each client connection it is servicing. With each client having its own thread, I/O operations may block and restart as needed. But as efficient as this may be in terms of throughput, it is a very inefficient use of server resources. Threads consume memory; each thread has its own "stack" for local variables, and switching between running threads (context switching) adds overhead to the CPU. While threads are relatively lightweight (in theory it is possible to have hundreds or thousands running on a large server), at a certain point, the resources consumed by the threads themselves start defeating the purpose of starting more threads. Often this point is reached with only a few dozen threads. Creating a thread per client is not a very scalable option.
An alternative approach is to create "thread pools" where a fixed number of threads pull tasks from a queue and return for more when they are finished. This recycling of threads makes for solid scalability, but it has historically been difficult to implement efficiently for servers in Java because stream I/O (for things like sockets) has not fully supported nonblocking operations. This changed with Java 1.4 and the introduction of the NIO (new I/O) package, java.nio. The NIO package introduces asynchronous I/O channels: nonblocking reads and writes plus the ability to "select" or test the readiness of streams for moving data. Channels can also be asynchronously closed, allowing threads to work gracefully. With the NIO package, it should be possible to create servers with much more sophisticated, scalable thread patterns.
With Java 5.0, thread pools and job "executor" services are codified as utilities as part of the new java.util.concurrent package, meaning you don't have to write these yourself any longer but can use or extend these base classes. We'll talk about them next when we discuss the concurrency utilities in Java 5.0.