7.3.2 Utilities for Process-parallel Computing

		and characteristics of Beowulf-class systems, now and in the future. The range of applications that may be suitably hosted by Beowulfs will be strongly determined by this aspect of the system class. Some applications are sufficiently regular and employ minimal communications that they are natural targets for Beowulfs. A second group of applications, if captured with appropriate latency-tolerant algorithmic techniques, can also be supported, although with greater program development effort. Important examples of these have already been demonstrated. But a third, hopefully small, class of applications that exhibit substantial intrinsic communications demands requiring frequent global synchronization may never be served adequately by future Beowulf-class systems. The performance and generality of Beowulf systems are likely to be communications limited even as faster networks become commodity products. The performance advances of the emerging Gigabit per second commodity networks are being offset by the increase in demand resulting from faster processors and I/O buses.

		10.7 New Programming Models

		The limitations imposed by communication latency go beyond performance issues. As indicated above, managing latency even influences the programming model adopted by most Beowulf applications. While efforts have been made to devise distributed shared memory systems on top of Beowulf-like systems, they have been unable to hide the latency from the programmer. Because of this, programmers of Beowulf are always conscious of the need to manage locality of data and program flow in order to mitigate the effects of latency. Techniques include data partitioning, static and dynamic task distribution and load balancing, message buffering and other means of reducing communication and taking best advantage of existing resources. But the fragmented name space imposes the additional burden of working with a non-unique mapping between variable names and physical locations. Some attempts to minimize this particular consequence of distributed computing have been made with implicit programming models such as BSP and HPF. But even these require careful attention to data partitioning and distribution. An added complication is the possibility of non-uniform access times resulting from complex caches and networks. New approaches to programming are needed that strike the right balance between expressiveness and abstraction. The programmer wants to specify what he knows about the data structures, control flow, etc, and then leave the rest to the compiler or runtime system. It is easy to err in on direction or another, i.e., either allowing so much abstraction that good performance cannot be