System software development is a slow, tedious, and bug ridden process. Because of the relatively small market, vendors may not be able to maintain, let alone extend, the parallel system software of their high end MPPs. Software environments for new MPPs have been a source of continuous difficulty for the HPC community. In contrast, Beowulf leverages the heavy software development investment made by the academic and research computer science community. Linux and MPI are just a couple of examples of successful complex software projects that have been carried out in this context. In fact, there is now a trend of commercial vendors at least partially embracing these same software components.
Beowulf uses the same semantics and syntax for message passing that commercial MPPs use so that they provide the same user application programmer interface for that programming model. Programmers familiar with such machines as the IBM SP or the Intel Paragon are able to work directly with Beowulf without a significant learning curve. Beowulfs have been used as development platforms for software destined to be run on commercial MPPs as it is much cheaper to use a Beowulf while the software developer is given complete control over the low cost machine during the buggy period of code development.
It may be surprising, but Beowulfs are highly robust, mature, and stable as well as accessible to users. After a brief burn-in period, Beowulfs have remained continuously operational for more than half a year without rebooting.
2.4 A Critical Technology Convergence
The fundamental concepts underlying Beowulf-class computers come from multiple sources and in some cases have a long heritage. If so, then why are Beowulfs only now receiving such wide attention and why has there been a recent explosion in their implementation and application? The reason is that the evolution of certain trends both technically and in the marketplace have only recently combined with sufficient compelling force to advance the Beowulf model with respect to alternative approaches. Only in the last few years have the hardware and software technologies achieved sufficient capabilities, while market forces have opened the opportunity for cost effective Beowulf systems.
One important factor is the high performance computing market and the commercial response to it. Unfortunately, while many other areas of the computing industry are experiencing a period of rapid growth, high performance computing is static if not shrinking. Most companies with an exclusive focus on the high end have either gone out of business or been subsumed by other larger companies. Few
