Chapter 26. How Grid Computing Works

The world's biggest supercomputer is not an individual machine with massive processing power. Instead, it's a collection of tens of thousands of normal-sized individual computers like the one sitting on your desk, whose power can be combined into one giant system.

That's the promise of grid computing. And it's not just a promise. It's already working.

The theory behind grid computing is fairly straightforward. It combines the computational power of many computers, servers, databases, and other computing resources in a network or across the Internet. A main server or servers breaks up large computational tasks, such as solving complex problems relating to biochemistry, into smaller subtasks and sends those subtasks to computers that are part of the grid. When those computers have idle time and resources, they perform the subtask, and send the results back to the server or servers, which collates all the subtasks and solves the computational problem.

There have been many grid implementations, but the largest and best-known so far is the Search for Extraterrestrial Intelligence project (SETI@home), overseen by the University of California, Berkeley. In it, four million people in 226 countries downloaded a screensaver that kicked in when their computer was idle, and analyzed radio signals for signs of extraterrestrial intelligence. SETI@home scientists claim that the combined resources of its computing grid were more powerful than all the supercomputers on the planet combined. Similar grids have been used for mapping genomes and analyzing proteins.

Similar grids, however, will be increasingly used by corporations to apply their own computing resources to large problems. In this kind of architecture, all the computers in a corporation would register themselves with the corporate grid, and when idle time was detected in computers, they could be applied to other tasks. For example, if a server is normally used only a few times a month to process a corporate payroll, it could be used for other computing tasks when it would be otherwise idle if used as part of a grid.

Companies such as IBM and Sun are increasingly using grid computing as well. For example, IBM has launched a multibillion dollar "on-demand computing" program in which businesses can outsource computing tasks to IBM, and IBM in turn uses a grid to do that processing.

To date, most grid projects use their own individual architectures. However, there is an attempt to create a set of standards that can be used when building grids and grid projects. The standards are being established under the aegis of the Globus Project, which already has a set of proposed standards out, and has released toolkits that developers can use to help build grids.

It's likely that in the long term, grid computing will become a standard part of the way that businesses and the Internet work. It will have moved from the search for extraterrestrial life to the center of corporate life.