| The idea of on demand computing is to allow companies to pay for only the amount of computing power they use, as they would pay for electricity from the power company. IBM's strategy for "on demand" offerings is to create a computing infrastructure using servers, storage, middleware, and distributed networks that is seen by users as a flexible, integrated common resource that can span traditional enterprise boundaries. In an on demand environment, IT resources are gathered from multiple physical locations to create an integrated "virtual" computing environment. Traditional IT environments typically dedicate IT resources to applications. Substantial economic and competitive efficiencies are gained using this strategy, since virtualized resources can be shared between applications to achieve much higher utilizations . On demand deployments have already been initiated during 2003 and 2004 in large enterprises , including the IBM environment to save billions of dollars ”e.g., supply chain management applications. An on demand infrastructure is the basis for business transformations, including being a services delivery platform for e-hosting, web portals, and mission-critical applications. Grid computing is one of the first logical steps to creating an on demand environment. Virtualizing Resources: VMware and IBM Virtualization Engine VMware was founded in 1999 to offer server management software that allows mainframe-class virtual machines to be created using industry-standard computers (see http://www.vmware.com). VMware is one of the proprietary and evolutionary means to manage or virtualize Windows and Linux servers for "on demand" offerings to gain efficiencies with clusters of servers. An example is that IBM can often consolidate eight to 10 customer Domino application servers onto a larger one using VMware. A simple "rule of thumb" for migrating to VMware is to add up the processing power of the source servers and compare that to the processing power of the target server. If the target xSeries 445 servers have four 2.4Ghz processors, the total Ghz of the target server is 9.6Ghz. Based on that simple guideline, we could consolidate about 10 single processor NT boxes with average processor power of 1Ghz onto one IBM xSeries 445 using VMware. Announced in April 2004, IBM's Virtualization Engine (VE) software suite is a major industry advancement in managing servers, storage, and network resources across multiple locations and heterogeneous platforms (see www.ibm.com/servers/e-server/about/virtualization). At one end of the suite, the software allows "micro-partitioning" of a single CPU into ten pieces, thus allowing a 4-way system to have 40 CPUs, each an independent fault-tolerant partition to maximize system utilization. IBM VE suite includes TotalServer and TotalStorage technology to enable enterprise optimization of heterogeneous pools of servers, storage file systems, and disk volumes across networks in a secure and efficient environment, something not easily done before. To the user who runs applications in the IBM VE environment, the resources are "virtualized" as a common resource, and this offers powerful new control over a heterogeneous infrastructure. Grid Computing Grid computing is a major evolutionary step that virtualizes an IT infrastructure. It's defined by the Global Grid Forum (www.gridforum.org ; see also www-fp.mcs.anl.gov/~foster/Articles/WhatIsTheGrid.pdf) as distributed computing over a network of heterogeneous resources across domain boundaries and enabled by open standards. While the industry has used server cluster technology and distributed computing over networks for nearly two decades, these technologies cannot in themselves be grid computing. What makes grid computing different is the use of open source middleware to virtualize resources across domains. Middleware makes it possible to apply policy-based control to the management of those resources. Key factors that differentiate grid computing from previous distributed computing technologies are as follows : -
The computers and resources involved can be heterogeneous. They can run different operating systems and use different server and storage platforms that may even be owned by different individuals, companies, labs, or universities. Grids may include clusters, individual servers, or entire data centers as part of the resources that are virtualized for sharing with other members on the grid. -
Grids are based on open standards such as Globus and Global Grid Forum based on protocols like SOAP and WSDL (Web Services Description Language) to define all resources as services, for discovery, management, security, etc., in a layered OGSA industry-approved Open Grid Services Architecture (OGSA). -
Grids are capable of massive scaling to share and manage applications, data, and resources between local, campus, regional, or international locations. -
Grids are based on policies created to define job scheduling, security, resource availability, and workload balancing across multiple applications and user domains. Grid computing can be thought of as a services delivery platform. Multiple users can launch jobs into the grid environment running simultaneous applications with access to multiple databases over networks. Grid computing offers individuals, national labs, universities, and commercial enterprises a previously unavailable flexibility to perform internal IT computing tasks faster and at much lower costs, to manage critical capital expenditures, to leverage open standards, and to access new markets. Grids enable new or improved services that were simply not possible in the past. For example, small bio- genetics companies can now have easy access to massive grid computing services (IBM Deep Computing) or a large international bank or telecommunications company can grid-enable their customer web portals or large call centers to optimize costs and to win in a competitive environment. Around 2001, it became apparent that the mix of grid software, high-performance computing applications, proprietary solutions from vendors , and increasing need for individuals to collaborate across a grid had become a real challenge for interoperability and infrastructure management. It was also apparent that while many grid efforts were remarkably successful, a new stage of common development and open standards would be required to make grid computing infrastructures a commercial success. In the commercial world, applications have been typically "pinned" to dedicated resources within administrative domains under layers of security. Expensive IT resources are typically underused in anticipation of peak capacity loads and the launch of new commercial services is gated by the availability of capital funding to buy new platforms. A powerful yet flexible framework would be needed ”but one based on open standards. IBM is an acknowledged industry leader in grid computing and a key collaborator in defining open standards. To enable grid computing and on demand, open standards have been and are being adopted to create flexible, reusable components that can be inserted into a layered "services-oriented architecture" (SOA). An example of a services-oriented architecture is the well-known Open Services Grid Architecture (OGSA) now adopted by the industry through the Global Grid Forum. IBM's own on demand architecture is itself a SOA that will embrace the OGSA. The key concepts are that open standards allow applications to ride on a layer of middleware (OGSA) which, because of flexibility and reliability needs, calls for resources in the physical infrastructure below. Web Services hold the key to grid computing for commercial applications and business applications that will take advantage of the grid through XML, UDDI, SOAP, and WSDL. A major advancement in early 2004 occurred with the public announcements by GGF to "refactor" the OGSA grid standards to merge with Web Services standards to create a WS-RF Web Services Reference Framework and corresponding family of Web Services standards through year 2004. Adoption and development of standards are underway by the international grid community comprised of more than 100 companies in the Global Grid Forum, researchers, and commercial software providers to extend existing grid standards. An important observation is that most commercial applications ”e.g., in banking, life sciences, manufacturing, and telecommunications ”have not been written specifically to run on a grid. Yet since 2003, IBM has successfully launched nearly 200 grid engagements with 35+ public grid references and has found that many stable UNIX-based applications are capable of running in an OGSA grid environment without major software porting. Specific knowledge of the application, workflows, architecture, and platform performance is required to extract the most business value from specific grid implementations . Grid Computing ”The Reason The adoption of grid computing by commercial enterprises for mission-critical operations is rising quickly in 2004 and is seen by business and national leaders as a way to effectively transform their enterprises and organizations. Grid computing is a practical first step to creating a scalable, responsive "on demand" infrastructure. There are powerful business reasons to build or join a grid services infrastructure: -
Dramatically improve % utilization of dedicated IT resources that are now typically "pinned" to an application or platforms, either within or between enterprises. -
Enable new services to customers at lower cost and faster time to revenues . -
Create a highly resilient infrastructure (an inherent feature of grids) for disaster recovery or mission-critical "business continuity" services. An example of the first business reason to deploy grid computing is emphasized by the following table (see Figure 15-20) derived from market studies from 35 large global companies (see IBM Scorpion Study 2001) that shows a low average % utilization for IT resources. The results have remained fairly consistent for different enterprises and reflects some of the realities in which capacity is overprovisioned and sometimes guarded with anticipation of peak demands. In addition, other studies have shown that people and administration, maintenance, and service comprise some 60% of the overall cost of typical data center operations, while the other 40% of costs is from all hardware/software platforms. One can easily observe that low average % utilization of hardware/software platforms has a corresponding cost in effective use of skilled people. Grid computing offers a powerful solution to breaking down the barriers created by IT resources that are held in silos and have low utilization. Sharing the resources with grids has shown dramatic (2 to 200 times) improvements in process cycle times with huge reductions in costs for those steps. Once higher value business applications are put onto a grid, they typically expose the next weakest link in business application processing. A grid computing infrastructure creates a services delivery platform that will lower costs, help fund the next round of improvements, enable flexible launch of new services, and create a " virtuous circle" of valued improvements. See Figure 15-21. Figure 15-20. Grid computing ”the reason. 
Figure 15-21. Grid computing ”the concept. 
Grid Computing Is Becoming Reality Commercial adoption of grid computing during 2003 and 2004 has been rapidly lead by six active industry global segments: R&D, life sciences and pharmaceuticals , government services, finance services, industrial and manufacturing, and most recently, telecommunications. Leaders in grid computing like IBM have focused on building grids for mission-critical or highly-valued categories of applications. These are: -
"Business analytics acceleration" of financial derivatives portfolio, insurance actuarial calculations, seismic and petrochemical calculations -
Engineering and design for CAD/CAM in aerospace, automotive, and electronics -
R&D in life sciences for modeling bimolecular components, agricultural chemicals -
Government and organizational collaboration by diverse users across grids -
Enterprise optimization of business processes in petrochemical, finance, and telecommunications Grid computing is in an early commercialization stage but is already providing impressive business value. OGSA and Web Services have addressed lower layers of the architecture to drive interoperability and these standards have been launched into the final approval process during the first quarter of 2004. Other goals of the OGSA standards development are given in great detail on the Global Grid Forum site (www.gridforum.org), in which the family of Web Services standards will be merged. Web Services enables better interoperability and improved distributed system integration. Finally, grid computing needs to develop a base of deeper commercial experience to address future opportunities in which grid services providers can offer services to unknown third parties, allow secure access for their services, and accurately bill for services rendered, all in an automatic manner. The technology underpinnings for this are several years away. The History of Grid Computing Grid computing for commercial applications has been successfully adopted in 2004 by executive leaders in companies in Europe, Asia, and the Americas. Now it has been extended well beyond the mix of distributed computing technologies originating from various universities, supercomputing centers like San Diego, national labs like Argonne, and international scientific consortia like CERN during the late 1980s through the 1990s. One of the origins of grid computing can be traced to cluster computing. Clustering was made possible by the advent of powerful desktop workstations in the 1980s. These machines, which could deliver nearly one million operations per second, were easily affordable and connected by high-speed networks. It wasn't long before researchers began to make these workstations work collaboratively. To gain access to new computing power that would be created by sharing resources, various researchers wrote their own software and collaborated to support the many specific implementations of early grid computing. A critical enabler was the rise of new Internet standards and protocols (see the seminal reference on grids, "The Grid: Blueprint for a New Computing Infrastructure" by Foster & Kesselman, 1998). This activity resulted in the development of the earliest-known grids of computers and enabled large-scale, resource-hungry problems to be solved cost-effectively. Several early grid initiatives have attempted to provide a more formal model for sharing of resources in wide area cluster environments. The Business Benefits of Grid Computing Grid computing is a first logical step that can be taken by individuals or an organization to building a more powerful, lower total cost infrastructure. After business process re-engineering, workload consolidation, and simplification of the IT environment, executives should logically consider grid computing as part of their strategy to transform their businesses. Grid computing environments have proved valuable in a variety of application domains. Industry leaders have chosen grid computing to win commercial benefits in applications for business analytics acceleration, as well as enterprise optimization (see the previous section). Although grid computing infrastructures conform to the OGSA and emerging standards, they can be implemented and optimized for their intended applications. One can think of a "style" of grid computing implementation represented by business analytics acceleration that is more computationally intensive, while government organizations require more collaborative use of common applications and synchronized databases to create a "collaborative grid." Some applications require a hybrid grid environment combining computationally intensive and collaborative styles to address "enterprise optimization." An example of the latter might be a grid to support massive billing systems used by telecommunications companies. Examples of grid application domains and their applicability to business environments include the following: High-performance distributed computing applications leverage large numbers of network-connected computing resources so users can collaborate to solve complex problems. By distributing the work, you can reduce the amount of computing time needed to complete an application. Grid computing dramatically increases the number of computing resources available to a distributed computing application. In a business environment, this capability enables an enterprise to improve the quality of service of its distributed applications, even when unplanned events that affect the enterprise occur. Collaborative support has been traditionally used for activities such as enabling geographically dispersed teams to simultaneously analyze information contained in scientific datasets and displaying this information graphically for ease of analysis. In a business environment, companies can leverage the mechanisms grid computing provides for collaborative support across multiple departments and across multiple companies while protecting intellectual property and confidential information. Furthermore, these collaborative capabilities support virtual collaborative organizations (i.e., multiple businesses pooling their resources to solve a mutual problem), in which the servers and storage devices contributed by each participant work together seamlessly and optimally. Grid computing also helps to enable the development of on demand utilities. Through the use of advanced resource sharing capabilities provided by the grid, an on demand utility can offer computing power and storage resources to companies that subscribe to these services, and these companies use only what they need. This approach can save companies substantial IT investments while giving them access to supplemental computing power and storage resources during peak usage periods. Remote access to resources is an inherent feature of grid computing environments in which the scientific community already has demonstrated access and use of remotely located telescopes and data mining of medical databases over the Internet. In a business environment, the same mechanisms that make these operations possible also can be used to securely access and exchange distributed sources of business information in real-time and at high-speeds. A Real-World Business Grid Application To better illustrate the applicability of grid computing to business applications, we'll examine a real-world private exchange application that benefits from the grid's capabilities. Let's assume that our private exchange application enables product suppliers to make available information about their products (e.g., product description, availability, and prices) to buyers who browse catalogs to find products of interest. Also, the exchange is considered private, as it's accessible only to registered buyers and suppliers. Let's also assume that we have thousands of registered buyers and suppliers and a product catalog that contains millions of products. In this environment, each supplier has its own enterprise information system where it stores its product information. However, registered suppliers must also store all their product information in a large global exchange repository. Furthermore, because each supplier stores its product information in a proprietary format, each supplier is responsible for converting its format to the format supported by the exchange repository. Additionally, the supplier is responsible for publishing the product information in the product catalogs provided by the exchange; this requires yet another transformation of the information in the supplier's information systems. Suppliers also typically need to synchronize their product information with the global exchange repository to update pricing and description information. Registered buyers can then browse the product catalogs created by the suppliers to identify the items they wish to purchase. Because the registered buyers and suppliers need a mechanism for implementing the private exchange, they set up a grid computing environment where they can contribute servers and associated storage. To manage the computing and storage resources, these buyers and suppliers use grid service middleware so that they can share resources in a more seamless and transparent fashion. Following are the benefits the grid service-based computing environment provides for our private exchange application: Sharing of storage resources allows the cost of storing the global exchange and the product catalogs to be distributed among the buyers and suppliers that have contributed storage to the grid computing environment. How the buyers and suppliers choose to distribute the cost of storage requires some negotiation between the parties involved. The ability to share the cost of storage is significant, as both of these repositories have massive storage requirements. Moreover, even though the storage has been distributed throughout the grid, the grid service infrastructure virtualizes this storage support such that it appears available as a monolithic resource from a centralized hosting site in the environment. Sharing of processing resources enables several different servers connected to the grid to share and perform the work of transforming product information into the formats supported by the exchange repository and the product catalogs. The grid service infrastructure handles the processing at each server, and as a result, it appears to each supplier as if it has a much larger cluster of servers available for performing this computing-intensive task. Improved business resiliency, reliability, and quality of service for accessing product catalogs results from the availability of a larger pool of servers. This greater number of servers allows the hosting of product catalogs at a variety of sites in the grid, improving reliability should an unplanned outage occur at some of the locations hosting portions of the grid. In addition, during peak product catalog usage periods, numerous resources can be dedicated to the hosting of product catalogs on servers, thus improving the quality of service for those accessing and searching the catalogs. OGSA and WebSphere Open Grid Services Architecture (OGSA) is a services-based architecture that builds on Web services technologies. IBM is working with Web services standards organizations to ensure there's a single set of standards that meets the needs of both conventional applications and applications that are optimized for a grid environment. The WebSphere software platform will provide tools and runtime support that leverage these standards. IBM will also use this platform to provide a high-quality implementation of OGSA that will be available on IBM and other vendors' server hardware. WebSphere will likely become the runtime environment of choice not only for the grid infrastructure but also for applications as they evolve to leverage the benefits of a grid environment. Because OGSI and OGSA are open standards, application developers can use any vendor's implementation of the hardware and software infrastructure to build and deploy their applications. IBM's Grid Strategy IBM is addressing Grid Computing across four dimensions ”storage virtualization, global file systems, database management, and system management. The company's strategy concentrates on five focus areas ”research and development, engineering and design, business analytics, enterprise optimization, and government development ”for customers in the aerospace, automotive, financial, government, and life sciences industries. IBM is strongly committed to the Open Grid Services Architecture (OGSA ”a set of specifications and standards that form a common framework for building commercial grid solutions. The company also actively participates with the Global Grid Forum in the development of the standards. IBM's grid offerings, as well as their server, storage, and middleware products, will support OGSA. Over 100 Grid Customers Up and Running Although grid technology is still very much under development for corporations, IBM has more than 100 grid customers up and running worldwide. The latest grid offerings are for the financial services industry. IBM and software provider, SAS, have teamed to provide a new Grid offering in the banking industry. Using the offering and its own network and IT resources, a bank can use its data to better understand its customers and their banking needs, reducing its cycle time for executing statistical models and providing more sophisticated and accurate customer analysis. This should help banks better identify, acquire, and retain customers. The other grid offering, developed in conjunction with DataSynapse, is designed to help risk managers implement a grid infrastructure to support real-time credit limit monitoring. The offering can also form the foundation for a credit risk application infrastructure that complies with banking organization and Federal government reporting requirements. Examples include the Patriot Act, which provides U.S. law enforcement officers with information and tools to investigate and prevent acts of terror, and recently toughened New York Stock Exchange reporting requirements. Real-World Examples of IBM's On Demand Implementations Here are some examples of IBM's On Demand implementation for customers. On Demand at the U.S. Open Tennis Site Figure 15-22 shows the home page of the U.S. Open Tennis Site. The U.S. Open infrastructure, which is run by IBM, when not being used for U.S. Open Tennis, is used to explore protein folding ”as a preview to the Blue Gene Project. Figure 15-22. Grid computing with the 2003 U.S. Open Tennis Web site infrastructure. 
Blue Gene Research on the U.S. Open Tennis Web Site Infrastructure Blue Gene is a super-fast computer. While it's true that Blue Gene promises to break all records in terms of processing speed (a quadrillion calculations per second, or one petaflop) and sheer number of processors (65,000 nodes for Blue Gene/L, the first in the Blue Gene family targeted at 360 teraflops), an interesting aspect of the story involves Blue Gene software. The U.S. Open infrastructure has been used as a grid to help run protein folding simulation experiment. The Computational Biology Center at IBM Research is running Blue Gene science applications today using the U.S. Tennis Open infrastructure for a grid before the actual research version of the Blue Gene supercomputer is available. IBM scientists in labs around the world have been developing applications that will allow life sciences researchers, for the first time, to explore the intricacies of protein folding. The Blue Gene science program is already in full swing, well in advance of the hardware. When the technologies are not needed for tennis, this same infrastructure is being used for other workloads ”in this case, running applications for protein folding. IBM researchers are using capacity at the U.S. Open for three major projects, as discussed next. Understanding Protein Aggregation Blue Gene software will help researchers look not just at how proteins fold up to their correct biological structure, but also what happens when folding goes awry. Misfolded proteins can clump together, or aggregate. Different proteins have different propensities to aggregate, and these aggregates can be completely disordered (like pieces of spaghetti tangled together) or show some degree of order (like Lego bricks stuck together). Amyloid fibrils, a class of protein aggregates, are involved in a number of major diseases, including Alzheimer's, Creutzfeld-Jakob Disease (prion), and transthyretin amylodosis. A research group at UCLA has recently reported the discovery of a tiny protein fragment that can form amyloid-like aggregates. IBM researchers are using the U.S. Open infrastructure to carry out simulations on concentrated solutions of these four amino acid peptides to help understand the process of protein aggregation and to give a physical picture of the interactions that stabilize the aggregated state. Folding Simulations of the Villin Headpiece The villin headpiece is a 36-amino acid alpha-helical protein and is one of the fastest folding proteins known (it folds in less than 10 microseconds, which is ten-millionths of a second). As part of the Blue Gene project, IBM is trying to understand why this protein folds so quickly. Since protein folding is a statistical process, the researchers need to simulate numerous folding events that may occur by a number of different pathways in order to determine the rate and the overall process of protein folding. That type of simulation takes enormous computing power. Simulations of Protein-Protein Interactions IBM researchers are also doing simulations of a protein-protein interaction module composed of 35-40 amino acids. This module is a subject of general interest because several signaling complexes that the domain mediates have been implicated in human diseases including Muscular Dystrophy, Alzheimer's Disease, Huntington Disease, Liddle Syndrome of hypertension, and, recently, in cancer. Grid Experiment Turns into Real-World Healthcare Business Another example of a real-world grid and on demand system is a digital medical archive ” funded by IBM and the National Library of Medicine. The archive of digitized mammograms, which draws on the shared computing resources of hospitals in Chicago, North Carolina, and Toronto, is expected to create a comprehensive records system that will facilitate near-instantaneous access to breast cancer screening data and transform the way hospitals deliver healthcare information to doctors and patients . The National Digital Mammography Archive (NDMA) is another example of on demand technology. Factors such as shared computing resources, open standards, and "autonomic" self-managing servers are making it possible for institutions across North America to distribute critical information quickly, cut waste out of supply chains, and reduce administrative costs associated with phone or paper-based processes. In the case of NDMA, server clusters, or grid technology, are putting high-resolution medical images, records, and clinical history in the hands of radiologists at three hospitals, letting them compare an individual's mammography records to make fast, reliable diagnoses. On Demand Is Still Evolving As some 200 medical centers in the U.S. and Canada log onto the scalable, secure, and economically feasible grid, breast cancer data will reach more radiologists as soon as they need it. Patients will be the ultimate beneficiaries. So although grid and on demand technology are still evolving, there are many real-world examples in the works. Grid and on demand technology both provide an excellent base for collaboration tools for the enterprise. |
