Section 2.3. Distributed Computing

2.3. Distributed Computing

Where the term service is used in this book, we usually assume that the service does not necessarily reside on the same physical machine as the calling application. The capability to call a remote computer program from another computer program in a seamless and controlled way is at the heart of our understanding of services.

An important challenge of distributed computing is to find abstractions for both remoteness and the actual service task at the same time. Although remoteness can be hidden from the service consumer, the service provider must still choose its invocation scenarios with the right granularity.

The distributed computing infrastructure was developed in the last 30 years. Business computing originally meant mainframe computing and involved large computers with costs in the multimillions of dollars, performing tasks mostly on their own. At some point, they morphed into more interactive multi-user systems. Rather than distributing the computing power, only data capture and display was distributed using terminal devices such as the DEC VT100 or the IBM3270. Some of the first things such systems had to share among themselves were data and output devices such as tape recorders or printing systems. In the early 1970s, computers became smaller and cheaper. The price/performance ratio made computer technology suitable for a broad range of applications. Research institutions quickly realized that they could operate both more economically and more independently when they were able to use various small computers rather than one mainframe system. At the universities of Stanford and Berkeley, two research programs eventually led to the creation of the Unix operating system. The Stanford University Network spun off the company Sun Microsystems in 1982, which is today one of the largest vendors of Unix computers. Unix is different from its predecessorsand many of its successorsin that its design quickly adopted the network as an essential part of the system environment. Two ideas fostered this design perspective: facilitating remote control of computers and programs and providing services to other computers in the network. The first train of thought created tools such as telnet and the Berkeley r-tools suite. The second one featured remote printing and the transparent provision of storage spacethe NFS file system released by Sun Microsystems in 1984. In fact, the latter was the original raison d'etre for the SUN-RPC standard, the first Remote Procedure Call system.

Even though distributed computing was available as early as 1980, it was mainly confined to the academic world until well into the 1990s. Unix computers mainly acted as so-called workstationspowerful computational and visualizing enginesin research facilities and universities. In the business environment, a hub-and-spoke distribution model prevailed until well into the 1990s. A number of desktop computer systems would typically access a central system for storage and printing. Often, these file servers used an entirely different operating platform from its clients. As structured and relational databases became more mature, businesses adopted a client/server approach. A large chunk of the application resided with the client that remotely accessed a database server. Execution logic was split between client and server as databases, most notably Sybase, introduced the concept of functions that were executed within the database and that did not need to be shipped with the client applicationso-called stored procedures. Another remarkable innovation was Novell's NetWare Loadable Modules (NLM), which were programs that ran on the server.

The next logical step was to blur the distinction between the client and the server. Combining concepts from distributed computing platforms such as the Distributed Computing Environment (DCE) with the newly emerging paradigm of object-oriented development, CORBA (Common Object Request Broker Architecture) was created. Instead of providing servers, which expose large numbers of remotely accessible functions, the functionality was now broken down into uniquely identifiable, remotely accessible objects that were able to manage their own state. Different objects could communicate with each other by means of an Object Request Broker (ORB). To connect to these objects, no knowledge of where they actually reside is necessary. Instead, an ORB provides abstraction mechanisms, such as naming services, which take care of the runtime discovery of the objects. Similarly to object-oriented programming, CORBA embraced the concept of programming by interfacein fact, all CORBA objects can be implemented in various programming languages, while their interfaces are described using a common Interface Definition Language (IDL).

Technically very elegant and sophisticated, CORBA promoted the actual reuse of live objects but lent itself to a design with rather small entities. CORBA's vision of distributed business objects never fully materialized because its fine-grained model often proved to be too complex to be suitable for the purposes of enterprise-level software reuse. In addition, CORBA programming is rather demanding for inexperienced developers because they must cope with many complex technical concepts, such as the non-trivial language mappings of the CORBA IDL. However, CORBA is still a widely used distribution technology, especially in telecommunications and financial services, although we rarely see the full feature-set of CORBA used in these applications (see Chapter 9, "Infrastructure of a Service Bus," for more details).

As a result, the evolution of distributed infrastructures changed its direction in the mid 1990s, taking the limitations of the early distributed object architectures into consideration. In a new attempt, the idea of clustering a set of objects into a single server was developed to provide a higher level of abstraction and to increase the richness of the service that such a server could offer. Driven by the high demand for more sophisticated platforms for Internet applications, Sun Microsystems introduced Enterprise Java Beans (EJB) in 1997. Similar to CORBA, EJB also relies on a distributed object model. However, the EJB model is based on a controlledand therefore usually limitednumber of servers that host the actual objects. EJB was inspired by CORBA but also by older application containers and transaction monitors, such as CICS or Tuxedo. EJB caters for different types of objects, including data-centric entity beans, as well as session-oriented objects. For example, stateless session beans do not need the concept of instance identity. Another strength of EJB is the container concept, which is responsible for the management of resources (objects, connections, transactions, etc.) in an EJB server. Although not a new concept (the core resource management concepts found in EJB can be traced back to CICS and other mainframe transaction monitors), EJB put a lot of effort into making resource management as transparent to the developer as possible. Finally, similar to other remote computing platforms, such as DCE and CORBA, EJB includes higher-level technical services, such as transaction management, naming services, and security.

In addition to core-application remoting technologies, such as RPC, CORBA, DCOM, and EJB, the 1990s saw the emergence of a large number of additional distributed computing middleware solutions, addressing distributed transaction management (e.g., the X/open-based CORBA Object Transaction Service, the Java Transaction Service, and the Microsoft Transaction Server), messaging (e.g., CORBA Notification and Java Messaging Service), EAI (Enterprise Application Integration, often including message routing and transformation, as well as application adapters), security (most notably the Secure Socket Layer), and many other problem areas.

Although they provided a great infrastructure for the development of individual systems, the sheer number of different distributed computing concepts, standards, and products also caused a problem: middleware heterogeneity. Given that most of the middleware was initially developed to address the problem of application heterogeneity, this is an ironic development, caused by the fact that it proved to be literally impossible to impose enterprise-wide standards for middleware in large enterprises (refer to Section 1.6 in Chapter 1, "An Enterprise IT Renovation Roadmap").

As a result of this development, XML became popular in the mid 1990s as a middleware-independent format for the exchange of data and documents between different applications. XML is basically the smallest common denominator upon which the IT industry could agree. Unlike CORBA IDL, Microsoft IDL, or Java interfaces, XML is not bound to a particular technology or middleware standard and is often used today as an ad-hoc format for processing data across different, largely incompatible middleware platforms. XML does not require a heavy-weight infrastructure, such as an ORB, and it comes with a large number of tools on many different platforms, which enables the processing and management of XML data, including different open-source parsing APIs, such as SAX and DOM. XML's inherent flexibility positions it as the most suitable standard for solving the application heterogeneity problem, as well as the middleware heterogeneity problem.

However, XML's great flexibility is potentially also its biggest problem because efficient application integration and data management requires higher-level data structures and messaging formats. As a result, a plethora of higher-level standards that attempt to address these issues has emerged in XML space. It took a long time for people to agree upon even the core standards for the specification of XML-based complex data types, such as XML DTDs and Schemas.

Recognizing the need for higher-level XML messaging standards on one hand, and attempting to leverage the ubiquity of the Internet on the other, engineers at Microsoft invented XML-based Web services with the creation of SOAP (Simple Object Access Protocol) in 1998. The initial SOAP version was specifically designed to work on the widely established HTTP Internet protocol, enabling server-to-server communication over the infrastructure that was already established for browser-to-server communication. Given that the established Internet infrastructure already solved many pressing problems, such as security (SSL, firewalls, access control, etc.), load balancing, failover, and application management, this seemed like a logical next step.

Using standard HTTP POST and GET request, SOAP clients were able to call functions using the established Internet infrastructure. Microsoft later also developed an interface definition language for SOAP services called WSDL (Web Service Definition Language). WSDL describes the service interface, much as IDL describes the object interface when using CORBA. With the problem of middleware heterogeneity in mind, SOAP and WSDL are designed to enable the definition of various bindings to different lower-level communication protocols, for example, to enable SOAP communication over an existing messaging middleware. SOAP was carried forward by IBM and Microsoft and became a W3C recommendation in 2001.

As we see later in this book, XML-based Web services (e.g., based on SOAP and WSDL) can be a great platform for a Service-Oriented Architecture. However, it is important to keep in mind that Web services are not the only viable technology platform for an SOA because the SOA architectural concepts are not dependent on a single technology platform.

The development of distributed computing architectures, such as DCE, CORBA, DCOM, EJB, and XML Web servicesand the real-world experience won with it in large-scale application developmentshas provided the basis for the concepts of Service-Oriented Architectures. Similar to object orientation, which today seems to present the endpoint in the development of programming concepts, service orientation is the result of a long evolutionary process and has the potential to finally provide some stability in an environment that has been constantly evolving over the past 30 years.