Section 5.3. The enterprise services layer

5.3. The enterprise services layer

While it is clear from the term that enterprise services are the central element of ESA, it is far from clear what that means in detail. One measure of expertise in ESA is the length of the answer someone can give to the question, what is an enterprise service? The more dimensions that someone can discuss with some degree of substance, the more likely that person will be able to quickly make sense of ESA and put it to work in an organization. This section will take you on a tour of all of the dimensions of enterprise services.

5.3.1. What is the basic structure of enterprise services?

The key to understanding enterprise services is the idea that enterprise services are primarily two things, an interface and an implementation, that are broken into three parts when put into action: the consumer proxy that invokes the service interface; the provider proxy that accepts the invocation of a service, manages the communication with the consumer, and passes the information to the implementation; and the implementation that does the work. Figure 5-4 makes this relationship clear.

Figure 5-4. Enterprise services in consumers and providers

To really understand how all of the parts work together, we will discuss what happens during two phases of design and then look at what happens at runtime.

The first phase to be examined is that of designing an enterprise service. There are two ways that enterprise services are designed: either from the outside in, or from the inside out. Outside-in design happens when process automation is being designed and modeled and an interface to a service that does not yet exist is designed to meet the needs of the process. In this case, the interface is defined in detail, development tools are used to generate the service provider proxy, and then traditional coding methods using Java, ABAP, or other tools are used to create the enterprise service implementation.

In the case of inside-out design, the service implementation exists and the interface must be designed on top of it. Sometimesfor example, when a BAPI or an IDOC is being turned into an enterprise servicethe enterprise service interface can be generated from the definition of the BAPI^[*] or IDOC. Other times, when many BAPIs, IDOCs, or other existing functionality are going to be combined to create a new enteprise service, the service interface must be constructed by hand. The details of outside-in and inside-out development are described in Chapter 15.

^[*] Not all BAPIs can be enabled as enterprise services.

The second design phase consists of the process of using a service in a service consumer, such as a composite application. In this phase, a development or modeling tool uses the description of the service interface that is stored in the Enterprise Services Repository to generate the consumer proxy code that invokes the service by sending XML messages to the provider proxy that ask the service implementation to do the work.

Now we must consider what happens when a service interface is designed. As Figure 5-5 shows, each service interface consists of a set of service operations.

Figure 5-5. Service operations

A service operation represents one of the functions of a service. A service operation is like a method on an object in object-oriented programming, or a remote function call to a BAPI or some other function module in ABAP. As part of the definition of a service interface, each service operation must be defined in detail. For example, for a service that can create a purchase order, service operations must exist to create, read, update, and delete the purchase order. Of course, you must also define the data that will be passed to and from the service consumer and provider. We will discuss how this data is defined in the next section.

Perhaps the simplest way to think of service operations is as the inbox and outbox for services. Each service operation defines what, if anything, will be sent into the service and what will be sent back once the processing is done. It is important to remember that service operations can be synchronous or asynchronous. In a synchronous service operation, the service proxy on the consumer waits for the response before proceeding. In an asynchronous service operation, the service consumer just sends messages that begin the service operation, and then continues its work. Then a response is sent back in another message, when the service operation is complete. Almost all of the interaction for web services has been standardized through the WS-* series of standards.

So, now we come to the subject of what happens between the receipt of an incoming message from a service consumer and the response from the service provider. The answer is that the service implementation accepts the information from the incoming message related to the service operation and then the service implementation does the work, as shown in Figure 5-6.

Figure 5-6. Service implementations

Service implementations are programs, pure and simple. They can be implemented in any language, run on any platform, and do anything that a computer program can do. This separation of how the implementation is performed from the definition of the service is one of the most powerful aspects of enterprise services and services in general. It means that as long as the service works the same way, you can change the implementation without disrupting the service's users. For example, if a legacy mainframe application now is encapsulated as a service, nobody needs to worry about when this application is upgraded to a more modern, lower-cost version because the services hide the implementation and the changes are invisible to the users.

Services, in essence, form a contract between the service providers and the service consumers. The clarity that the definition of a well-defined service provides is being used to formalize all sorts of relationships. IT departments are defining what they offer the rest of the enterprise in terms of services. Companies such as Amazon, Google, and Yahoo! are defining their offerings to developers in terms of services. Suppliers and manufacturers are defining their relationships in terms of services.

In the early days of services, almost all services were implemented as shown in the left side of Figure 5-6. Underneath a service interface was a service implementation that was created through custom coding techniques using Java, ABAP, or other languages. When well-established forms of APIs existed, such as BAPIs and IDOCs, it was easy to generate the code that defined a service's interface for the existing functionality that was used as the service implementation. Many services in use today have been written this way.

But the vast importance of services has changed the way applications are being created to make the creation of services easier, faster, and more flexible. The core problem is this: APIs created for mainframe/client/server applications were created to solve all sorts of different needs. Sometimes these APIs existed to facilitate bulk loading of data. Other times the APIs were created to solve a common integration challenge. Yet other times they existed to support development of UIs. Almost none of the APIs of previous generations was created to provide functionality needed by processes, process steps, and other utilities to support an SOA. The internal architecture of many programs did have reusable components, but these were designed to be reused inside a program, not by composite applications and other service consumers seeking to flexibly automate processes.

When the work of creating services began on top of existing APIs, developers quickly found out how difficult it was to create services using APIs created for other purposes. Sometimes the APIs did not have all the power they needed. Other times they had unintended side effects. When the developers turned to writing new code using the objects for internal use, similar problems arose. For example, it was not uncommon for objects such as sales orders, purchase orders, or invoices to be huge and to have many interconnections to other objects. This was fine for the context of internal use inside an application, but it was too large a granularity to efficiently support services.

The solution to the challenge of creating service providers that could easily support the creation of enterprise services is to organize functionality around the concept of business objects, which are objects that are internal to a service provider and are designed to be reusable building blocks in automating processes and process steps, and in providing utilities to support service consumers such as composite applications. This structure is shown on the left side of Figure 5-6. Inside a service provider, coding in Java, ABAP, and other languages is almost inevitable, but service providers also can use process orchestration and modeling to improve flexibility and reduce the cost of change.

Although this explanation is just a quick introduction to the topic, a clear understanding of the internals of enterprise services goes a long way to reduce the confusion that can sometimes creep into SOA discussions. We will provide this explanation in the next few sections.

5.3.2. What are global data types and how are they related to enterprise services?

Our discussion of enterprise services did not address the issue of how data formats will be standardized, an issue that must be addressed if ESA is to work as efficiently as possible. Let's imagine what would happen without a standard for basic customer information, such as name and address, for example. That would mean the definition of customer information would be different from one service interface to another service interface, which means that one of two things would have to happen: either service interfaces would have to be written to accept customer information in many different formats, or the service consumers would have to do the work of translating the customer information into the format used by each service interface. As anyone in the IT business with a little gray hair knows, this is a massive and annoying problem that has been addressed by general-purpose data mapping technology and special-purpose suites of adapters.

Of course, everything would be so much simpler with just a little standardization so that data types in UIs were the same as those in service interfaces and business objects. That is precisely the job of the global data types used in ESA. Based on the data type methodology of ISO 15000-5 and the UN/CEFACT Core Component Technical Specification, SAP's global data types provide a standardized set of data that is described using XML schemas that are stored in the Enterprise Services Repository.

The idea is that whenever possible, people will use standardized global data types instead of making up new ones. The initial collection of global data types will be expanded using SAP's governance process for business content, which is part of the Process Integration Council, an internal standards setting and engineering governance body at SAP. With global data types in place, ESA can deliver a level of efficiency of which previous generations of enterprise software were incapable.

5.3.3. Does ESA have an Enterprise Service Bus or the equivalent?

In addition to the concept of SOA, another concept called the Enterprise Service Bus has become current. While SOA is the general concept of building software by using services, the Enterprise Service Bus is something much closer to ESA: it is the idea of creating a framework for a standard set of services that can be reused to do the work of enterprise computing. One way to think of ESA is as an infrastructure and set of tools for creating an Enterprise Service Bus. Of course, ESA goes beyond just creating the roadways for a service bus. Unlike most other approaches to SOA, ESA provides services to run on the roadways.

Our discussion so far has papered over many of the details of how services will work, and to complete our study of enterprise services, we should provide these details. Figure 5-7 completes the picture of how enterprise services work by adding a new layer, the Enterprise Services Infrastructure, which does quite a few jobs.

The goal of the Enterprise Services Infrastructure is to hide from service consumers all of the details that go into managing secure, reliable, and optimized communication between service consumers and providers. Here are a few of the most important issues that the Enterprise Services Infrastructure addresses:

Reliable messaging

When a service consumer sends an XML message to a service provider, how does it know that the message was received? New standards for reliable message delivery have been created and are used by the Enterprise Services Infrastructure to perform this task.

Security

Sometimes it is OK to send an XML message across a network without encryption, but other times it would be a huge problem. The Enterprise Services Infrastructure uses standards for security to address this problem.

Figure 5-7. Enterprise Services Infrastructure

Synchronous/asynchronous messaging

A service consumer can call a service synchronously, meaning that the caller waits for a response, or asynchronously, meaning that the caller does not wait and the response is sent later. The Enterprise Services Infrastructure handles the details of these two different types of conversations between service consumers and providers.

Optimizing message traffic

If a service asks that one million rows of a database be returned, the nature and behavior of the messages between the service consumers and the service providers will be quite different compared to a situation in which only one byte is asked for. The Enterprise Services Infrastructure is built to allow different conversations between service consumers and providers to be optimized in different ways. Special care is taken in the Enterprise Services Infrastructure to ensure that UI-related services perform well.

Support for patterns

As we already mentioned, business objects may have methods that are created according to patterns. These patterns may then be reflected in patterns of service operations and may be used to support higher-level patterns for UIs or process automation. The Enterprise Services Infrastructure is aware when patterns are being used and helps optimize their use.

How all of this works in practice deserves a guide of its own, and in fact, senior SAP engineers are working on such guides.

5.3.4. How are database transactions handled in ESA?

Transactions are mechanisms that ensure that all related changes to a database occur either at once or not at all. In ESA, the complexity of implementing transactions is complicated by the fact that data consistency must be ensured across many services that use data stored in different service providers. SAP has developed several approaches to managing transactions that will be made public in future versions of SAP NetWeaver.

5.3.5. This is all rather technical; how do enterprise services get the business meaning they are supposed to have?

So far, our discussion has focused on the form of enterprise servicesnamely, how enterprise services fulfill their mission from the technology perspective. ESA falls apart, however, if enterprise services are not designed properly to fulfill their business mission that is, to fill the role of reusable building blocks for processes, process steps, parts of process steps, and key utility functions. Hundreds of thousands of fine-grained services do nothing to help manage complexity and make applications more flexible and easier to build and change. If a service is too granular, it means the service's consumers have to be fiendishly complex. On the other hand, if services are too large, service consumers lose flexibility. A behemoth service can fulfill one purpose but cannot be recombined to meet other needs.

There is no silver bullet that guarantees enterprise services will be designed to the proper level of granularity. Service design is an art, and it is hard to ensure good art. Fortunately for SAP, experience is frequently a process of good art, and SAP has plenty of that. SAP has many processes in place to prevent bad art from creeping into the process of service design. The ES-Community helps customers, ISVs, and systems integrators (SIs) to design services that will be widely used across the SAP ecosystem (see Chapter 6). SAP has an internal process for vetting requests for services from various development departments inside SAP and from the ES-Community and then making sure that all the requests are reconciled and supported by an inventory of services that has a coherent design at the right level of granularity (Chapter 8 includes a description of this process).

Perhaps the most comprehensive guide to the art of enterprise services design is the Enterprise Services Design Guide, which is available on the SAP Developer Network (SDN; http://sdn.sap.com). This guide explains the context for enterprise services design and presents the outline of a three-step process that involves the following:

Discovery

First, a business-driven analysis reveals the scenarios that might benefit the most from enterprise services. Pain points, opportunities for creating value, and the business drivers of ESA are used to find promising scenarios. The business processes and the existing supporting solutions for these scenarios are then analyzed to determine the requirements for services that would improve support for the business.

Design

Once a promising business process has been identified, the design process begins. A single service, a system of services, or a program of service enablement may satisfy the requirements. In any of these cases, a design must be invented, evaluated, and refined in order to meet the requirements outlined in the discovery process. This is usually an iterative process in which the design work improves the understanding of the requirements, which may then be changed in ways that affect the design.

Documentation

The final step of the methodology involves properly documenting the new services and entering them in the Enterprise Services Repository.

The process of designing a single service or a system of services is set forth in the Enterprise Services Design Guide, along with guidance about the issues that are generally involved. One of the most innovative aspects of the process is the recommended way of running a program of service enablementthat is, looking at a large application, many large applications, or even an entire enterprise, and identifying where services might help.

The idea is that generally you find certain patterns for a common way of improving an application by adding enterprise services when a program of service enablement is underway. The Enterprise Services Design Guide calls these patterns design contexts. For example, in SAP's program of service enablement, in the user productivity areas are several ways that help users interact successfully with computer software. One of those design contexts involves using a work list-oriented UI design pattern to provide users with self-service functionality and reduce dependence on experts. The design context for this improvement shows how a series of UI elements can be used in various types of screens to enable this work. In other words, the design context is a sort of template for a successful UI that enterprise services can support. A design context explains a lot about how the enterprise services should be designed, which should help us achieve the goals of our methodology: to increase productivity and quality. A design context is not a design, but rather the shape of a design that may apply repeatedly.

As a program of service enablement is carried out, designers know that a design context may apply by using indicators, features of a business process that suggest it might benefit from a solution based on the design context. Each design context has its own indicators. One of the indicators for the work-list-oriented design context is when a large group of workers must frequently consult an expert to do certain tasks. This indicator suggests that some sort of self-service could help. (For a more detailed description of the indicators, see Chapter 8.)

The Enterprise Services Design Guide has proven popular with ISVs, SIs, and SAP customers involved in service design, but it doesn't answer one question: how do services fit into general business processes or those of a particular industry? SAP, of course, has had to link business processes to systems that automate them since the inception of the company, and has created an elegant approach that uses SAP Business Maps, solution maps, and business scenario maps to show the process structure of how various businesses translate into SAP solutions. Now, instead of those maps leading to various components of R/3, they lead to services and systems of services that are performing the automation. You can see this approach in action on the Enterprise Services Preview on SDN, which allows solution maps to be navigated down to enterprise services.

5.3.6. How do enterprise services fit into business process modeling?

This question will be answered in more detail in the section "The process orchestration layer," later in this chapter, but the short answer is that enterprise services are the smallest level of container for reusable building blocks for automating processes.

To understand what this means, it is important to understand that modeling of businesses and business processes takes place on various levels. At the highest level, business analysts create the business maps and scenario maps mentioned in the previous answer. These contain various process components, which are large collections of processes, such as order processing and order fulfillment. Inside the process components, business processes are modeled in terms of processes and process steps. When we say modeled, we mean that the relationships among the elements of the model are described. At the highest level, these relationships may represent a large flow of data. At the lower levels, where the processes are described, the relationships may describe which data is being passed back and forth and which enterprise services are going to be invoked. The most detailed description of automated processes exists inside the enterprise services in the code or in the models used to build the service implementations.

5.3.7. What is service composition? How can you build enterprise services from other enterprise services?

Enterprise services are designed for reuse in many different contexts. But a particular composite application may need only part of one enterprise service or a combination of many. Service composition is the task of transforming reusable enterprise services into a single service that meets the needs of one application.

Service composition can take place in service consumers or in service providers, but most commonly, it occurs in composite applications. A variety of different tools can be used for service composition, including the SAP NetWeaver XI Integration Builder and tools that are part of the SAP Composite Application Framework (SAP CAF).

5.3.8. What are the main categories of enterprise services?

Enterprise services are like numbers in that they are generally useful and you can categorize them in thousands of ways. The authors of the Enterprise Services Design Guide created one of the most useful categorizations, and it involves the following four categories:

Component services

These services keep track of the contextthe relationships, data, and external informationrelated to an important business function. A component service keeps track of relationships that are not represented by simply a data field. For example, a service that provides access to the credit limit for a customer and her existing balance is a classic component service. The service would provide not only the credit limit and balance, but also access to external providers of credit ratings, information on currently unpaid invoices, and payment history. Such information could be provided by other services consumed by the credit management service.

Process services

These services trigger the execution of a process or a process step and then manage the execution of that process. Process services assume control of execution when called and then orchestrate services or other functionality to carry out consistent execution of a process.

Entity or engine services

These services model and provide access to a business object or engine. An entity service might represent a create order, which makes sure that every time an order is created, all business rules are checked and all data is consistent. An engine service provides access to commonly used functionality such as a pricing or tax calculation engine.

Utility services

These services perform some commonly used functionality for other services or for consumers of services. A service that provides the allowed values for a specific field (a so-called value help service) is a classic example of a utility service.

Services are sometimes categorized by the business benefit they provide or by the ways in which they assist composite applications. There is no official taxonomy of services yet created.

5.3.9. What kinds of enterprise services support the special needs of composite applications?

Composite applications have special needs and must meet higher expectations than previous generations of applications. For example, composite applications manage information that is distributed across many different databases. Previous generations of applications were usually based on a single database. Composite applications are created in an environment in which many applications have search, collaboration, analytics, reporting, and mobility included. Users have come to expect everything they need in one place, and composite applications must live up to these expectations.

Different kinds of services help composite applications to do their job, as shown in Figure 5-8. While the following categories are not the only types of services that support composite applications, they are the most important in terms of meeting special needs.

5.3.9.1. Master data management

Master data, the data about customers, partners, and other items used repeatedly, must be available for any application to do its job. Composite applications manage master data that is distributed across many different service providers. In such an environment, duplicate data may be stored in different places, data may not be consistent, and no one version of the truth may exist anywhere. SAP NetWeaver Master Data Management is a solution

Figure 5-8. Services that support composite applications

that helps manage this challenge by bringing together all of the distributed master data in a coherent form that eliminates duplicates, resolves conflicts among different forms of data, and provides a way to locate all data items quickly wherever they are stored. SAP NetWeaver Master Data Management also provides applications that allow data to be cleaned. Composite applications rely on MDM for quick access to high-quality master data.

5.3.9.2. Business Intelligence (BI) Consumer Services

SAP NetWeaver BI is a vast part of SAP NetWeaver that uses data warehousing, reporting, OLAP, and a huge menu of adapters to allow consolidation, cleaning, analysis, and reuse of both master data and transactional data. Parts of SAP NetWeaver BI are exposed through standards, such as the XML for Analysis (XMLA) standard that created a query language for OLAP queries that was independent of the implementation. The XMLA standard covers just a portion of the rich and wide pool of functionality in SAP NetWeaver BI, however. SAP NetWeaver BI Consumer Services provides access through services to all of this functionality, not just what was standardized. In addition, SAP NetWeaver BI Consumer Services has advanced techniques to allow data warehouses to be kept up-to-date in near-real time and to optimize the speed of queries. Composite applications benefit from all of this functionality to create a unified view of distributed and possibly inconsistent data, to create special-purpose repositories or data marts to meet the special needs of a particular composite, and to add reporting and analytical functionality to composites. SAP NetWeaver BI Consumer Services allows a composite to present a complete suite of functionality that allows information to be gathered and analyzed in one context, so appropriate action can be taken.

5.3.9.3. Analytics

SAP xApp Analytics refers to the services built on SAP NetWeaver BI Consumer Services and through other means to create a set of tools to make the job of adding analytic functionality to composite applications as easy as possible. SAP NetWeaver BI Consumer Services provides the backend functionality to get access to the data in SAP NetWeaver BI and to do the number crunching. SAP Analytics provide the building blocks, such as UI building blocks, interactive graphics functionality, reporting, and other flexible information containers, that allow users of composite applications to understand the data presented to them so that they can take the appropriate action.

5.3.9.4. Knowledge management

SAP's services for knowledge management bring the world of unstructured documents and search into composite applications. Knowledge management services provide for flexible containers that can associate unstructured documents with structured transactional information. Knowledge management search services allow both unstructured and structured information kept in multiple repositories to be searched to create a consolidated set of results. Using SAP's knowledge management services, composite application designers can bridge the gap between structured and unstructured data and create one complete picture in a composite application.

5.3.9.5. Collaboration

Collaborative functionality such as instant messaging, bulletin boards, email lists, web conferencing, and shared collaboration rooms have become an expected part of modern enterprise applications. The best applications integrate collaborative functionality in the context of an application so that users are not forced to move back and forth between multiple environments to do their work. SAP's collaboration services allow composite applications to bring collaborative functionality into an application context when and where a user needs it.

5.3.9.6. Mobile

Mobility services allow any type of application, whether an enterprise application or a composite application, to extend its reach to mobile devices. It provides one way of adapting an application that will allow it to support any number of specific devices.

All of the services just mentioned serve a similar purpose: expanding the potential of composite applications to meet any need in the most elegant way possible. With these services, composite applications should be able to bring together functionality from multiple service providers in a way that creates a more comprehensive context to meet users' needs.

5.3.10. What development tools are used to create enterprise services?

Enterprise services are created in many different ways depending on what is being developed and who is doing the development. We will discuss this in more detail later in this chapter and later in the book, but to complete our picture of enterprise services at this point, it is worth examining the different ways enterprise services come into being:

From the inside out

You can use SAP NetWeaver Developer Studio or ABAP Workbench along with the Web Service Infrastructure to make existing functionality such as BAPIs or IDOCs accessible as enterprise services. You can expose an existing BAPI, IDOC, or many other sorts of functionality through an automated wizard. If many services need to be combined or new functionality needs to be added, you can use Java and ABAP coding.

From scratch

You can use SAP NetWeaver Developer Studio or ABAP Workbench to define and code an enterprise service by hand. You can use this technique to create new services that extend the functionality of an existing enterprise application or custom services that may provide functionality unique to a customer.

From the outside in

You can use SAP NetWeaver XI Integration Builder and SAP NetWeaver XI Integration Repository, the precursors to the Enterprise Services Repository, to define interfaces for enterprise services in the context of a process that is being modeled and automated. Once the interfaces are defined, the services are implemented using one of the other methods.

Modeling environments

You can use SAP NetWeaver Visual Composer or the SAP CAF to create enterprise services through various forms of modeling. SAP NetWeaver Visual Composer focuses on the UI but will eventually play a role in modeling guided procedures. The SAP CAF allows modeling of the application logic, business objects, and external services inside a composite.

The common part of all of these methods is the fact that the interface description for the enterprise service is stored in the Enterprise Services Repository, which evolved from the SAP NetWeaver XI Integration Repository.