Section 11.1. The Enterprise Perspective

11.1. The Enterprise Perspective

As described in Chapter 1, the main motivation for creating an SOA is the desire to increase agility of the enterprise IT systems. In addition, SOAs offer benefits at several levels, ranging from a reduction of technology dependence to a simplification of the development process to an increase in flexibility and reusability of the business infrastructure.

The ultimate goal of the additional reusability and flexibility provided by an SOA is the Agile Enterprise, in which all processes and services are completely flexible and can be rapidly created, configured, and rearranged as required by business experts without the need for technical staff (see Figure 11-1). Among other things, this facilitates a superior time-to-market for new business initiatives. This vision of an Agile Enterprise reconciles the growing demands of a global and rapidly changing business environment with the limitations of current technological and organizational infrastructures. Consequently, the Agile Enterprise is not so much characterized by a fixed state of the enterprise but rather by an ongoing change process within the enterprise.

Figure 11-1. SOAs and their benefits.

Several sources contribute to the pressure on enterprises to instigate ongoing changes. Enterprises are continually forced to reinvent themselves by offering their customers new or better services to stay ahead of or keep pace with competitors. In order to remain as cost effective as possible, partners and suppliers must be regularly evaluated and, if need or opportunity arises, substituted by alternatives offering better quality, prices, or conditions. In addition, mergers and takeovers often require the integration of new systems, applications, and processes into the enterprise's IT and business infrastructure.

However, as unrelenting as this pressure toward constant change is, it is often obstructed by technological and organizational obstacles. For example, replacing a partner or supplier might be straightforward in theory, but it could require major efforts in reality. These efforts could involve a complete redevelopment of technical interfaces and a major redesign of business processes. Similarly, integrating a newly merged or acquired company's IT infrastructure and processes might call for a large-scale integration project. In addition, although offering new or better services might be desirable from the business perspective, it often proves to be technologically or organizationally infeasible. Typically, this can arise for two reasons:

Too time consuming. In some cases, developing the desired functionality might take too long to be of any use. For the introduction of a new product or service, there often is a window of opportunity (time-to-market) that must not be missed.

Too resource intensive. In most cases, the main risk to feasibility is the cost incurred in achieving the desired functionality. For example, replacing a supplier is only profitable if the long- and short-term costs for doing so do not exceed the savings obtained by the replacement.

For an enterprise to become an Agile Enterprise, it is therefore necessary to construct a technological and organizational infrastructure guaranteeing as much flexibility as possible. SOAs provide the means for achieving this flexibility by leveraging an appropriate architecture. In the following subsections, we describe the benefits of introducing an SOA at the level of the enterprise in detail.

11.1.1. AGILITY

One of the primary motivating factors for using SOAs is the potential increase of agility they offer. In order to fully understand this benefit, we must identify the different levels at which enterprise projects are threatened by complexity. Inevitably, complexity diminishes the enterprise's agility. The following elements of complexity can be distinguished:

Technology. Technical products and solutions used in enterprise projects are usually complex. Sometimes, they result from yearlong efforts to capture more and more functionality and lean towards the baroque in size and manageability. Sometimes, they are early versions of some new trend and bring with them all the teething problems of immature technology such as the bad performance of XML parsers or incompatible SOAP implementations in the early days of Web services. Consequently, technology used in enterprise projects can have complex deployment and installation procedures that work against smooth introductions. In particular, finding staff qualified for handling the technology can be a major obstacle.

Business processes. Often less obvious but at least as critical as technological complexity is complexity related to business processes. This complexity might be harder to grasp because processes tend to be much less visible and concrete than specific software. Ultimately, however, a complex business process results in a complex implementation. Sometimes, it is only during the project or even during implementation that the real complexity of the processes to be implemented becomes apparent.

Business functionality. Even if the basic elements of the business functionality are not complex, they can become complex when considered in their entirety. Sometimes, business functionality also contains conflicting elements due to different perspectives or application contexts, which adds to the overall complexity. Let's consider, for example, insurance products and tariffs. Each individual tariff is not particularly complex. However, the various permutations of these tariffs can result in an extremely complex insurance product portfolio.

Integration. Another source of complexity arises from the integration of applications. More often than not, enterprise projects will use and combine functionality from existing applications. Even if the individual applications are stable and well maintained, using them in a new context and integrating them with other applications is generally a complex task.

Maintenance. After an application is launched and running, it might seem that complexity is no longer an issue. However, maintenance of a complex system is far from trivial. Usually, components must be updated regularly, such as when additional functionality has been integrated or when new versions of software have been released. Such updates can cause complex issues similar to those encountered during the initial integration of the application.

Without an appropriate architecture, the risk of complexity preventing agility is very serious because changes might be infeasible or so expensive that the costs exceed the benefits. SOAs can help to significantly reduce complexity at all levels. This is of particular importance for enterprises, given their need to become more agile in order to react as quickly as possible to changing business environments and offer new services to customers, suppliers, and partners that make a difference with respect to competition. SOAs achieve their simplicity by the following means:

Decomposition. SOAs decompose large, complex systems into application frontends and services.

Appropriate granularity. The granularity of services is well suited to gain a high-level understanding of the entire system. Unlike other approaches, SOAs do not confuse practitioners by presenting too much detail.

Decoupling from technology. SOAs can be well understood without in-depth knowledge of technology.

Reuse. SOAs result in the high-level reuse of existing components. This streamlines the code base and reduces complex redundancies.

Documentation. Due to the service contract, every service is well documented, which adds to the comprehensibility of the SOA.

11.1.2. COST SAVINGS

We have already seen that agility and efficiency are the primary benefits of an SOA. This is mainly due to the fact that a commercial organization's agility and efficiency should help increase profit and aid in effecting savings. In this section, we discuss possible sources for cost savings. In particular, in price-sensitive, low-margin, mature markets in which the possibility of product innovations is rather limited, cost efficiency plays a key role. In these markets, often the price of the product makes the difference and hence provides the competitive advantage.

In general, we distinguish between direct (saving IT costs) and indirect contributions (saving business costs).

11.1.2.1 Cost Savings at the Business Level

SOAs can help to reduce costs in the enterprise's core business. These cost savings are largely related to the SOA's agility. There is a number of obvious examples:

Choosing the Cheapest Supplier. In low-margin markets, suppliers do not differ much. Nevertheless, even the slightest difference might make for competitive advantage in the long run. A major obstacle that can jeopardize the positive effect of constantly choosing the cheapest supplier is the inability of IT to support these changes within reasonable time and costs. An SOA solves a couple of issues that are related to a supplier change. Most importantly, an SOA provides an easy-to-use integration infrastructure that reduces the efforts for both business partners.

Streamlining Business Processes. SOAsparticularly process-enabled SOAs (see Chapter 6, "The Architectural Roadmap")are very well suited to support ever changing processes (see Chapter 7, "SOA and Business Process Management"). This enables the enterprise to make use of internal resources in the most efficient way. In traditional environments, however, costly adoptions of existing applications typically prohibit rapid change of business processes.

Improving Financial Reporting. Finally, SOAs contribute to more precise and up-to-date financial reporting. Due to its ability to make different parts of the architecture share live data, an SOA can enable financial reporting on the spot, which could mean that the most important business data becomes available on a daily basis. As a result, management decisions that were previously based on quarterly reports could be made more promptly, and longer-term trends could be taken into account in day-to-day business.

It should be noted that the possibilities of cost savings enabled by SOAs are not limited to the examples stated here.

11.1.2.2 IT Cost Savings

In principle, three major sources for IT cost savings can be distinguished:

Reduced project costs. In general, using an SOA in an enterprise project will considerably reduce project costs because it allows for more efficient implementation and deployment. However, for these cost savings to become effective, the SOA must be already in place. In the introduction phase, overhead usually outweighs reductions. In this phase, you must establish the fundamental development processes and put the technical infrastructurethat is, the service bus and the service repositoryin place. However, you will achieve lower costs at several levels, particularly with respect to resources needed for updating or modifying code, integrating modules, and testing. One of the major benefits of an SOA consists of its significant simplification of both design and configuration of business processes. As a consequence, new business processes can be developed more simply, and the expenditure for modifying or optimizing existing processes remains justifiable.

Reduced maintenance costs. You can significantly reduce the long-term cost of IT systems by introducing an SOA. Due to the simplification of the application landscape, the streamlining of the code base, and technology independence, future changes can be made more easily. Maintenance efforts can be targeted to business functionality. Side effects can be reduced, and comprehensibility can be increased due to a clear decomposition of the application landscape in reasonable components (i.e., application frontends, services) and an up-to-date documentation of the purpose and interface of these components (service contract). A good example is the encapsulation of complexity covered in Chapter 8 with a discussion on process integrity. Applying the recommendations of Chapter 8, you can make an application fit for efficient and targeted maintenance by separating critical code sections from less critical ones. The critical code sections (encapsulated in distinct services) can then be handled with special care.

Future proof solutions. Finally, correctly employing an SOA guarantees future proof solutions. This is mainly due to three reasons. First, an SOA abstracts from the underlying technology and is hence not tied to any idiosyncrasies or shortcomings. Thus, it can be used in conjunction with other technologies, which is particularly useful in cases where the underlying technology becomes obsolete. Second, an SOA guarantees protection of investment because it enables you to integrate functionality from extant systems in an enterprise instead of replacing them. The SOA endeavor creates a functional infrastructure that can be reused in future scenarios independently of its original purpose.

Most of these benefits are more or less directly related to the high degree of reusability provided by SOAs, which we will now discuss in more detail.

11.1.3. REUSE AND THE RESULTING BENEFITS

Reuse has long been a holy grail of the IT industry. The initial idea focused on reusing code through class libraries or code templates. However, within an enterprise, code reuse often has less far-reaching impact than the reuse of runtime components. Being able to reuse a component at runtime by linking it into a number of different sub-systems means not only that we are sharing a common code base, but more importantly that the different sub-systems are now sharing the same application data. This significantly reduces redundancies and inconsistencies in business data, which is a huge benefit.

It is important to point out that it is a key benefit of an SOA that it contributes to runtime reuse.

The use of a service is not confined to the project for which it is initially developedit can also be reused in other applications. This allows for a holistic IT strategy in which projects are not separated activities but instead contribute to each other and can be combined to achieve an overall synergy.

Moreover, usage is not tied to the technical infrastructure on which the service has been deployed. The service interface provides an abstract layer that can be used in the context of arbitrary technical infrastructures. Therefore, the service itself can be used in different technological contexts that guarantee protection of investment.

Another major benefit of reusability concerns design, modification, and optimization of the implemented business processes. When using an SOA, a business process can be composed directly from services or individual service operations. On one hand, it is possible to create a new servicethat is, an intermediary or process-centric service (see Chapter 5, "Services as Building Blocks")simply by arranging or "choreographing" existing building blocks. On the other hand, modifying or optimizing an existing service is also straightforward because it only requires a rearrangement of the operations used in the service.

Finally, reuse of existing code significantly reduces the risk of failure in enterprise projects. Using proven components, that is, code that has already been in operational use, eliminates time-consuming debugging and functional testing. It also enables the project to proceed iteratively in small steps. Enterprise projects can start with a comparatively small scope implementing the basic functionality. Based on this initial implementation, new functionality can be added step by step, reusing the functionality that has already been proven.

11.1.4. INDEPENDENCE FROM TECHNOLOGY

As previously stated, SOAsin particular the service contracts that describe servicesprovide an abstraction from the underlying technology.

This independence offers several benefits for an enterprise. Most importantly, business considerations can assume their natural role as the driving force for decisions.

Focus on technology can be a major problem in IT projects. This is not to say that technology is irrelevant, but often, discussions about "the right technology" threaten to tie up resources that could be better spent designing and optimizing business functionality. SOAs help to shift the attention from technological issues to questions of service functionality and service design. In other words, the question of which services to offer is separated from the question of how to implement the services.

Independence from technology also increases independence from software vendors. The opposite of a service-oriented approach can be caricatured as a project in which the functionality and features of a particular software product or technology determine the scope of the project. Applying an SOA enables choosing the best of breed products and combining them as required by the particular application, not as stipulated by software vendors.

However, contemporary application landscapes are characterized by a variety of different incompatible technologies (e.g., J2EE versus .NET). This heterogeneity makes it particulary hard to create and maintain cross-departmental business processes. Due to the tight coupling of business functionality to specific technology, it can be arbitrarily difficult to force different parts of the architecture to work together. SOAs mitigate these obstacles by decoupling the technologies of the provider and consumer of specific business functionality. SOAs do not eliminate the heterogeneity itself but rather its impact on agility and efficiency. It is the very nature of big application landscapes to tend toward heterogeneity. Fighting this heterogeneity is infeasible, and SOAs therefore embrace heterogeneity as a given fact and cope with it in the best way possible.

Independence from underlying technology permits a decoupling of technology lifecycles from the lifecycles of business services. Thus, the introduction of a new technology within an enterprise does not require a makeover of the business processes and services. Similarly, new business services can be introduced without the need to wait for new, innovative technology to become mature, stable, and affordable.

Although an SOA provides a lot of technological independence, you will never completely eliminate technology dependencies. In fact, every SOA requires a certain amount of technical infrastructure (see Chapter 9, "Infrastructure of a Service Bus"). Distributed logging and data transport are two prominent examples. However, if the SOA is properly deployed, this technical infrastructure is largely encapsulated.

11.1.5. ADEQUATE BUSINESS INFRASTRUCTURE

The traditional IT infrastructure of enterprises is characterized by a strong focus on technology and monolithic applications with fixed functionality. Technology, processes, business rules, and core business logic as well as data are tightly coupled. These applications are usually inflexible and can only serve as an appropriate business infrastructure in their original context. Thus, as the business context of the enterprise evolves over time, these applications often do not keep up with the enterprise's business strategy, which might have evolved in an entirely different direction.

SOAs, if applied correctly, can correct the mismatch between the demands of the ever-changing business environment and the constraints imposed by a rigid IT infrastructure. The business infrastructure of enterprises that implement an SOA will differ from traditional infrastructures. Instead of carefully planning and executing large-scale projects spanning several years, a step-by-step approach becomes feasible.

The scope of enterprise projects will thus become twofold: on one hand, projects will create new services and functionality for immediate use, while on the other hand, the services developed will later serve as building blocks for future projects. More importantly, SOAs decouple technology, processes, business rules, and data. This implies that if one of these becomes obsolete, the others can still remain in use.

Consequently, an SOA-enabled enterprise will create a sustainable business infrastructure over time that consists of flexible building blocks that can be continually reused.

11.1.6. MORE EFFICIENT DEVELOPMENT PROCESS

Using an SOA in an enterprise project also impacts the development process considerably. Chapter 13 will describe in detail how project management should be conceived when applying an SOA. At this point, we concentrate on how an SOA can help to reduce the complexity of the development process in general.

One obvious advantage of the contract-driven SOA approach is a natural relationship between business-oriented project artifacts such as use case descriptions and technical deliveries such as interfaces or even services. From the perspective of the development process, the main benefit of using an SOA is that it allows for a high degree of modularity, which in turn makes it possible to decouple the development process. Let us more closely examine what this means.

First, it means that the services to be developed in an enterprise project can be implemented independently of each other. By its very definition, a service is self-contained and can be used autonomously. In other words, it is not implicitly tied to the environment in which it has been implemented. If the service needs other services to perform correctly, all these dependencies are explicitly modeled and implemented as service calls with well-defined interfaces.

As a consequence, development teams in an SOA-based enterprise project can be decoupled, such that each team is responsible for implementing a specific list of services. Interaction between the teams is then reduced to a minimum, focusing mostly on the agreement of service interfaces. However, the definition of interfaces is one of the main development tasks in any development. An SOA provides a guideline for the interface definition and simplifies the process of achieving them. Project management can focus on managing small development teams and can employ efficient techniques not applicable for large teams with highly interconnected tasks. The overhead of project management is thereby significantly reduced.

A critical phase in most development projects is the integration of code developed by different teams. Problems that had not been apparent when developing the individual modules often surface in the integration phase. One major reason for the integration phase becoming a bottleneck is the inability to separately test and debug code. Once again, the SOA helps to reduce complexity in this respect because it enables the independent testing of individual service operations to a certain degree.

11.1.7. EVOLUTIONARY APPROACH

SOAs can be distinguished from other architectures because they explicitly address many of the enterprises' non-technical constraints. One of the most important constraints is based on the fact that large organizations tend to evolve in small steps. The organization's IT infrastructure should enable a similar evolution. There are several reasons for this:

Risk of failure. The most important reason is that IT systems are at the heart of modern enterprises. They are in many respects mission-critical. On one hand, they enable the enterprise to fulfil its obligations, while on the other, they are essential to conduct its business. A major failure could lead to serious financial and operational consequences for the corporation. This implies that any change to the enterprise IT systems must be performed with great care.

Capacity of change. The rollout of new business functionality is not only a technical problem but also a major challenge for the business organization. Every organization has a limited capacity for change due to the need for training and the day-to-day adoption of new business processes. In practice, a number of manual amendments to the electronic processes must be carried out, and the time required for these adjustments must be taken into consideration.

Involve stakeholders. In practice, you must convince key players and decision makers to support an endeavor such as the rollout of new business functionality. It becomes increasingly difficult to achieve this support when a large number of departments (or even business units) is involved.

Feasibility. The size of a new piece of functionality is a major criterion in regard to its feasibility. The bigger the piece, the more cross-dependencies there are to be considered. You also have to involve more people in order to cover the range of functionality.

SOAs are particularly well suited to enable a step-by-step approach due to two major characteristics.

First, SOAs enable an efficient decomposition of large segments of functionality into largely decoupled components of manageable sizei.e., application frontends and services. A project organization can directly follow this decomposition. Big projects that appear to be too risky, too difficult to rollout in the business organization, or technically infeasible can be broken down into subprojects that can individually be brought to success. Even if a single subproject fails, there is no overall threat to the enterprise. Second, unlike other architectures, SOAs are not tied to any specific technology. This enables an enterprise to be very flexible while introducing new functionality. More importantly, SOAs can make changes or amendments to technology and business functionality by treating them in an independent manner.

However, the real world is even more complex. Endeavors such as the introduction of a new architecture paradigm do not follow a long-term project plan. Instead, it is a fact that the requirements of day-to-day business have a higher priority. This implies that the introduction of an architecture will be more like an ongoing effort that accompanies the major business projects. As discussed in Chapter 6, SOAs reflect this requirement by their very nature. They enable an increase in both the scope of the SOA's technical foundation and its "content"the business functionalityin small steps.

11.1.8. SOA ENABLES FEEDBACK AT DIFFERENT LEVELS

It is undisputed that the enterprise architecture impacts all levels of the enterprise's organization with a variety of different stakeholders involved. It is therefore essential that the enterprise architecture address the key needs of these stakeholders (we will discuss the appropriate benefits from the perspective of the individual stakeholders in Section 11.2.).

However, it is not only the architecture that impacts their stakeholdersit also works in reverse, or at least it should. All stakeholders should be able to influence the architecture in order to make the full range of knowledge of the entire enterprise available to create an architecture in the best way possible. Many previous approaches have been comprehensible only to IT experts and therefore do not reflect this essential requirement. Consequently, only a few people have been able to contribute to the architecture, with the effect that valuable knowledge has not been used to make decisions crucial for the entire enterprise.

SOAs provide useful abstractions at different levels. They are not only comprehensible to IT experts but also to representatives of the functional departments. This characteristic of an SOA opens up vast knowledge from many people to contribute to its success and, incidentally, enables the management of the IT and business departments to gain appropriate influence on the main architectural decisions. It can also facilitate buy-in from the department members consulted.

11.1.9. MITIGATING RISK

The risk-mitigating effect of an SOA is a key benefit. Depending on the concrete business of an enterprise, this could even be the most important benefit of an SOA. According to the Standish Group [Sta2001], the key reasons for problems in IT projects are:

• Weak specifications of project objectives

• Bad planning and estimating

• Inadequate project management methodologies

• New technology and insufficient senior staff on the teams

At a minimum, some key characteristics of an enterprise SOA can help to partially address these issues in a number of ways.

Service documentation helps to clarify project objectives. The documentation of business-oriented, coarse-grained services is an invaluable tool for project managers to help close the gap between business requirements and technical specifications. Service specifications not only represent a contract at the technical level, but also between business and technology. This is a crucial part of the ongoing alignment of business and IT objectives. It is critical to ensure that the high-level documentation of services is suitable for the entire project team. Only then it ensures that the technology meets the business objectives.

Service orientation leverages involvement of business departments. The concept of the service contract enables the involvement of the business department. It enables domain specialists to drive the development process, and by doing so, you can significantly reduce the risk that the project outcome does not match business department expectations.

Service orientation helps planning and resource estimation. The abstraction level at which business services are designed (or at least should be) is exactly the right level of granularity at which they can be used as a powerful tool for planning and estimating project resource requirements and duration. Service-orientation helps to decompose large and complex systems into manageable sub-systems, which can be more effectively planned and estimated.

Service orientation can be leveraged at the project management level. Service orientation is not a replacement for sound project management strategies, but when incorporated into established project management methodologies, it can provide a powerful complementary tool. The loosely coupled nature of SOAs supports divide-and-conquer strategies. The decomposition of complex systems into manageable services is an essential tool for effective project management strategies. Service orientation enables you to roll out functionality in small steps, which dramatically reduces project risks because potential problems can be identified early in the development process. This is described in detail in the discussion of the thin-thread approach in Chapter 13.

SOAs facilitate the integration of existing and new functionalityindependent of the underlying technology. This enables projects to reuse proven core functionality of mission critical sub-systems, such as billing or invoicing. Relying on proven functionality significantly contributes to risk reduction.

A well-managed SOA can reduce the need for senior technical staff. As SOAs support simple yet flexible service components, which encapsulate the underlying technology of the service implementations, SOAs can help to reduce the need for senior technical staff. The decoupling of service components enables developers to stay in their respective technical domains, such as Java or COBOL development. In order to achieve this, services must be well designed and properly documented. In addition, the process for designing, documenting, implementing and rolling out services must be standardized and documented, too. SOAs do require senior technical staff for implementing this properly. Chapter 12 describes the role of an SOA architecture board, which should be responsible for these types of processes.

Finally, we must add that even failed projects can contribute to the business infrastructure of the SOA by amending business services or service operations that can be reused in future projects. As a consequence, the enterprise can benefit from at least a fraction of failed project efforts in the long term.