Step 3: Make Sense of the Processes

Once the enterprise data is understood, and baseline information such as the enterprise metadata model has been created, the decision must be made as to how to approach the enterprise business model. This decision will depend on how the particular application integration problem domain is addressed. This view of the enterprise at the process or service-oriented level requires understanding and documentation of all business processes and how they relate to each other, as well as to the enterprise metadata model.

As with the database analysis procedures outlined previously, it is desirable to use traditional process-modeling techniques, such as object modeling (e.g., the Unified Modeling Language, or UML) to create business processes. What's more, instead of creating the business processes from a set of application requirements, it is preferable to document existing business processes and methods to better understand what they do and, therefore, how to integrate them at the service-oriented level through a composite application.

Process Integration

We already stated that identifying processes that exist within an enterprise is a difficult task, requiring the analysis of all the applications in the enterprise that exist in the application integration problem domain. The task is made even more difficult due to the possibility that many entities embrace numerous types of applications over a number of generations of technology.

Once the applications in the problem domain have been analyzed, the enabling technology employed by each application within the problem must be identified. With applications in the enterprise dating from as long ago as 30 years, this step will uncover everything from traditional centralized computing to the best distributed computing solutions. Once this identification has been made, the next step is to determine ownership of the processes and, subsequently, develop insight into those processes for example, why information is being processed in one manner versus another, or why multiple arrays are used to carry out a particular operation. In most cases, ownership will rest with the application manager, who determines when the application is to be changed and for what reason(s).

Finally, the documentation for the application will need to be found. In a small number of instances, applications have been well documented and well updated as they have matured. The vast majority of the time, applications are poorly documented, or the documentation has not been updated to keep up with the maturation of the application. Proper documentation has a profound impact on SOAI. Therefore, even if documentation is nonexistent, it is imperative that baseline documentation be created as the application integration solution is put into place. This is an area of significant expense as applications are opened in order to be understood and their features documented.

Process identification is important to the SOAI project life cycle. Without understanding the processes, moving forward is like a guessing game or playing blind man's bluff. There can never be real certainty that the enterprise is being accurately integrated. While data is relatively easy to identify and define, processes are much more complex and difficult.

Process Cataloging

In the same way that IOAI requires a data-level catalog, SOAI requires a "process catalog" a list of all business processes that exist within an enterprise or, at least, within the problem domain. For example, if an application integration architect needs to understand all the processes that exist within an inventory application, he or she will read either the documentation or the code to determine which processes are present. Then, the architect will enter the business processes into the catalog and determine the purpose of the process, who owns it, what exactly it does, and the technology it employs (e.g., Java or C++). In addition, the architect will need to know which database a particular business process interacts with as well as its links to the metadata model. The catalog will require other information as well, such as pseudocode, data flow diagrams, and state diagrams, which define the logic flow for the particular process. In the world of traditional, structured programming, this would mean a flowchart, or pseudocode. However, in the world of object-oriented development, this refers to the object model. Thus, it is important not only to identify each process but to understand how the logic flows within that process.

Other information may be maintained in the catalog as well, information that may include variables used within the processes, object schemas, security requirements, and/or performance characteristics. Each process catalog must maintain its own set of properties, custom-built for each specific application integration problem domain.

The Common Business Model

If the enterprise metadata model is the end state for IOAI analysis, then the common business model is the end state for SOAI. Simply put, the common business model is the aggregation and high-level model of all objects, methods, properties, procedural logic, batch processing, and everything else in the enterprise that processes information.

The common business model is assigned this name (and not the "object model") for one reason: In order for it to document and account for such a wide array of techniques and technologies, the model must remain, at least at first, an independent paradigm. Many of the applications written in the last ten years use the object-oriented paradigm for both programming and design. However, most traditional applications (those older than ten years) use a structured application-programming paradigm, as well as structured design. The common business model must remain independent of both.

Having an independent paradigm results in a simplification of the business processes in the enterprise or the application integration problem domain. The common business model does not map out a complex, detail-oriented model, but rather a high-level understanding of the processes in the enterprise along with their functions and features all with an eye toward those that may be combined or reused.

Business modeling is important for another reason, as well. The ultimate goal of application integration is to move beyond the simple integration of applications to the much more complex integration of business process flow analysis and design. Thus, the business model provides the basis of understanding that allows for the creation of a proper business process flow that is necessary within the enterprise. To accomplish this, the current state and the desired state along with a plan to get from one to the next must be documented.

For the sake of clarity, application integration architects should use the Unified Modeling Language to define the common business model. Using UML affords application integration architects a twofold benefit. First, an abundance of tools is available to create and maintain UML. Second, a deep pool of information exists regarding leveraging UML for a variety of purposes.

The real challenge to creating a common business model is scope. The temptation and thus, the trap to rearchitect and redesign each source and target application is very real. However, creating a business model is not an application development problem. It is an application integration problem. The application integration architect must recognize that the purpose of the common business model is to document existing processes, not to change the source and target applications. While it is certainly possible to rewrite everything from scratch in order to integrate the enterprise, such an endeavor would largely outweigh the value of application integration.

Leveraging Patterns for Service-Oriented Application Integration

Patterns are always interesting. More than interesting, they are also useful in the context of SOAI. They enable the application integration architect to identify common business processes among the many business processes that already exist within the enterprise. Or they may be used to identify new processes in need of integration. Using patterns in this way is simply borrowing them from the world of application architecture and applying them to the world of enterprise architecture a perfect fit.

Patterns arise over and over during considerations of application integration architecture. Patterns formalize and streamline the idea of application integration to build once at the application level and then reuse throughout the solution domain. Patterns describe recurring design problems that appear in certain design contexts. They describe generic schemes for the solution of the problems, solution schemes created by defining their components, responsibilities, and relationships.

A three-part schema is part of every pattern: context, problem, and solution. Context describes the circumstances of the situation where the problem arises or which circumstances have to be true for the pattern to be valid. For example, an object uses data from an external system. The problem is what is addressed by the pattern. The challenge for the architect or designer is to capture the essence of the problem or what design issues define the problem. For example, changing the state of the object means that the object and the external data are inconsistent. As a result, there is a need to synchronize the external data and the object. The resulting solution is the well-proven and consistent resolution of the problem.

Types of Patterns

All patterns are not created equal. There are types, or categories, of patterns to be used in specific situations in the application integration problem domain. For example, some patterns assist application integration architects in structuring a system into subsystems, while others allow the architect to refine the subsystems and applications and the ways in which they interact. Some patterns assist application integration architects in determining design aspects using pseudocode or a true programming language such as C++. Some address domain-independent subsystems or components, or domain-dependent issues such as business logic. Clearly, the range of patterns makes it difficult to define neat categories for them. However, there are three useful, and generally accepted, categories: architectural patterns, design patterns, and idioms.

Architectural patterns are complex patterns that tend to link to particular domains. They provide application integration architects with the basic system structure subsystems, behavior, and relationships. It would not be unfair to consider these architectural patterns as templates for application integration architecture, defining macro system structure properties. Design patterns are not as complex as architectural patterns. They tend to be problem domain independent. These patterns provide application architects with a means of refining the subsystems and components, and the relationships between them. The goal in using these patterns is to find a common structure that solves a particular problem. An idiom is a low-level pattern coupled with a specific programming language. It describes how to code the object at a low level.

Pattern descriptions, or micro-methods, provide specifications for deploying the pattern. They give application architects problem-independent analysis and design methods that are found in traditional, object-oriented analysis and design methodologies such as UML.

Application-to-Application Integration

Patterns, as recurring phenomena, provide application integration architects with a methodology for defining and solving recurring problems. They allow application architects to create reusable solutions that can then be applied to multiple applications. The ensuing information is too often kept "in the architect's head," where it is either forgotten or simply unavailable when the problem arises again. Unfortunately, application integration architects, even the good ones, tend to be poor documentation generators.

Extending the previous concept, it becomes clear that patterns do a good job of recording design experience across a development group. They level the playing field, providing every team member with access to this valuable information. Application integration developers and application architects may create a database of architecture patterns with search features and links to code and models. Patterns also provide a standard vocabulary and a base set of concepts for application integration development and application architecture. For this reason, terms used to represent concepts should be carefully chosen. Thoughtful terminology results in greater efficiency because terms and concepts no longer have to be explained over and over. When endowed with a common terminology, patterns provide a natural mechanism for documenting application architectures. The vocabulary should be consistent across many projects. If this consistency is adhered to and the patterns are accessible and understandable to all members of the application development teams then patterns can be employed with striking success.

Patterns allow application integration developers and application architects to build complex architectures around predefined and proven solutions because they provide predefined components, relationships, and responsibilities. While patterns don't provide all the design detail necessary to solve a problem at hand, they do define a starting point. Moreover, they provide application integration architects with the skeleton of functionality.

Using patterns, application integration architects can manage software complexity because all patterns provide predefined methods to solve a problem. For example, if the problem is integrating an enterprise, there is no need to reinvent the wheel. The task is simply to find the pattern that describes the problem and then to implement the solution. (This assumes that only sound patterns with working solutions are captured.)

The Value of Patterns

Patterns add value to the application integration architecture and the software development life cycle. They have the potential to increase the quality of most system development and to provide effective integration without increasing cost. However, they are not necessarily the panacea for all that ails the system. Patterns should not be assumed to be a mechanism for replacing a sound methodology, or a sound process. Patterns are able to become part of each, and in many instances, some pattern-oriented enterprise architecture is already taking place. Patterns simply formalize the process, filling in the gaps and providing a consistent approach.

There are those who describe patterns as the "second coming" of enterprise architecture. As comforting as that assertion might be, there is no evidence to validate it. Patterns are, simply, another available tool to be used. What is more, patterns are valuable only if they are practiced consistently for all applications in the enterprise. Capturing patterns for a single application is hardly worth the effort. Application integration architects need to establish the infrastructure for patterns, creating standard mechanisms to capture and maintain the patterns over the long haul, refining them when necessary.

Finally, patterns only provide an opportunity for reuse within the application integration problem domain. They are not a guarantee of success. Object-oriented analysis and design methodologies formalize the way reusable elements (objects) are implemented. Patterns define architectural and design solutions that can be reused. There is a difference. Patterns need not be linked to object-oriented analysis, design, or implementation languages. Nor do they need rigorous notation, a rigid process, or expensive tools. Patterns can be captured easily on 3x5 index cards or a word processor. The ideal way to capture patterns in the context of application integration is within the process catalog described previously. Doing so allows both process information and pattern information to be gathered at the same time.

Again, the concept of patterns is a very simple one. However, patterns, as described by the pattern community, are often too difficult to understand and use. The depth and range of academic double-speak regarding patterns has "spooked" many application architects. Hopefully, as with other technologies based on complex theory, patterns can be freed of the shackles of academia and be made available to the masses so they can become a primary tool for integration at the service level.