Chapter 12: Semantic-Based Enterprise Application Integration and Systems Integration

Systems integration (SI), or enterprise application integration (EAI), consumes the lion's share of SI maintenance and development budgets. It is estimated that between 20% and 67% of the annual SI budget is devoted to building and maintaining integration, with the consensus seeming to be in the 30% to 40% range.^[78][79][80] This is value-added work in only the most limited sense of the term. It is worth doing, and it provides some value; otherwise people wouldn't do it. However, the amount of effort it consumes is far out of proportion to what it could and should cost.

More important for the topic at hand, it has been estimated that 95% of this cost is attributable to semantics.^[81] That number seems a bit high, even to a semantic fanatic like myself, but my experience would suggest that at least half the cost of integrating systems comes down to resolving semantic issues. In this chapter we cover the mechanics of systems integration; we will look at integration within your enterprise and integration with your trading partners. Then we'll look at two approaches that help with the semantic issues in integration: enterprise message modeling and semantic brokers.

What Is Integration?

Before we examine how integration works, or how it should work, I'll spend a brief moment on the industry and the acronyms, so that the remainder will make more sense. The exact size and shape of the "systems integration" market is hard to pin down, but by almost any source it is a multi–hundred-billion-dollar industry.^{[82][83][84][85]} The "industry" consists chiefly of the following:

• Professional services firms (Accenture, EDS, CSC, BearingPoint, etc.) who integrate existing and new software systems on a consulting basis

• Software firms (Web Methods, Tibco, BEA, IBM, etc.) who sell software that makes it easier to integrate existing systems

• The share of internal Information Systems (IS) staff devoted to building and maintaining interfaces between existing and new systems

This "industry" was originally called SI, but more recently it has specialized into EAI for internal integration and business-to-business (B2B) and supply chain management (SCM) for interorganizational integration. A study of 400 enterprises by PwC and the Meta Groups found that the average organization had 68 application systems.^[86] We have encountered firms that have thousands. There is a great deal of redundant information in those systems, as well as events recognized in one system that must be reflected in another. A study of 600 enterprises by the Hurwitz Group found that only 10% of the enterprises had fully integrated their most mission-critical processes.^[87]

On top of this vast and underpenetrated need, the industry has added another dimension. The hot topic in the industry currently is the "need for speed." This shows up in a number of acronyms, including zero-latency enterprise (ZLE), straight-through processing (STP), and real-time enterprise (RTE). Each of these is based on the theme that it is not enough to integrate your systems; they must be integrated in a way that removes time delay.

So hundreds of billions of dollars are being spent annually on a need that seems to be insatiable, and the majority of it seems to be related to semantics. Let's take a closer look.

Where the Need for Integration Comes From

To understand this better, we need to delve into the nature and mechanics of the dark art of systems integration. To put this in context, let's imagine that Bill from the Swetsville Zoo did what so many businesses do: He bought an application software package to handle his inventory. Let's call this the inventory management system.

Traffic picks up at the zoo. Bill opens a gift shop and decides he needs a point-of-sale system (a cash register with a computer appendage). He does a detailed requirements project and a software selection project and buys a state-of-the-art point-of-sale system. There's only one problem: It is not integrated with his inventory management system. What does this mean?

Integration

Here's a good operational definition of integration for business systems:

Integration

Two systems are integrated if an event in one system (system A) that might potentially affect decisions being made in another system (system B) is always reflected in system B in "business real time."

Whether or not you fill in the question mark in Figure 12.1, and how much you are willing to spend to do so, depends almost entirely on the value of the decisions to be made or the efficiency in automating the action to be taken. How rapidly and reliably it needs to occur depends on what "real time" is for these functions.

Figure 12.1: Where application integration fits in.

"Real time" is relative for all business functions. For stock trades it may be milliseconds, whereas settlement is still measured in days. For many business functions it is measured in seconds or minutes. Industries that rely on physical movement, such as trash collection, can measure time in days and still be in "real time." The issue is whether your system's end-to-end latency matches up with your business requirements.

At the Swetsville Zoo, Bill will only notice this when he adds a new sculpture to the zoo and someone wants to buy it, or when someone buys one and it must be marked as sold. This example is intentionally simple, but many businesses waste millions of dollars from information leakage of this type.

The other issue that rapidly comes up is the number of applications, and therefore the number of interconnections, that must be supported. The number of possible interfaces goes up geometrically as the number of applications increases. Systems integrators are fond of pointing out that the number of possible interconnections between n systems is [n(n − 1)]/2. Whereas there are only 3 possible interconnections between 3 systems, there are 45 possible interconnections between 10 systems and 2278 possible interconnections between the 68 systems that the "average" company has. These are only the theoretically possible number of interconnections. Nobody comes anywhere close to that. First, just because there is a theoretically possible connection between two systems doesn't mean they have any data in common. Second, it is usually not economical to build all of the possible interfaces.

^[78]Mario Apicella, "Making Application Ends Meet," Infoworld, Feb 22, 2002.

^[79]Dr. Wolfgang Martin, "The Second Wave," Meta Group. Available at http://www.ids-scheer.com/sixcms/detail.php/15360?_country=3582.

^[80]See http://www.csc.com/solutions/enterpriseapplicationintegration/offerings/717.shtml for further information.

^[81]Jim O'Leary, "Tying it Together," Feb 9, 2000. Available at http://www.omg.org/news/meetings/workshops/presentations/eai_presentations2/Tying%20it%20Together.pdf.

^[82]Gartner Group IT Service Market Statistics, Oct 31, 2000.

^[83]Kim Girard, "Middle Management Enterprise Integration Software's New Home on the Net," Business 2.0, June 13, 2000, pp 92–93.

^[84]"The $700 Billion Help Desk," Industry Standard, July 30, 2001, p 27.

^[85]"This Decade's Trends," Application Development Trends, May 1, 2001, p 64.

^[86]Reported in VAR Business, Nov 11, 2002, p 102.

^[87]See http://www.massecomm.org/news/news.asp?NiID=231 for further information.