| Over the past few years, I (along with many of my coworkers) have had the invaluable opportunity of working closely with IT professionals who were trying to build or buy something like an ESB. As part of the research for this book, I met, spoke with, and interviewed many IT leaders across a variety of industries to gain an understanding of their integration challenges, and of how they are applying ESB integration concepts to uniquely solve them. The concepts in this book draw from that experience. This book represents a new direction for O'Reilly books. It is the first of a new "Enterprise Series" of books from O'Reilly. This book is targeted to integration architects, project managers, CIOs, CTOs, and basically anyone who has a need to integrate applications and who understands the bigger picture of the integration issues within their IT organization. At the same time, this book will also be valuable to everyday IT developers who need to understand ESB technology for their individual integration projects. As we will see in the first few chapters, the new "integration architects" are actually everyday IT developers, and the ESB concept brings the power and capability to solve their integration needs into their reach. This book provides an architectural overview of the ESB concept, and includes discussions of many practical uses and integration patterns that explain how an ESB can be applied toward today's application integration challenges. Ideally, readers of this book should possess some knowledge of technical concepts, but this is not a hard requirement. The introductory chapters provide a conceptual overview, including the requirements, drivers, and catalysts that have fostered the emergence of this new technology category. My intent in this book is to explain the ESB concept with just the right amount of detail that is suitable for individuals across many disciplines within an IT organization and with varying levels of technical and business acumen. This book takes a different tack from my previous O'Reilly books, and from most O'Reilly books in general, in that there are no code samples to speak of. You will, however, find a considerable number of diagrams that explain architectural concepts of the ESB. Although this book represents a slightly new direction for O'Reilly, it also possesses a trait that O'Reilly books have become known for: the "Missing Manual" approach to a particular technology. The ESB is being rapidly adopted across a variety of industries. Many middleware, integration, and infrastructure vendors have also gotten "on the Bus" and are either shipping an ESB already, or have plans to build one. A big part of the ESB is its commitment to standards-based integration. An ESB is designed and built from the ground up with standard components and standard interfaces. However, there is no specification for the ESB itself. This book will serve as a guide to define what an ESB is, as viewed through the eyes of those of us who pioneered the ESB concept and have been working with IT professionals on real-world ESB-based integration solutions. Even other vendors who want to join in with the growing ESB market trend, and are trying to figure out exactly what an ESB is, are welcome to use this book as a guide. Overview of the Chapters Chapter 1 gives an overview of the ESB, including the many characteristics that define it. It also explains the evidence of industry adoption and market attraction of this new concept in service-oriented integration. Chapter 2 provides a summary of the drivers, both business and technical, that have contributed to the need for a new approach to integration. It also examines some surprising statistics showing that, in general, the enterprise is far from connected, and it explores the shortfalls of Enterprise Application Integration (EAI) approaches to date. It explains the characteristics of an "accidental architecture" which helps you to identify your own architecture issues. Lastly, it explains how the ESB concept draws from best practices of previous integration approaches, and shows that you can refactor toward an ESB and away from an accidental architecture in incremental steps. Chapter 3 examines some key concepts of the ESB, including the many requirements, technology drivers, and forces in the IT climate that led to the creation of the ESB concept. This is explained in the context of the recent history of the evolution of the ESB, illustrating the point that an ESB is not merely an academic exercise; it is born out of necessity, based on real requirements and difficult integration problems that couldn't be solved by any preexisting integration technology. The discussion concludes with a high-level study of an ESB deployment, in which a manufacturer exposes inventory management and supply chain optimization functionality to its remote distributors as shared services through an ESB. This study, along with many others, will be revisited in more technical detail in later chapters. Chapter 4 explores the use of XML as a means for providing the mediation between diverse data structures and formats as data is passed between applications. It also examines how ESB-enabled data transformation and content-based routing services can support an XML data exchange architecture that insulates individual applications from changes in data structures, as integrated business processes improve over time. Chapter 5 explains Message Oriented Middleware and its role in an enterprise integration strategy. A MOM is a key part of the ESB architecture, as it provides the underpinnings to the network of virtual channels that an ESB uses to route messages throughout an extended enterprise and beyond. This chapter explains how the definition of a MOM has evolved to support the highly diverse and distributed topologies and open standards that an ESB integration requires. (In other words, it's not your Mom's MOM.) The chapter goes on to explain the key differences between the low-level coding of using a MOM versus the higher-level configuration aspects of using an ESB. Chapter 6 explains the details of what makes an ESB an ESB. The ESB provides an architecture that brings together all the concepts described in previous chapters into an infrastructure that can span an extended enterprise and beyond. This chapter describes the ESB service container a key enabler of the highly distributed, event-driven SOA and its concept of endpoint abstraction. Chapter 7 covers the service invocation model i.e., the underlying framework that provides the SOA in an ESB. We will discuss multiple forms of process routing, including the concept of itinerary-based routing and the role it plays in enabling a highly distributed SOA across independently deployed services. We will also discuss some fundamental ESB services such as content-based routing and XSLT transformation, and explore how service types can be defined and then reused for different purposes through elements of configuration. Finally, we will see how additional capabilities can be layered on top of an ESB through an advanced service, such as an XML persistence service and an orchestration service. Chapter 8 examines how the ESB can extend its MOM core to create the flexibility in protocols necessary to connect to applications in an adaptable and non-intrusive way. This includes an explanation of how XML and SOAP messaging can be integral parts of an ESB strategy, yet also be flexible enough to carry other data formats such as EDI X12 messages. This chapter also shows the details of a partner integration using an ESB, and examines the use of third-party adapters and custom integration services for integrating with SAP. Chapter 9 explores the most common form of integration being done today: bulk data transfer and batch updating using Extract, Transform, and Load (ETL) techniques. We will discuss the business impact of the latency and reliability issues associated with this integration method, and examine the prescriptive steps for migrating toward real-time integration using an ESB in the context of a case study using pattern sketches. Chapter 10 examines the various Java components in an ESB. The ESB is a platform-neutral concept, and could be implemented without any Java involved. However, a good ESB should take advantage of Java components due to the large adoption of Java-based technology across IT departments. A fairly extensive list of specifications that are utilized within an ESB comes out of the Java Community Process (JCP) on a regular basis; this could be the subject of a whole other book. Some Java specifications are particularly worth calling attention to because they have a special impact on the operation of an ESB. Chapter 10 will discuss the following specifications and their impact on making an ESB a more effective integration environment: Java Business Integration (JBI): A specification describing the way integration components, such as ESB services, can be plugged together in a vendor-neutral and portable fashion. J2EE Connector Architecture (JCA): A specification that defines how to use a standard set of interface contracts for creating adapters to connect into, and interact with, enterprise applications. Java Management eXtensions (JMX): A specification for remote management defining the means by which an application can interface with a management infrastructure and management consoles.
Chapter 11 examines some common uses of an ESB in integration scenarios. The ESB concept already has a number of common uses that solve some very common and challenging integration problems. These include the Validate, Enrich, Transform, Operate (VETO) pattern and its derivatives, a "two-step XRef" pattern, and several variations of backend integration patterns that are driven by enterprise portal requirements, such as a federated query/response pattern and forward caching using an ESB caching service. Chapter 12 provides an overview of the evolving web services specifications (including the WS-* family of specifications), and explains how web services and ESBs will move forward together. Appendix A contains the list of vendors who are currently offering or are promising to offer an ESB product in the future. I will try my best to keep this up to date with subsequent printings of the book. Chapter 13 contains references to other books, specifications, and web links that were used in supporting research for this book. | 