The Integration Services Engine: An Analogy

Before you learn about buffers, asynchronous components, and execution trees, you will start with an analogy — traffic management. Have you ever driven in a big city and wondered how the traffic system works? I find it remarkable to consider how the traffic lights are all coordinated in a city. In Manhattan, for example, a taxi drive can take you from midtown to downtown in minutes — in part because the lights are timed in a rolling fashion to maintain efficiency. The heavy fine assessed to anyone who "locks the box" (remains in the intersection after the light turns red) demonstrates how detrimental it is to interfere with the synchronization of such a complex traffic grid.

Contrast the efficiency of Manhattan with the gridlock and delay that result from a poorly designed traffic system. Everyone has been there before — sitting at a red light for minutes despite the absence of traffic on the intersecting streets, and then after the light changes, you find yourself at the next intersection in the same scenario! Even in a light-traffic environment, progress is impeded by poor coordination and inefficient design.

Bringing this back around to Integration Services, in some ways the engine is similar to the grid management of a big city because the Integration Services engine coordinates server resources and data flow for efficient information processing. Part of the process to make a package execution efficient requires your involvement. In turn, this requires knowing how the Integration Services engine works and some important particulars of components and properties that affect the data processing. That is the purpose of this chapter: to provide the groundwork of understanding Integration Services that will lead to better and more efficient design.