Data Integration


Data integration is a process by which data from disparate systems can be merged and used uniformly by transforming it in a universally acceptable and recognizable format.

Modern enterprises have a wide variety of applications interacting with each other using different message formats (see Figure 36.1). The data formats can be of different kinds, such as binary, XML, plain-text, and delimited text. WebLogic Integration helps applications to exchange data by supporting data translation and transformation. XML has become the standard for exchanging information between applications.

Figure 36.1. Data integration in a modern enterprise.

graphics/36fig01.gif

Some of the important terms in data integration are explained in the following sections.

Translation

Translation is conversion of data from one format (or language) to another format (or language). Binary data originating from a legacy system is typically converted to XML. For the translation process, each field of the binary data is converted to XML according to metadata defined for that field. The metadata typically includes the name of the field, the data type, the size, and a flag to indicate whether the data is optional or required. Figure 36.2 shows an example of data translation of some sample legacy data into XML.

Figure 36.2. Data translation.

graphics/36fig02.gif

Transformation

Transformation is a process that leads to the alteration of a given data object. For example, one XML document is transformed to another XML document using Extensible Stylesheet Language Transformation (XSLT). This process of transformation using XSLT requires identifying, copying, and manipulating data elements from one XML format to another. When XSLT is used, the Extensible Stylesheet Language (XSL) is used to describe a series of transformations that can be performed on a given XML document. Figure 36.3 shows the translated data (as shown in Figure 36.2) being transformed into another XML document.

Figure 36.3. Data transformation using XSLT.

graphics/36fig03.gif

Data Integration Support in WebLogic Integration

WebLogic Integration enables data integration by supporting the translation of binary legacy system data to XML and vice versa. After the data is converted into XML, it can be easily transformed into a specific XML grammar and can then be used to trigger WebLogic Integration workflows. Within the workflows, the data can be transformed to different formats and passed onto different subsystems for further processing. WebLogic Integration provides various components to support the translation from non-XML data into XML and vice versa. These components or functional areas are

  • Design-time component

  • Runtime component

  • Plug-in to BPM

The Design-Time Component

The design-time component of WLI, which facilitates data integration, is an application called Format Builder. Format Builder helps in creating descriptions and input formats that match inbound data (such as binary data records) that needs to be converted. A description is typically composed of the layout and hierarchy of various data elements in binary data. When you use Format Builder, the description of the data is saved in an XML grammar called Message Format Language (MFL) . This grammar describes the metadata of the binary data to be converted and is used by the runtime component and plug-in to BPM to translate binary data to an XML document or vice versa. Format Builder also creates a DTD or XML Schema for the XML document created from a translation for validation purposes. Format Builder will be discussed in detail later in this chapter.

The Runtime Component

The runtime component of WLI provides functionality that can be used to translate data from a binary format to XML and vice versa. This runtime component can be used in many ways; for example, it can be used as an Enterprise JavaBean (EJB) or as a business operation from a workflow in Studio. Figure 36.4 illustrates the runtime process flow.

Figure 36.4. Runtime process flow.

graphics/36fig04.gif

Plug-in to BPM

The Business Process Management (BPM) component of WLI provides functionality to design, execute, and monitor complex business processes using a process engine. As described in Chapter 34, Business and Workflow Process Management Using WLI, BPM supports a plug-in framework to extend its basic functionality by modifying the behavior of components, including Start nodes, Event nodes, and Task actions. WLI includes a data integration plug-in that provides translation capabilities (XML to binary and vice versa) through a Task action. This functionality helps support an exchange of information between applications by making it possible to translate binary data from legacy systems into XML. This plug-in also provides the capability for event data processing in binary format and in-memory caching of MFL documents and translation object pooling for performance.

Data Integration Tools

XML has become the standard for universal formatting of information and therefore is used as a standard for data integration in WLI. XML is used for both messaging and data storage. Various tools provided in WLI help in processing XML documents: for creating XML documents within a workflow, storing and retrieving XML documents, searching XML documents, and transforming and translating data. Some of these tools will be covered in the following sections.

XML Instance Editor

The XML Instance Editor is a tool that is used to construct well- formed XML documents within a workflow. Typically, an XML document is populated within a workflow with content of workflow variables . Also, element/attribute values may sometimes be extracted from an XML document and assigned to a workflow variable. The Instance Editor facilitates developing this kind of functionality. The XML Instance Editor is accessible via various action dialogs in Studio, such as

  • Set Workflow Variable (see Figure 36.5)

    Figure 36.5. The XML Instance Editor.

    graphics/36fig05.gif

  • Post XML Event

  • Send XML to Client

  • Invoke Exception Handler

The Instance Editor also provides functionality to import or export XML documents from or to the file system or an XML repository.

XML Repository

The XML repository stores XML documents and related information for WLI, thereby promoting reuse of XML- related components across workflows. Contents of the XML repository may be exported or imported. The following types of entities can be stored in the repository:

  • Document type definition (DTD)

  • MFL files

  • XML Schema (XSD)

  • Text files

  • XML documents

  • XSLT template documents

Each entity stored in the XML repository, shown in Figure 36.6, must have a unique name. All the standard CRUD (Create, Read, Update, and Delete) functions are supported.

Figure 36.6. The XML Repository.

graphics/36fig06.gif

XPath Wizard

The XPath Wizard assists in generating XPath expressions to locate textual data, elements, and attributes in an XML document. The XPath Wizard, shown in Figure 36.7, is accessed from the Expression Builder tool.

Figure 36.7. The XPath Wizard.

graphics/36fig07.gif

Format Builder

The Format Builder is a tool used to assist in defining formats for inbound and outbound data. The formats can be created, viewed , and updated using this tool. The Format Builder by itself does not require a running instance of the WLI Server. However, the WLI Server needs to be running if the formats created are to be stored in the WLI repository. Before creating a format, you need to determine the following information:

  • Definition of binary field data (delimiters, length, and so on)

  • Names of fields in XML

  • Organization of fields

  • Occurrences of repeating groups

Converting Binary Data to XML Using Format Builder

Consider an example in which you use Format Builder to convert binary data to XML. The binary data file to be used in this example is located at examples1\person.data .

 
 ManishDevganBoulderColoradoUSA\r\n 

The MFL file that is generated by Format Builder is located at examples1\person.mfl .

To define a format for the conversion, start the Format Builder tool by following these steps:

  1. Define a root element called Person .

  2. Add a field to the root element by choosing Insert, Field, As Child (as shown in Figure 36.8).

    Figure 36.8. Adding a String type field using Format Builder.

    graphics/36fig08.gif

  3. Define the XML Field Name ( FirstName ), data type ( String ), and a delimiter ( ) and then click Apply.

  4. Repeat step 3 and define other fields LastName , County , State , and Country .

  5. Define a field called CarriageReturnLineFeed of type Literal and define the literal value as \r\n (as shown in Figure 36.9).

    Figure 36.9. Adding a Literal type field using Format Builder.

    graphics/36fig09.gif

To run and test the example, choose Tools, Test. When the Test window is up, open the binary data file by choosing File, Open Binary and then run the sample by choosing Translate, Binary to XML (see Figure 36.10).

Figure 36.10. The Format Tester.

graphics/36fig10.gif

Contivo Analyst

BEA WebLogic Integration bundles Contivo Analyst, a design-time graphical mapping tool that can be used to generate XSL stylesheets. The WebLogic Integration runtime can execute these stylesheets to transform a source message to a target message. For more information on Contivo products refer to http://www.contivo.com.

Data Integration and BPM

When you create an MFL using Format Builder, the transformation between XML to and from binary data can be performed from within a BPM workflow. Two actions are provided in the BPM Action List in Studio:

  • XML to Binary

  • Binary to XML

Advanced Data Integration Using the Data Integration Plug-in

As you learned in Chapter 34, a workflow can be started or triggered by JMS messages on the WLI BPM event queue. Messages placed on the JMS queue can be in any format. A binary message on the JMS queue can be transformed by the Data Integration plug-in into XML, which can trigger a workflow.

– To learn how a workflow can be triggered, see Business and Workflow Process Management Using WLI, p. 1251 .


Refer to Figure 36.11. Here, a binary message triggers a workflow.

Figure 36.11. Data Integration plug-in.

graphics/36fig11.gif



BEA WebLogic Platform 7
BEA WebLogic Platform 7
ISBN: 0789727129
EAN: 2147483647
Year: 2003
Pages: 360

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net