10.4 Mapping with Other Applications

For projects that involve more than just data quality assessment, there remains a step for matching data rules from source systems to target systems. This is an important step that is often overlooked, only to cause problems after the data has been removed to the new environment. Mapping analysis varies by type of project, although the comparison process is the same.

Migrating to a New Application

When replacing an existing application with a new one (often a packaged application), you need to consider the business and data rules of the new system. It does not much matter if a rule required of the old system is not enforced by the new system. The important point is to ensure that the data in the old system will support the rules of the new system. If they do not, it may be difficult to move the data from the old to the new. If the data is movable, you will end up with some of the data being in violation of new system rules.

If possible, identify the data rules of the new system and use them in profiling the data from the old system. This means restating them relative to the column names and structure of the old system. In this way, you can see the violations that exist in the old data before it is moved.

The best way to profile is to profile both the rules of the old system and the rules of the new system. Ignoring old system rules leaves open the possibility of not finding inaccurate data in the old system. This is useful whether it causes data migration problems in the new system or not.

When you are done, you need to look at all of the rules. If a rule exists in the old system and not the new system, you need to consider whether the new system ought to have the rule. If the rule expresses a business policy, you may have uncovered a difference in behavior of the new system that the conversion team needs to consider. If a rule exists in the new system and not the old system, you need to consider whether it is necessary in the new system. If it is valid in both systems, you do not have to be concerned.

Incompatible rules become issues the conversion team needs to resolve. Often they can change the customizing parameters of the new system to make it behave more like the old system and more like the corporation wants it to behave. If data is incompatible, the team has to craft a transformation method to allow the old data to populate the new system.

Consolidating Systems

Projects that combine data from more than one system have to consider how rules play against each other in the various systems. These projects are generally spawned by mergers and acquisitions where the source systems were built and maintained without any knowledge of the other systems. The probability that all data structure and rules will line up is next to zero.

As in the previous section, you need to profile all of the rules of each system against their respective data in order to dig out inaccurate data. You will then know which rules are applicable to each system. You then need to identify the rules of the target system and profile those rules against the data in each of the source systems to determine which data will cause problems in the consolidation. This is identical to the process defined in the previous section.

In consolidations, you may be pushing all data to a new target system or picking one of the existing source systems and using that as the target. In any case, the analysis of data rules and their compatibility with the data can influence the decisions made on which system to use as the target or in how to modify the target system to be more accommodating of the data.

Extracting into Decision Support Systems

Data rules are generally not of particular concern in databases built from operational systems and used for reporting or decision support. The important point of these databases is that they contain accurate data. The only reason the data import functions would execute a data rule would be to check for inaccurate data. This means that there is really no matching of rules to be done. Carrying forward rule checks to the load process may be indicated by the amount of violations detected in data profiling. A better solution would be to add the checkers to the operational systems and get the data clean long before it is moved to the decision support data stores.