Chapter 13: Summary

The intention of this book is to promote the use of analytical techniques in pursuit of better accuracy of data in corporate databases. It has focused on the concept of accurate data and techniques that are particularly suited for finding inaccurate data. It outlines a complete process for formulating metadata rules and using them to efficiently evaluate data and to improve the completeness and accuracy of the metadata.

The book has several specific messages that together make a story. A summary of that story follows.

13.1 Data Quality Is a Major Issue for Corporations

Note 

Corporate databases are plagued with poor-quality data. The quality has become an ever-increasing problem.

Can anyone doubt that this is true? The evidence is everywhere. The problem of poor data quality is pervasive, affecting all organizations with significant information systems activity. Corporate executives no longer deny data quality problems. Data quality is getting on more radars all the time.

Note 

The cost to corporations for poor data quality is high. Corporations generally do not know the cost.

Quality improvement programs routinely disclose a sufficient number of issues to demonstrate the high cost of poor data quality. The number is always higher than anticipated. Day-to-day operational costs due to bad data are estimated as high as 20% of operating profit.

Note 

The cost of poor data quality is increasing. The quality of data is decreasing.

As corporations get more complex through product line expansions, mergers, and acquisitions, requirements to comply with increasingly complex government regulations, and many other factors, the demands on data go up. As corporations move data up the food chain to play increasing roles in corporate decision making, the cost of poor-quality data is magnified. The increasing use of the Internet as a data source and the acquisition of data from outside the corporation are causing a decrease in the quality of data.

Note 

Poor-quality data and poor-quality metadata frustrates implementation of new business models.

This statement is true for many corporations. The enormous evidence of failed projects, projects that last years, and projects that finish but fall far short of expectations all point to a lack of understanding of the data beforehand and a lack of appreciation of the quality of the data.

Note 

Poor data quality is a pervasive issue that impacts all corporations and organizations with significant information systems.

The reasons for poor data quality are the rapid growth in technology, along with the rapid implementation of information systems to use that technology. The change rate of corporate systems has been relentlessly high. As a result, few corporations have been able to avoid data quality problems.