12.5 Closing Remarks

Value tests can be a powerful tool for discovery of inaccurate data. On the surface you would not expect to find much. However, many examples exist of analysts finding severe problems in the data that went unnoticed through normal rule checking.

This is the last step of data profiling. If you have executed all of the steps, you have done a lot of work. You should have a very detailed understanding of the data, an accurate data profiling repository, much additional adjunct information in that repository, a lot of data on inaccurate groups, and a bunch of issues for corrective actions to take to the implementers.

Although the task of data profiling is large, it is generally very rewarding. The returns for just the short-term value of identifying and correcting small problems generally more than offsets the cost and time of performing data profiling.

Data profiling can generally be done with a small team of data profiling analysts, bolstered by part-time team members from the business and data management communities. Compared to other approaches for data quality assessment, it is a low-cost and quick process if the analyst is armed with adequate tools and expertise at performing data profiling. It generally does not take long to gain experience. Most of the process is common sense and follows a natural path through developing and testing rules.