Chapter 7. The Physical Implementation

only for RuBoard - do not distribute or recompile

In this chapter, we will explore ways to create the physical data warehouse. There is a traditional left to right flow of data from the source systems to the warehouse. The operational process begins by the periodic extraction of data. The data has to be tested for quality. The quality of data is a major headache for most data warehouse developers, and we will be exploring this issue in some depth. Poor data quality will threaten the success of the whole project.

Also, we need to consider what happens to the warehouse when changes occur in the source systems. Changes can be very slight , having little or no impact, or they can be massive, as in the case of the introduction of a new enterprise resource planning (ERP) system that will, at a stroke, change the major source of data to the warehouse. Designing a data warehouse that is able to cope with such changes is similar to designing a 100-story building in an earthquake zone.

Then there is the requirement to operate the warehouse in a lights-out environment. Each day, new data is extracted, quality checked, loaded, and summarized. The way in which we design and build the warehouses is not usually conducive to a lights-out approach.

Let's begin this chapter with an overall look at the architecture.

only for RuBoard - do not distribute or recompile