Development Environments

The development of vital business applications does not happen ad hoc on someone's personal computer. Most organizations require some kind of formal or structured approach for developing these applications, testing them, and delivering them. Some organizations (and some projects) require more structure than others. Also, on BI applications, some application components need more structure than others.

For example, an informal and dynamic development approach for building the front-end access and analysis components is quite appropriate. Front-end applications are usually built with flexible tools that lend themselves quite well to rapid and iterative development cycles. It is quite common for the application track to go through several stages of prototyping, especially multiple iterations of operational prototyping, while performing analysis, design, coding, and testing activities almost all at the same time. However, developing the back-end ETL process in such an informal and dynamic way is not appropriate. ETL development requires a more formalized or structured approach because of its size and complexity. Even when ETL tools are used, the activities of the ETL development track are much more similar to a large operational systems development project than to the dynamic prototyping activities of the application development track.

To support these different types of activities, organizations usually set up different development environments for different purposes. While smaller organizations may have only two environments (development and production), large organizations usually have at least four different environments:

1. The prototyping environment, where the testing of the technology and the solidifying of the project requirements occur

2. The development environment, where the programs and scripts are written and tested by the developers

3. The QA environment, where the operations staff tests the final programs and scripts before allowing them to be moved into the production environment

4. The production environment, where the programs and scripts run after being rolled out

Depending on the overall setup of the environments, early prototyping activities (such as creating show-and-tell, mock-up, proof-of-concept, visual-design, and demo prototypes ) typically take place in a special-purpose prototyping environment, while development activities (including operational prototyping) are performed in the development environment. However, it is just as common to perform all prototyping and development activities in the same development environment. In either case, the entire BI application should be moved to the QA environment for final QA and acceptance testing before being implemented in the production environment.

If the different development environments are configured differently, moving your application from one environment to another could have major implications for your BI project.

The prototyping and development environments are usually configured similarly, as are the QA and production environments. The configuration differences are typically between the development and production environments. Key considerations appear below.

• If the application works well in the development environment, there is no guarantee that the application will run equally well in the production environment.

• It is conceivable that the migration costs from one environment to another could be substantial.

• New or different tools may be required for differently configured environments.

The Web Environment

Another environment that is becoming more and more popular for BI applications is the Web. Since most OLAP tools are Web-enabled, the data from the BI target databases can be and often is published company-wide through the intranet. A subset of that data can also be made available through a separate portal to business partners via the extranet or to customers via the Internet. Special security and authentication measures must be taken in the Web environment. Only qualified persons should be able to access authorized databases, and all access requests must pass through a firewall.

In addition to being a data delivery platform, the Web environment can also be a source for BI data. Capturing Web logs is a standard practice on Web sites, and the ability to extract, filter, summarize, and report the log data for click-stream analysis is a popular type of BI application (Web warehouse). Click-stream analysis can help identify customer interest (number of hits), gauge the effectiveness of Internet advertisements, and track the results of promotions. Table 12.1 shows a list of commonly available Web log data.

Table 12.1. Common Web Log Data

Click-Stream Data

Client IP address

User ID

Date and time of server response

Request (GET, POST)

Status from server to browser

Number of bytes sent

Prior site information (URL, path , documents, and so on)

Browser name

Cookie information