Monitoring the Utilization of Resources

You must continuously monitor the utilization of various resources in a BI decision-support environment, especially the utilization of computers, networks, and personnel. If any one of these resources is neglected, it may potentially become a bottleneck for the BI applications.

Computer Utilization

Computer utilization includes the central processing unit (CPU), input/output (I/O) channels, random access memory (RAM), direct access storage devices (DASDs) or disk drives , and other related hardware. These devices should be dedicated for the BI decision-support environment and should not be shared by other applications. Since utilization of these devices increases over time, they should be monitored .

It is vitally important to use appropriate monitoring and alert utilities that will detect any resource problem and sound an alarm in case of an actual or pending system failure or resource shortfall. Selecting an appropriate monitoring utility is especially critical in a distributed environment.

Network Utilization

Network utilization may be a relatively minor issue for the day-to-day execution of BI applications, but it may be a big issue for the ETL process. One large telecommunications organization found out that even with access to the latest and greatest communications technologies, it had insufficient bandwidth to transmit its source data from the mainframe to the database server to populate its BI target databases in a timely fashion.

When monitoring your network utilization, also consider the following points:

• Bandwidth may be a serious problem for BI applications that frequently involve several levels of drill-down access into detailed data. Queries after queries may be executed by many people from many locations, each potentially returning huge amounts of data across the network.

• If bandwidth is an ongoing problem, one potential solution is to move toward a distributed BI implementation and away from a centralized strategy. However, if a lot of communication and integration among the distributed BI target databases is required, this solution could make the situation worse .

Personnel Utilization

A BI decision-support environment is a high-maintenance environment. This is especially true during the initial development and deployment of the first few BI applications. It is also true to the extent that manual processes are used during or after the ETL cycles to validate and analyze load statistics produced by the ETL runs. A BI decision-support environment requires dedicated support from a number of IT personnel, such as:

• Application developers

• Data administrators

• Database administrators

• Hardware and operating system specialists

• Middleware specialists

• Network administrators

Good technicians are often hard to find. Many technicians want to work on BI applications, especially the multidimensional ones, because these challenging applications involve new technologies (and thus they look good on a r sum ). However, because the operational systems still need to be supported ” since they are more mission-critical than BI applications ”a "tug of war" over available staff is often the result.

This is especially true for database administrators who already have a full plate supporting the operational systems. Some senior database administrators have to be released from those duties to become the chief designers and managers of the BI target databases. These database administrators are responsible for designing, building, monitoring, tuning, and maintaining the BI target databases and, to some extent, the database access calls from the BI application programs and access and analysis tools (e.g., writing passthrough queries). In addition, they have to be concerned with monitoring and managing the increasing data volumes and increasing database usage, which will require even more of their time. Redistributing the workload of database administrators is a culture shift and an issue in many organizations that perceive the database administrator's function as nothing more than running DBMS maintenance procedures on production databases.