Business Intelligence Basic Concepts

In the broadest sense, business intelligence has come to mean using information to make better business decisions. Many definitions list a term from the 1970s, decision support system (DSS), as a synonym. Weve always liked this term because it is practical and descriptive of what were trying to build. The term business intelligence began to rise in popularity at the end of the 1990s. It primarily referred to the structured data access layer between the users and the data warehousewhat we call the BI applications. However, at that time, the industry analysts were describing business intelligence as separate from the data warehouse, like you might build one without the other.

Although it is conceptually possible to build business intelligence applications without the benefit of a data warehouse, this rarely happens. A well-built data warehouse adds such incredible value to the data through the business dimensional model and the ETL process that it makes no sense to replicate this effort in order to build a stand-alone BI application. Most business intelligence applications are an integral part of the user - facing end of the data warehouse. In fact, we would never think of building a data warehouse without business intelligence applicationsthats why one of the three tracks in the middle of the Lifecycle diagram in Figure 8.1 is dedicated to BI applications. Thats also why we use the words data warehousing and business intelligence as a single term.

Figure 8.1: The BI application specification and development steps in the Lifecycle

Theres no commonly accepted definition of what constitutes a BI application, so were offering our own. BI applications are the delivery vehicle of business intelligencethe reports and analyses that provide usable information to the business users. BI applications include a broad spectrum of reports and analyses, ranging from very simple, fixed-format reports to sophisticated analytic applications with complex embedded algorithms and domain expertise. It helps to split this spectrum into two categories based on the level of sophistication. We call these categories standard reports and analytic applications .

Standard Reports

Standard reports are at the basic end of the BI application spectrum. They are typically relatively simple, predefined-format, parameter-driven reports. In the simplest case, they are pre-run, static reports. Standard reports provide users with a core set of information about whats going on in a particular area of the business. Standard reports may sound dull, but theyre often the workhorse BI applications of the business. These are the reports people look at every day. Most of what people asked for during the requirements definition process would be classified as standard reports. Thats why we encourage you to develop a set of standard reports as part of the Lifecycle approach. Some typical standard report titles might be:

• YTD Sales vs. Forecast by Sales Rep

• Monthly Churn Rate by Service Plan

• Five-Year Dropout Rate Time Series by School

• Direct Mail Response Rates by Promotion by Product

• Audience Counts and Percent of Total Audience by Network by Day of Week and Time of Day

• YTD Claims vs. Forecast by Vehicle Type

• Call Volume by Product as a Percent of Total Product Sales

Analytic Applications

Analytic applications are more complex than standard reports. They are usually centered on a specific business process and encapsulate a certain amount of domain expertise about how to analyze and interpret that business process. They may go so far as to include complex, code-based algorithms or data mining models that help identify underlying issues or opportunities. Another advanced feature in some analytic applications is the ability for the user to feed changes back into the transaction systems based on insights gained from using the application. At the far end of the spectrum, analytic applications are sold as black-box, stand-alone applications in part to protect the intellectual capital that went into their creation. Some common analytic applications include:

• Promotion effectiveness

• Web path analysis

• Affinity program analysis

• Shelf space planning

• Fraud detection

• Category management

The more targeted an analytic application is, the easier it is to justify a standalone implementation. In other words, you can do promotion analysis without having to spend all that time and energy building the BI system. This has the ring of truth, but thats because the justification applies the entire cost of building the BI system against the value of the specific BI application.

Build or Buy?

It is possible to build your own advanced analytic applications using standard front-end tools and custom code to capture and apply the best practice business rules. For example, we worked with a company that was building a custom fraud detection application to identify suspect transactions in an Internet-based business. They chose to build rather than buy because they felt their unique data sources and unusual business processes would make it difficult to use a purchased application. There are a lot of analytic application providers out there that would disagree with this decision.

Originally, vendors tried to sell packaged analytic applications as off-the-shelf solutions. This has not worked well for exactly the reasons our client chose to build: unique data sources and unusual business processes. However, purchased analytic applications can be a great starting point. If they align reasonably well with your industry, company, and business functions, they can add significant value with relatively low effort. This is particularly true if the vendor has built their applications on a well-designed dimensional model. This will make the mapping of your model to theirs relatively easy. Unfortunately, many of the applications weve seen are based on data models that are tightly tied to the applications themselves . As a result, packaged analytic application implementations typically involve a fair amount of customization.

VIPS

Dont think you can postpone or omit BI applications from your project. There is a common misconception , fueled by the simplicity of the front-end tool demonstrations , that end users will build their own reports if we just give them the right tool. Remember, the folks who will use these BI applications are generally the top managers in the organization. They often do not have the time or interest needed to learn to build their own reports. At the same time, while their self-service analytic skills may be lacking, they are often the primary decision makers in the business. In many respects, the primary purpose of the BI system is to provide them with the information they need in a form they can use. Besides, they generally have the power to approve spending for major projects, like the BI system.

 

Another limitation (and advantage) of purchased applications is that the internal business rules and calculations are often considered intellectual property by the companies that build these packages. This could be bad because it may mean you cant review or change the underlying business rules. However, it could be good because the company that created the applications may be better at the particular business process than your company is.

There are as many analytic application packages out there as there are business functions and industries. The most common analytic applications are budgeting and forecastingprimarily because these functions are the most standardized business functions and can be replicated across businesses and industries.

BI Application Developers

Including BI applications as part of the deliverables puts the DW/BI system team in the role of desktop application developers. This is very different compared to the ETL development role. Where the ETL developer is building a system that will be used strictly by other developers, the BI applications developer is creating a product that will be used by potentially hundreds or thousands of non-technical users.

In fulfilling the BI application developer role, you need to deal with many issues that may be unfamiliar. You need to be clear about the market demandthe need for and value of your product; you need to be clear about its specifications; you need to pay careful attention to the user interface, testing it to make sure it makes sense to users and they can easily find what they need. You must test and validate the content of the reports, including all computed fields and calculations. You need to document these applications, paying special attention to the definitions and descriptions you use. You need to offer support to the users when they are having difficulty. Finally, you need to maintain the BI applications and add new ones as the business evolves and business needs change.

THE DANGER OF DASHBOARDS

One common analytic application we often have to deal with is the Executive Dashboard. The Executive Dashboard is typically envisioned as the senior executives window into the business. Its a collection of key measures (KPIs, BPMs, and so on) that are meant to tell the executive the current state of the business and its general direction. This sounds useful, and looks great in the vendor demo, but you need to be particularly careful of this type of analytic application for several reasons:

• Its hard to build and manage. The executive dashboard must pull information from across all the major business processes otherwise you cant really tell whats going on in the business. This means either the DW/BI system must be substantially completed, or you will have to build custom extracts and manual feeds to generate the dashboardand regenerate it every day, week, or month (depending on its time granularity).

• Dashboard information is often not that actionable . The tools dont give users a rich context for the information or any indicators of cause-and-effect. Without knowing the cause, you cant really determine what the appropriate response might be. While sales may be going down, it will still take analysts and frontline managers to figure out the likely causes and recommend appropriate actions. Features like the stoplight (red, yellow, and green indicators) or the speedometer are information poor. If the light turns red, what do we do, stop? If the sales speedometer starts slowing down, what do we do, hit the gas? The corporate equivalent might be for the CEO to call up the VP of Sales and start yelling.

• Its often not that useful. In many cases, the main value of the dashboard is as an early warning toolmanagement does not want to be caught by surprise. (Note that this could make it the most important project on your list.)

Some dashboards can bring huge value to the organization. Corporate performance management systems or balanced score cards can be great ways to tie an organizations performance to its goals. However, this means the organization has to clearly identify its goals and agree on how to measure them. Most of the work to create these systems is organizational and political. If you start with the dashboard first, you are starting at the wrong end of the animal.