Measures of Success

---

How you measure success is determined, in part, by the goals you wish to achieve. On an intangible level, you know your implementation is successful when there are a number of ways to measure the success of a BusinessObjects implementation:

• People have heard of BusinessObjects (or the name of your BI project/ application).

• The business sees IT as a partner and not as a gatekeeper who holds the key to corporate data.

• Users feel empowered to get to the information they need to do their jobs.

• Analysts feel they spend less time collecting data and more time analyzing data, using it to make informed decisions.

It's a paradox that a tool that allows quantitative measurement of business goals is seldom measured itself. In some respects, this is true of many IT projects in which the measure of success is simply whether the application is delivered on time and on budget. In achieving IT goals, there are a number of ways to measure success:

• Several custom OLTP reports eliminated

• Reduction in IT overtime or contract programmers for developing custom reports

• Elimination of duplicate, competing report systems

• Number of users trained versus number of users who log into BusinessObjects

• Number of queries executed each month

• Number of standard reports accessed

  Tip 

  Refer to Chapter 15, 'Monitoring User Activity,' for approaches to tracking BusinessObjects usage.

In measuring achievement of business goals, the great debate is how much can be attributed to implementing BusinessObjects versus other variables that help achieve the goal. For example, in Table 2-1, one of the company's goals is to improve market share. This can be measured by changes in revenue over time or for particular market segments. BusinessObjects provides the information to measure progress and to do more targeted marketing. However, achieving the goal may require increased promotion, improved product support and innovation, better training of customer service personnel, reduced employee turnover, and so on. Exogenous change may remove a competitor from the market, allowing a company to improve market share without having taken any other action. When several variables contribute toward achieving that goal, then assign a reasonable percentage for how much BusinessObjects contributes toward achieving the goal. In the following table, BusinessObjects contributed 15 percent to increased market share. Is this an exact number? No. Can it ever be precisely measured? No. It is merely one measure of success. So if a five-billion-dollar company increases its revenues by 10 percent in an otherwise flat market, you can say BusinessObjects contributed $500,000 toward achieving this goal (10% ´ $5B ´ 15% = $500K).

Table 2-2: BusinessObjects Contributes to the Cost Reduction or Revenue Improvement

Action to Improve Market Share	Percent Contribution
Increased promotion and modified ad campaign	30%
Improved product line	25%
Better employee training, customer service, reduced turnover	30%
BusinessObjects access to information to focus marketing efforts on most likely buyers, ensure order compliance, reduce product defect	15%

With the more specific goals described earlier, the measure of success may be an improvement over the initial situation. For example, the oil and gas company wanted to ensure customers were invoiced within three days of shipment. If the current average was seven days, how much has the days-to-invoice improved since measuring progress with BusinessObjects? You can convert this goal to a financial impact, as the earlier the invoice is sent, the faster the money can be collected. What is the value of four days' worth of accounts receivable? For the physical therapy office, curing patients in fewer visits increases patient satisfaction and doctor referrals.

Some companies that have implemented BusinessObjects can cite individual cases where BusinessObjects directly affected the bottom line. A manufacturing company used BusinessObjects to do a gap analysis of production costs between two similar facilities; they identified $1 million in operating inefficiencies. Without BusinessObjects, they would not have had the data to identify this opportunity. So perhaps you would say BusinessObjects is 50 percent responsible for the cost savings; the remaining 50 percent can be attributed to eliminating the inefficiencies. The beauty of this example is that the company started implementing BusinessObjects as a follow-on to an ERP implementation. The goal was for IT to eliminate custom, disparate reporting systems, and now, BusinessObjects is a strategic asset that has helped the company achieve a number of business goals and measurable business benefit.

Owens and Minor was one of the early adopters of WebIntelligence (WebI) and is a frequent award winner in the business intelligence industry. As a medical supplies distributor, their data warehouse contains information on suppliers' delivery performance and hospitals' purchasing volumes. By providing both the hospitals and distributors access to this intermediary data, the company attributes $100 million in new business to their BusinessObjects extranet implementation. Their WebI implementation provides a competitive advantage and holds strategic value. Yet, it too had humble beginnings. According to Don Stoller, the director of information management, the original goal was to improve productivity of the field sellers who needed access to information while visiting clients. Four years after the initial WebI implementation, WebI not only offers a competitive advantage but is its own revenue stream, as external customers pay for access.

ROI as a Measure of Success

Return on investment (ROI) is another measure of success and one that is often used to fund the project. While it is fairly easy to measure the cost of the BusinessObjects implementation (the investment portion), it is not easy to measure the return. As you saw in the preceding section, it's debatable how much of a revenue increase you can attribute to BusinessObjects versus other factors. Even when ROI is used to fund a project, companies rarely go back and measure the actual ROI. It is a precise number derived from imprecise inputs. IDC first published a study on the ROI for data warehouses in 1996. IDC determined the average three-year ROI was 401 percent for the 62 projects measured. The Data Warehouse Institute published a study in 2000, showing an ROI of 300 percent. While 47 companies participated in the study, less than a quarter measured ROI. In December 2002, IDC released another ROI study focusing on the value of business analytics, the applications that reside on top of a data warehouse. The average ROI was 431 percent, and the median was 112 percent, with less than a year payback period. Some companies had returns of more than 2,000 percent, and IDC reported that the most successful projects were when the business analytics implementation corresponded with business process improvements.

The ROI being such an imprecise measure, it's not surprising many companies never go back and calculate it for a BusinessObjects implementation. You know your project is successful according to all the other measures of success described in the preceding sections. Nonetheless, it is a number that provides a basis for comparison to other BI implementations and IT initiatives. It also is a measure well understood by finance users, a significant group of BusinessObjects users. In this respect, knowing your approximate ROI is a useful tool in promoting BusinessObjects.

The basic formula for calculating ROI over a three-year period is

ROI = [(NPV Cost Reduction + Revenue Contribution )/Initial Investment] ´ 100

Net Present Value (NPV) considers the time value of money. In simplistic terms, if the company had one million dollars to deposit in a bank today, next year, assuming a meager five-percent interest, it would be worth $1,050,000. The formula to calculate NPV of a three-year cost or revenue is

NPV = F/(1 + r)+ F/(1 + r)² + F/(1 + r)³

F is the future cash flow from the cost reductions and revenue contributions. R is the discount rate for your company. Five percent may be the interest a bank is willing to pay, but companies will have a different rate that takes into account the expected return for other investments and opportunity costs from investing in BusinessObjects versus other capital projects.

To take the earlier example of improved market share (Table 2-2), assuming

• $500,000 revenue contribution each year

• $400,000 annual savings by eliminating two custom report programmers @ 2 ´ 2,000 hours ´ $100 an hour

• 10 percent discount rate

• $1 million initial investment in hardware, software, training, and consulting to implement BusinessObjects

The projected ROI for a three-year period is

For additional information on evaluating the ROI for your implementation, William McKnight teaches an ROI course for The Data Warehouse Institute and provides a useful spreadsheet on his web site (www.mcknight-associates.com/downloads). Base Consulting provides a white paper and case study (www.baseconsulting.com) in the resources section of their web site.

---