ROI Case Study

A company with a simple web site for dispensing product information recently decided to move more of their business online, in an attempt to expand their access to customers while reducing the costs of their business transactions. Some of their competitors were already making this transition, placing further competitive pressures on them.

The company started several parallel projects in an attempt to recoup some of the time lost to their competitors. The IT group worked on sizing the infrastructures to handle a specified user and transaction volume, while the application group began designing the new web applications.

The new web applications were rushed into production quicklyas soon as the logic was tested and obvious bugs were found and fixed. This ready, fire, aim! strategy bypassed realistic load testing and other end-user QoE evaluation functions, leading to disappointing results. Initial measurements taken after deployment indicated that the actual customer volumes and transaction activity were substantially lower than the projections. The initial response to the disappointing behavior was to consider spending more for additional computing power and network bandwidth, which is the common response.

Cooler heads prevailed, fortunately, and argued for better measurements to clarify the situation rather than possibly compounding the problem with misplaced efforts and poor investments. A set of synthetic transactions established that the server response times and the network delays were not the problem; in fact, they were very low. These measurements were helpfulthe technology was performing well, although the site wasn't. They indicated that other alternatives needed investigation.

An engagement with a professional services firm that used web analytics was initiated, and they collected information on the user behavior within the web applications. The data clearly illuminated the problems. The major contributor to the disappointing outcome was that the web application was difficult to navigate, and many customers would simply click away after becoming frustrated.

The key web pages were those that guided potential customers to the offered products and services and hopefully converted their interest into actual sales. However, the navigation paths involved traversing a large number of pages with confusing content and links that were not obvious. Further discussions indicated some of this was done deliberately so that other cross-selling opportunities could be presented with each new page. This is the same annoying dead-end strategy used by many sites that churn out new pop-up ads with each page.

The web applications were able to be changed fairly quickly because they were built with JavaBeans and were, therefore, easy to modify. Shorter paths to the key content were constructed, and the intervening pages were designed with simpler layouts. Upselling was linked to the key pages rather than adding distractions on the path.

The synthetic transactions were used to verify that the changes did not add any significant delays to the original baselines. The operational results verified that the web application design, rather than the underlying technology, was the problem. The number of customers remained steady for the first six weeks or so. The critical change was that the number of customers reaching the key pages tripled and generated a 15-percent revenue gain.

The number of customers began to increase, partly by word of mouth and partly from repeat visits. Over the next six months, the number of customer visits doubled, and revenues per customer also grew considerably as the application was refined. The impact of a cleaner, shorter navigation path was that the volume growth was accommodated without any new infrastructure investments. Much of the computing and networking load was reduced because customers were not linking through meaningless pages to reach the desired content.

The ROI evaluation is summarized in Table 13-1. The costs included the professional services of an analytics firm, the internal staff time to adapt and test the web applications, and measurement tools. A case could be made that the tools will be used for many other purposes and shouldn't be expensed solely to this effort. In this situation, however, a simple analysis was deemed sufficient.

Table 13-1. Case Study ROI Payback Summary

Cost		Benefit
Web analytics services	$ 80,000	Net revenue gain of 15 percent for first six weeks	$54,000
Rework web applications	$ 55,000	Net revenue gain over next six months	$2,153, 000
Staff time for testing	$ 7,500
Measurement tools	$ 22,500
Total	$ 165,000		$2,207, 000

The benefits included the revenue increases over the initial deployment period and for the following six months. The hard numbers indicate the project reached a break-even point in one month (on an annual basis). The project solved the problems and had a definite business impact.

There were other benefits that deserve mention as well. The infrastructure showed a 100 percent leverage, doubling traffic with no additional investments after the applications were modified, and the unit costs of transactions were halved as a result.