Case Study Analysis and Discussion of Findings

---

In this study, the data mining project at Company X consisted of eight cross-functional team members—a data mining specialist, a project manager, a senior director of strategic planning (the executive sponsor), a research supervisor, a business analyst, an end user analyst, a data architect and a database administrator. One interesting aspect about this project is that it was the first time Company X had undertaken a data mining project. Consequently, the executive sponsor and project manager jointly decided, at the outset, that the entire team should be present during key project activities in order to facilitate sharing of ideas. For that matter, the entire project team was even assembled during actual execution of the data mining algorithms. The relationship between the IT department and business end users at Company X was also remarkably cordial. In fact, eight months prior to the start of this study, the IT department had just successfully completed the implementation of an enterprise-wide product purchase transaction data warehouse. The business end users were thus far delighted with its operation, functionality and end user query capability. This same data warehouse was used for the project under study to provide the required historical product purchase transaction data. Finally, a multipurpose industrial strength data mining tool was used as it afforded the opportunity to demonstrate the capabilities of the three distinct classes of data mining algorithms—namely, clustering, associations and predictive modeling. The data mining specialist from an external organization was well versed in the use of this particular tool. Table 1 is a summary of various site visits, including an overlay of the major project activities observed from this study against the straw man of what a process for performing data mining projects might look like.

Table 1: Mapping observed activities to straw man

Proposed Phase	Phase Name	Proposed Activity	Site Visit #	Observed Activity
Phase 0	Discovery	Assess data centricity Assess analytics capability Develop analytics strategy
Phase 1	Entry	Prospecting Domain analysis Problem generation Problem assessment Data sensing	1	Formal meeting between Company X business sponsor and IW manager to initiate project
Phase 2	Launch	Problem refinement Problem validation Project planning Core project team Data mining approach Data strategy Project sponsor	2	Project team workshop to identify three business problems, project roles and project plan
Phase 3	Execution Infusion	Detailed problem definition Data sourcing Data enrichment Data mining algorithms Results interpretation Results validation Information harvesting Bus. strategy formulation	3–10	Data briefing Actual data mining Findings synthesis and presentation

The data mining activities carried out for this study cut across different phases as shown in Table 1. There are in fact a number of similarities and differences that emerge once the observed activities are mapped against activities thought to occur. One important observation to state up front is that the activities observed for the project under study did not exhibit a craft-like trial and error pattern. Although at the outset of the project, the outside data mining specialist did not explicitly present a formalized process for performing the work, the project did exhibit reasonably distinct activities, some of which can be mapped back to the activities in the straw man process. There were no activities that could be mapped to Phase 0. This was unexpected and a further examination revealed two main explanations for this. First, the use of statistical techniques and OLAP is quite widespread in Company X. For that matter, Company X is an analytical-driven company. As an example, during a recent promotional advertising campaign, Company X made major changes to the campaign in midstream because sales projection results were not being met. In other words, there was no reason for Company X to assess its capability to apply analytical techniques because they were already being used on a regular and consistent basis throughout the company. Second, for Company X, this particular data mining implementation project was viewed as an "experiment" and not a formal data mining technology adoption project; thus the need to develop a formal strategy for the use of OLAP, analytics and data mining was unnecessary.

During the first site visit, a formal meeting was held at Company X between the executive sponsor and the manager of information warehousing (IW) to discuss the parameters of the data mining project. Candidate business problems were identified by the executive sponsor based on their perceived business value to senior management. The issue of how much historical product purchase transaction data to mine was discussed and agreement reached. At the conclusion of this meeting, the project under study became a formal project; it received a formal project number and budget. It is the norm at Company X for a project to have an official sponsor before it actually becomes official. This observed activity can be mapped to Phase 1. At the second site visit in September '98, a workshop with the entire project team present was held; prior to the start of this workshop, the manager of information warehousing appointed a project manager to this project. In the workshop, the project team members were introduced, roles and responsibilities were assigned and a project plan was developed. In addition, there was extensive discussion about the original set of candidate business problems. Two of the original three business problems were thrown out and two new ones were introduced. The outcome of this meeting was (1) project team member agreement about the set of three business problems that could be "mined" with existing data and (2) consensus that these same business problems would likely yield patterns in data, after data mining algorithms were applied, such that new and additional insight about the problems could hopefully be obtained. The business problems were proposed by the executive sponsor and research supervisor and unless there was a serious data issue preventing the data mining of these business problems, the proposed problems became the de facto data mining problems for the project under study. The observed emphasis on clearly specifying the business problems up front was unexpected. However, given the analytical orientation of Company X, this is not entirely surprising. There search supervisor in fact played a dominant role in framing the business problem; at one point they even took on a "hypothesis phraseology." Finally, although no formal data strategy resulted from this workshop, data gotchas were tabled and discussed. The workshop activity can be mapped to Phase 2.

For the remaining eight site visits, one was for a data debriefing meeting, six were for actual data mining algorithm execution and results analysis and the final site visit was for attendance in the results synthesis presentation meeting. These observed activities can be mapped to Phase 3 of the straw man. The lack of a formal data strategy in Phase 2 became evident in Phase 3 and necessitated a data debriefing meeting. One of the business problems identified earlier required use of external third party data that Company X simply did not have. The explanation of the integration of third party data with Company X data was the main topic of the data debrief meeting. The execution of data mining algorithms, results interpretation and validation activities were observed as was thought to possibly be the case. However, no information harvesting or business strategy formulation activities were observed. This was surprising yet upon further analysis the absence of this activity was consistent with the spirit of the project under study. The project was informally viewed as an "experiment;" therefore multiple business problems requiring use of separate rather than complimentary data mining algorithms were used. Use of different data mining techniques to validate specific data mining results was not undertaken. In order for data mining results to be the basis for strategy formulation, extensive cross validation of results is strongly recommended.

During the course of this study, a number of activities in the straw man of what a data mining process might look like were not observed. In the absence of multiple cases combined with the unique parameters governing this project, no conclusions can be made about this. The project under study was viewed as an "experiment" and therefore did not involve a capital budgeting decision about the formal acquisition of a data mining tool. It is quite likely that developing an analytics strategy would have been an important activity had this particular project involved the purchase, integration and assimilation of data mining technology.

Finally, although the project under study did not produce "revolutionary results," the data mining results did provide a different perspective for analyzing some of the common business problems at Company X. In some organizations the project under study might be viewed as an outright failure since significant insights were not produced. However, at Company X, this project was not perceived as a failure because i) the culture at Company X is "failure forward," meaning that innovation and experimentation are encouraged, ii) from the outset, the expectation of this project was not to provide completely new and unexpected patterns in data and iii) according to the executive sponsor, the project was completed on time and within budget.

---