Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial Group

Linking the IT Balanced Scorecard to the Business Objectives at a Major Canadian Financial Group[1]

Win Van Grembergen
University of Antwerp, Belgium

Ronald Saull
London Life, Canada

Steven De Haes
University of Antwerp Management School, Belgium

Copyright 2004, Idea Group Inc. Copying or distributing in print or electronic forms without written permission of Idea Group Inc. is prohibited.

Abstract

The Balanced Scorecard (BSC) initially developed by Kaplan and Norton is a performance management system that enables businesses to drive strategies based on measurement and follow-up. In recent years, the BSC has been applied to information technology (IT). The IT BSC is becoming a popular tool, with its concepts widely supported and dispersed by international consultant groups such as Gartner Group, Renaissance Systems, Nolan Norton Institute, and others. As a result of this interest, the first real-life applications are starting to emerge. In this paper, the development and implementation of a departmental BSC within an Information Services Division (ISD) serving a Canadian financial group will be described and discussed. We use an IT BSC maturity model to determine the maturity level of the IT BSC under review.

[1]A version of this chapter also appeared in the Journal of Information Technology Cases and Applications, 5(1), 2003.


Introduction

Kaplan and Norton (1992, 1993, 1996a, 1996b) introduced the Balanced Scorecard (BSC) at an enterprise level. Their fundamental premise is that the evaluation of a firm should not be restricted to a traditional financial evaluation but should be supplemented with measures concerning customer satisfaction, internal processes and the ability to innovate. Results achieved within these additional perspective areas should assure future financial results and drive the organization towards its strategic goals while keeping all four perspectives in balance. For each of the four perspectives they propose a three layered structure: (1) mission (e.g., to become the customers' most preferred supplier), (2) objectives (e.g., to provide the customers with new products), and (3) measures (e.g., percentage of turnover generated by new products). The Balanced Scorecard can be applied to the IT function and its processes as Gold (1992, 1994) and Willcocks (1995) have conceptually described and has been further developed by Van Grembergen and Van Bruggen (1997), Van Grembergen and Timmerman (1998) and Van Grembergen (2000).

In this chapter, the development and implementation of an IT BSC within the Information Services Division (ISD) of a Canadian tri-company financial group consisting of Great-West Life, London Life and Investors Group (hereafter named The Group) is described and discussed. We use an IT BSC maturity model (adapted from the capability maturity model developed by the Software Engineering Institute) to determine the maturity level of the IT BSC under review. An important conclusion of the paper is that an IT BSC must go beyond the operational level and must be integrated across the enterprise in order to generate business value. This can be realized through establishing a linkage between the business Balanced Scorecard and different levels of IT Balanced Scorecards and through the definition of clear cause-and-effect relationships between outcome measures and performance drivers throughout the whole scorecard.


It Balanced Scorecard Concepts

In Figure 1, a generic IT Balanced Scorecard is shown (Van Grembergen & Van Bruggen, 1998). The User Orientation perspective represents the user evaluation of IT. The Operational Excellence perspective represents the IT processes employed to develop and deliver the applications. The Future Orientation perspective represents the human and technology resources needed by IT to deliver its services over time. The Business Contribution perspective captures the business value created from the IT investments.

USER ORIENTATION

BUSINESS CONTRIBUTION

How do users view the IT department?

Mission

To be the preferred supplier of information systems.

Objectives

  • Preferred supplier of applications
  • Preferred supplier of operations vs. proposer of best solution, from whatever source
  • Partnership with users
  • User satisfaction

How does management view the IT department?

Mission

To obtain a reasonable business contribution from IT investments.

Objectives

  • Control of IT expenses
  • Business value of IT projects
  • Provision of new business capabilities

OPERATIONAL EXCELLENCE

FUTURE ORIENTATION

How effective and efficient are the IT processes?

Mission

To deliver effective and efficient IT applications and services.

Objectives

  • Efficient and effective developments
  • Efficient and effective operations

How well is IT positioned to meet future needs?

Mission

To develop opportunities to answer future challenges.

Objectives

  • Training and education of IT staff
  • Expertise of IT staff
  • Research into emerging technologies
  • Age of application portfolio


Figure 1: Generic IT Balanced Scorecard

Each of these perspectives has to be translated into corresponding metrics and measures that assess the current situation. These assessments need to be repeated periodically and aligned with pre-established goals and benchmarks. Essential components of the IT BSC are the cause-and-effect relationships between measures. These relationships are articulated by two key types of measures: outcome measures and performance drivers. A well developed IT scorecard contains a good mix of these two types of measures. Outcome measures such as programmers' productivity (e.g., number of function points per person per month) without performance drivers such as IT staff education (e.g., number of educational days per person per year) do not communicate how the outcomes are to be achieved. And performance drivers without outcome measures may lead to significant investment without a measurement indicating whether the chosen strategy is effective. These cause-and-effect relationships have to be defined throughout the whole scorecard (Figure 2): More and better education of IT staff (future orientation) is an enabler (performance driver) for a better quality of developed systems (operational excellence perspective) that in turn is an enabler for increased user satisfaction (user perspective), that eventually will lead to higher business value of IT (business contribution).

IF

 

IT employees' expertise is improved

(future orientation)

THEN

 

this may result in a better quality of developed systems

(operational excellence)

THEN

 

this may better meet user expectations

(user orientation)

THEN

 

this may enhance the support of business processes

(business contribution)


Figure 2: Cause-and-Effect Relationships

The proposed standard IT BSC links with business through the business contribution. The relationship between IT and business can be more explicitly expressed through a cascade of Balanced Scorecards (Van der Zee, 1999; Van Grembergen, 2000). In Figure 3, the relationship between IT scorecards and the business scorecard is illustrated. The IT Development BSC and the IT Operational BSC both are enablers of the IT Strategic BSC that in turn is the enabler of the Business BSC. This cascade of scorecards becomes a linked set of measures that will be instrumental in aligning IT and business strategy and will help to determine how business value is created through information technology.

click to expand
Figure 3: Balanced Scorecards Cascade


Research Methodology

Case research is particularly appropriate for research within the IT area because researchers in this field often lag behind practitioners in discovering and explaining new methods and techniques (Benbasat et al., 1987). This is certainly true for the Balanced Scorecard and its application to IT. The Balanced Scorecard is becoming a popular technique, with its concepts supported and dispersed by consultants. A single case design is appropriate when "the investigator has access to a situation previously inaccessible to scientific observation" (Yin, 1994). Like Benbasat et al. (1987) we believe "that the case research strategy is well-suited to capturing the knowledge of practitioners and developing theories from it."

A case study research approach is used to study the phenomenon of the IT BSC and its development and implementation in a single organization. In case study research, the researcher is an observer/investigator rather than a participant (Benbasat et al., 1987). The Chief Information Officer (CIO) of the case company (also the second author of the article) applied the Balanced Scorecard technique to his IT organization. The other co-authors conducted all interviews (including interviews with the CIO) to gather data for this study. Their role was purely the role of observers who were interested in investigating how the IT BSC concepts they and other researchers developed in earlier publications were applied by practitioners and how the experience and knowledge of practitioners could help to improve the earlier proposed IT BSC frameworks. Although the CIO/author used one of the leading author's publications (Van Grembergen & Van Bruggen, 1997) to build his first scorecard, the leading author/researcher was never involved as an advisor in the further developments and implementations.

The initial research took place from the end of 1999 until mid 2000 (Period-1). The IT BSC project within the case company is still an ongoing project. During the revision period of August-November 2002 (Period-2), the article has been updated.

In both research periods, the data were collected through in-depth interviews with the CIO by means of multiple e-mail conversations and also through some casual face-to-face conversations when the authors met during international conferences on IT performance measurement. During the second research period in 2002, additional indepth interviews were conducted with the project manager of the IT Balanced Scorecard project. Also, six individuals who have key roles and accountabilities for scorecard deliverables at the Group were interviewed (including the Vice President Information Services, the Financial Control Director, the Operations & Technical Support Technology Services Director, the Mainframe Technical Support Manager, the Career Centers Director and the Project Management Career Center Leader). These interviews were done by means of e-mail and telephone conversations and an intensive workshop at the headquarters of the company. Data from other sources such as internal reports and slides from the CIO's presentations for his management were used to develop and complete an understanding of the case company, its processes, its technology, its IT organization and its development and implementation of the IT BSC.


Case Company Introduction A Tri Company

The Great-West Life Assurance Company, London Life and Investors Group are members of the Power Financial Corporation group of companies, with London Life as a wholly owned subsidiary of The Great-West Life Assurance Company. In 2001, MacKenzie Financial was also acquired by the Power Financial Corporation Group, but as the IT Balanced Scorecard project does not cover this company, MacKenzie's organization and IT division will not be taken into account in this article.

The Great-West Life Assurance Company is an international corporation offering life insurance, health insurance, retirement savings, specialty reinsurance and general insurance, primarily in Canada and the United States. Great-West serves the financial security needs of more than 13 million people in Canada and the United States. Great-West has more than $86.9 billion (all figures in this article are in Canadian dollars) in assets under administration and $477 billion of life insurance in force. Founded in Winnipeg in 1891, Great-West is now a leading life and health insurer in the Canadian market in terms of market share.

London Life was founded in Ontario in 1874 and has the leading market share of individual life insurance in Canada. London Life markets life insurance, disability insurance and retirement savings and investment products through its exclusive sales force. The company is a supplier of reinsurance, primarily in the US and Europe, and is a 39% participant in a joint venture life insurance company, Shin Fu, in Taiwan. London Life has more than $30 billion assets under administration and $142.6 billion of life insurance in force.

Investors Group, with its corporate headquarters in Winnipeg, was founded more than 70 years ago. Investors Group is Canada's leading provider of mutual funds, offering a wide spectrum of funds, including those created through strategic partnerships with some of the best known Canadian and international investment management firms. It also offers a wide range of insurance and mortgage options, and currently has $17.1 billion of life insurance coverage in force through three different carriers, and administers with more than $7.6 billion of primarily residential mortgages. Investors Group manages assets of $40.5 billion.


The Tri Company IT Merger

The trend in financial services industry consolidation was a motivating factor behind the acquisition of London Life by Great-West Life and the merger of the IT divisions of the three companies in November 1997. At that time, the tri-company IT expenditures had exceeded $200 million. The ability to reduce these costs and to achieve true synergies and economies of scale within the IT operations was clearly a driver and opportunity for the companies to realize. The merger enabled single systems solutions across all three companies to be explored and implemented as well as single operational processes. Forming a tri-company shared services organization positioned management to:

  • achieve world-class status as an information services group,
  • maximize purchasing power and operating efficiency,
  • leverage technology investments,
  • optimize technical infrastructure and application support costs.

Figure 4 depicts the current IT organizational structure of the merged IT division, which employed 812 full-time/part-time employees at the time of the second research period. Also, the position of the IT division relative to the higher reporting levels is indicated. Application Delivery and Technology Services are respectively the traditional IT department's Systems Development and Operations of the combined organizations. Application Delivery is separated from account management and people management in order to focus on continuous improvement of delivery performance. Account Management is the linkage with the clients/users. This component ensures effective communication and translation of business needs into IT processes and educates users on the IT corporate agendas. Account Management employs IT generalists who provide IT insights into business strategy and decision making. Career Centers are focused on the professional development of IT people and ensure attention to people issues in order to reduce turnover of talented IT employees. Corporate Technology enables the development of a common architecture and provides technology directions. The eBusiness Solution Center works on the introduction of new technologies that enable eBusiness solutions for The Group. Management Services focuses on running IT as a business and ensures effective financial management and management reporting, including IT scorecard reporting.

click to expand
Figure 4: Organization Chart of the Merged IT Division


IT BSC Project and Its Organization

Before the merger, the CIO of Great-West Life (who is the present CIO of the merged IT division) began focusing on the scorecard as a (potentially) effective measurement tool. His objective was to ensure that IT was fairly evaluated. In his own words: "Through the Balanced Scorecard I would know what was important to the business and I would not fall victim to the early termination syndrome. Or at least I would have a better chance of survival." However, once the three companies came together through the acquisition and merger of the IT groups, the stakes were raised considerably. Now, the IT division had exposures on multiple fronts with stakeholders who were concerned about the perceived loss of control over their vital IT services. This prompted an executive request for a formal measure of factors to measure IT success. The response of the merged IT division was to formalize the criteria into a new and extended IT scorecard based on the experiences gained within Great-West Life.

Senior management of all the three companies questioned the benefits of huge investments in IT and how more value might be achieved through better alignment of business strategy and IT strategy. Within The Group the specific concerns for the different stakeholders are shown in Figure 5.

Stakeholders

Key questions

Board of Directors

Executive Management Committee

Does IT support the achievement of business objectives?

What value does the expenditure on IT deliver?

Are IT costs being managed effectively?

Are IT risks being identified and managed?

Are targeted inter-company IT synergies being achieved?

Business Unit Executives

Are IT's services delivered at a competitive cost?

Does IT deliver on its service level commitments?

Do IT investments positively affect business productivity or the customer experience?

Does IT contribute to the achievement of our business strategies?

Corporate Compliance Internal Audit

Are the organization's assets and operations protected?

Are the key business and technology risks being managed?

Are proper processes, practices and controls in place?

IT Organization

Are we developing the professional competencies needed for successful service delivery?

Are we creating a positive workplace environment?

Do we effectively measure and reward individual and team performance?

Do we capture organizational knowledge to continuously improve performance?

Can we attract/retain the talent we need to support the business?


Figure 5: IT Concerns of the Different Stakeholders

The concepts of the Balanced Scorecard and its application to information technology were discovered through an Internet search, primarily through the web site of the IT Governance Institute (www.itgi.org). Departing from this web site, relevant publications on the IT Balanced Scorecard from academics and practitioners were identified and consulted. It was believed that the scorecard could provide an answer to the key questions of the different stakeholders.

The formal development of the IT Balanced Scorecard began in 1998, and from the start the objectives were clearly stated:

  • align IT plans and activities with business goals and needs,
  • align employees' efforts toward IT objectives,
  • establish measures for evaluating the effectiveness of the IT organization,
  • stimulate and sustain improved IT performance,
  • achieve balanced results across stakeholder groups.

At the beginning of the initial research period (December 1999), the situation was that the scorecard effort was not yet approached as a formal project and as a result, progress had been somewhat limited. In 2000 the formality of the project was increased and the CIO (Information Services Executive) was appointed as sponsor. In 2001, a project manager/analyst was formally assigned to the IT Balanced Scorecard project. Status to-date (end of 2002) is that the case company is still completing the scorecard: 66% of the measures are completed, 29% are in progress and 5% are not yet started.


Building the IT Bsc

It was recognized by the CIO that building an IT BSC was meaningful under two conditions which required (1) a clearly articulated business strategy, and (2) the new Information Services Division moving from a commodity service provider to a strategic partner, as illustrated by Venkatraman (1999) (Figure 6).

Service provider

Strategic partner

  • IT is for efficiency
  • Budgets are driven by external benchmarks
  • IT is separable from the business
  • IT is seen as an expense to control
  • IT managers are technical experts
  • IT for business growth
  • Budgets are driven by business strategy
  • IT is inseparable from the business
  • IT is seen as an investment to manage
  • IT managers are business problem solvers


Figure 6: IT Division as a Service Provider or Strategic Partner

The newly constructed ISD is viewed as a strategic partner. During several meetings between IT and executive management, the vision, strategy, measures of success and value of IT were jointly created. Typically, pure business objectives were used as the standard to assess IT. The vision and strategy of ISD were defined as:

  • ISD is a single IT organization focused on developing world-class capabilities to serve the distinct customer needs of its three sponsoring companies,
  • ISD operates as a separate professional services business on a full recovery, nonprofit basis,
  • ISD supports the achievement of company strategies and goals through the industry consolidation period,
  • ISD becomes the "supplier of choice" of information services,
  • ISD establishes a forward looking enterprise architecture strategy which enables the use of technology as a competitive edge in the financial service market place,
  • ISD becomes the "employer of choice" for career-oriented IT professionals in the markets in which ISD and The Group operate.

These issues go to the heart of the relationship between IT and the business and will be reflected in the IT strategic Balanced Scorecard, as is illustrated in Figures 7 and 8. Figure 7 shows the perspective questions and mission statements for the four quadrants: corporate contribution, customer orientation, operational excellence and future orientation. Figure 8 displays the measures for each perspective. The details regarding the individual perspectives and their measures are in the annex.

CUSTOMER ORIENTATION

CORPORATE CONTRIBUTION

Perspective question

How should IT appear to business unit executives to be considered effective in delivering its services?

Perspective question

How should IT appear to the company executive and its corporate functions to be considered a significant contributor to company success?

Mission

To be the supplier of choice for all information services, either directly or indirectly through supplier relationships.

Mission

To enable and contribute to the achievement of business objectives through effective delivery of value added information services.

OPERATIONAL EXCELLENCE

FUTURE ORIENTATION

Perspective question

At which services and processes must IT excel to satisfy the stakeholders and customers?

Perspective question

How will IT develop the ability to deliver effectively and to continuously learn and improve its performance?

Mission

To deliver timely and effective IT services at targeted service levels and costs.

Mission

To develop the internal capabilities to continuously improve performance through innovation, learning and personal organizational growth.


Figure 7: Perspective Questions and Mission Statements of the IT Strategic Scorecard

click to expand
Figure 8: IT Strategic Scorecard Framework


Maturity of the Developed IT BSC

At the beginning of the project, the IT BSC was primarily focused on the operational level of the IT department. It was acknowledged from the beginning that this could not be the end result. Therefore, actions were started to go beyond the operational IT BSC and to measure the true value of IT at the business level. The organization established two ways to demonstrate the business value, one at service delivery level and one at the IT strategy level. As will be illustrated hereafter, the goal is to evolve to an IT strategic BSC that shows how the business objectives are enabled by IT.

A cascade of Balanced Scorecards has been established to create a link between the scorecards at the unit level and the overall business objectives (see Figure 9). A link between the IT BSC and the Business BSC is not yet implemented, as there is currently no formal Business BSC for the Group. The scorecards at the unit level are classified into three groups: operational services scorecards (e.g., IT service desk scorecard), governance services scorecards (e.g., career center scorecard), and development services scorecards (e.g., application development scorecard). The measures of these unit scorecards are rolled-up or aggregated in the IT strategic Balanced Scorecard. This in turn is fed into and evaluated against the business objectives. In this way, the service (and value) delivered by IT is directly measured against the objectives of the overall business. Further, on an annual basis, the IT strategic BSC is reviewed by business and IT management and the result is fed back into the next annual planning cycle. This planning cycle defines what the business needs are and what IT must do to accomplish those needs.

click to expand
Figure 9: Cascade of Scorecards to Link Unit Scorecards, IT Strategic Scorecard and Business Objectives

For example, from the IT service desk scorecard (i.e., a unit scorecard which is situated in the operational services scorecard group), metrics such as average speed of answer, overall resolution rate at initial call and call abandonment rate (all three customer orientation metrics) are rolled-up to service level performance metrics in the IT strategic Balanced Scorecard. Other metrics of this unit scorecard, such as expense management (corporate contribution perspective), client satisfaction (customer orientation perspective), process maturity of incident management (operational excellence perspective) and staff turnover (future orientation perspective), will aggregate as part of the IT strategic scorecard. The overall view of the IT strategic Balanced Scorecard is then fed into and evaluated against the defined business objectives.

The second way to demonstrate business value is situated within the IT strategic Balanced Scorecard. The cause-and-effect relationships between performance drivers and outcome measures of the four quadrants are established as indicated in Figure 10. These connections help to understand how the contribution of IT towards the business will be realized: building the foundation for delivery and continuous learning and growth (future orientation perspective) is an enabler for carrying out the roles of the IT division's mission (operational excellence perspective) that is in turn an enabler for measuring up to business expectations (customer expectations perspective), that eventually must lead to ensuring effective IT Governance (corporate contribution perspective).

click to expand
Figure 10: Cause-and-Effect Relationships within the IT Strategic Balanced Scorecard

Establishing the link with the business objectives through a cascade of scorecards and defining the cause-and-effect relationships within the scorecards are important steps in determining the maturity of the IT Balanced Scorecard. This maturity can be assessed through a maturity model. We therefore used an IT Maturity Model (MM) to match the case company's scorecard level against the levels of the IT MM. The IT MM we used is based on the Software Engineering Institute's Capability Maturity Model CMM (Paulk et al., 1993). Our IT BSC Maturity Model highlights five maturity levels with the characteristics found in Figure 11.

Level 1 Initial

There is evidence that the organization has recognized there is a need for a measurement system for its information technology division. There are ad hoc approaches to measure IT with respect to the two main IT processes, i.e., operations and systems development. This measurement process is often an individual effort in response to specific issues.

Level 2 Repeatable

Management is aware of the concept of the IT Balanced Scorecard and has communicated its intent to define appropriate measures. Measures are collected and presented to management in a scorecard. Linkages between outcome measures and performance drivers are generally defined but are not yet precise, documented or integrated into strategic and operational planning processes. Processes for scorecard training and review are informal and there is no compliance process in place.

Level 3 Defined

Management has standardized, documented and communicated the IT BSC through formal training. The scorecard process has been structured and linked to business planning cycle. The need for compliance has been communicated but compliance is inconsistent. Management understands and accepts the need to integrate the IT BSC within the alignment process of business and IT. Efforts are underway to change the alignment process accordingly.

Level 4 Managed

The IT BSC is fully integrated into the strategic and operational planning and review systems of the business and IT. Linkages between outcome measures and performance drivers are systematically reviewed and revised based upon the analysis of results. There is a full understanding of the issues at all levels of the organization that is supported by formal training. Long term stretch targets and priorities for IT investment projects are set and linked to the IT scorecard. A business scorecard and a cascade of IT scorecards are in place and are communicated to all employees. Individual objectives of IT employees are connected with the scorecards and incentive systems are linked to the IT BSC measures. The compliance process is well established and levels of compliance are high.

Level 5 Optimized

The IT BSC is fully aligned with the business strategic management framework and vision is frequently reviewed, updated and improved. Internal and external experts are engaged to ensure industry best practices are developed and adopted. The measurements and results are part of management reporting and are systematically acted upon by senior and IT management. Monitoring, self-assessment and communication are pervasive within the organization and there is optimal use of technology to support measurement, analysis, communication and training.


Figure 11: Maturity Levels for the IT Balanced Scorecard

According to this IT BSC maturity model the case company is at the "Repeatable" stage (Level 2). Parts of the Level 3 maturity are achieved, but the basic principle of maturity assessment states that all conditions have to be fulfilled before moving to a higher maturity level. The challenge is to reach stage 4, the "Managed" level, within two to three years. It is understood that major milestones in this further development will be:

  • the detailed cause-and-effect relationships between the output measures and performance drivers have to be further elaborated,
  • short and long term targets have to be further defined,
  • individual and group objectives of IT employees have to be further linked to the IT BSC,
  • the scorecards have to be further integrated in the strategic and operational management processes.

The construction of cause-and-effect relationships is a critical issue in the further development of the IT strategic BSC. These relationships have not yet been explicitly defined, although they are implicit in the existing scorecard. E.g., the Professional development days per staff member measure (Figure 16 in annex) can be identified as a performance driver for the outcome measures Development process performance (Figure 15). The Corporate contribution perspective of Figure 13 is an enabler (performance driver) of the (generic) business objectives of the financial Group with its specific measures such as Business/IT alignment, Value delivery, Cost management, Risk management, and Inter-company synergy achievement. The CIO and its executive management are aware that an explicit articulation of these relationships has to be done and that it may help to improve the IT strategic BSC and its link with the business objectives, later on with the implementation of a Business BSC.

IT management is now in the process of determining how they might progress in terms of maturity level over time. The ultimate goal is to reach the "Managed" Level 4. Figure 12 displays the concrete improvements plans for the further development of the scorecard as articulated by IT management.

Improvements for 2002 (in progress)

  • improvement in unit cost measurements
  • target state and project architecture approval process
  • development of operational service baselines and targets, and unit scorecards
  • implementation of personal development days measurement process

Improvements for 2003 and later (in order of priority)

  • risk management measure
  • customer satisfaction
  • 'state of the infrastructure' assessment
  • 'lessons learned' sharing process
  • explicit articulation of cause-and-effect relationships


Figure 12: Improvement Plans for the IT BSC

The improvement in unit cost measurements should enable the organization to break down the IT activities' costs in a more detailed level. The second improvement for 2002 refers to the approval processes of both the overall enterprise architecture and the systems level architectures delivered through major projects. The enterprise architecture dictates certain architectural and technical standards for application and technical systems and is reviewed and reapproved on a regular basis. In 2002, operational services baselines and targets and unit scorecards will be developed and a measurement process for personal development will be implemented. For 2003 and the short-term future, the development priority is set on the establishment of a risk management measure. The goal is to develop an overall risk management strategy and measure our attainment of the defined target state risk level. Next, a regular survey process using generic questions needs to be developed to measure customer satisfaction and a process for assessing the "state of the infrastructure" will be implemented. This assessment compares the status-to-date of the existing infrastructure against the "to-be" position. Lowest priority but scheduled on the short- and midterm improvements plan is the sharing process of "lessons learned" on development projects.

Based on our maturity model shown in Figure 11, actions for the subsequent years should put the case company close to Level 4. It is the belief of the CIO that these plans are realistic but that "this desired timeline is probably quite optimistic and it may well take twice as long to accomplish these changes." However, the most important aspect is that all stakeholders in the process were engaged by the end of 2001 and that progress is made each subsequent year.


Lessons Learned

The following lessons can be attributed to this IT BSC case:

  1. Start simultaneously constructing a business and IT scorecard. The IT BSC within the case company was started within the IT organization primarily with the objective to ensure that IT is fairly evaluated by the business. This is a rather defensive approach and focuses merely on the internal IT processes. Although it is clearly recognized within the case company that a more explicit linkage with the business (with a business Balanced Scorecard) has to be developed and supported, the question still remains whether it is more appropriate (a) to start with a business Balanced Scorecard followed by the subsequent creation of the corresponding IT scorecards or (b) to develop both scorecards simultaneously. It is now our conclusion that it is probably more ideal to start simultaneously with both scorecards which requires both IT and senior management to discuss the opportunities of information technologies, which supports the IT/business alignment and IT Governance process.
  2. Consider the scorecard technique as a supportive mechanism for IT/business alignment and IT Governance. Recurring issues in IT practice and IT academic publications focus on how to align IT and business and how to control IT. It is our strong belief that a cascade of business and IT Balanced Scorecards may support both processes. However, as is shown in this case study, the Balanced Scorecard is only a technique that can only be successful if the business and IT work together and act upon the measurements of the scorecards. The Balanced Scorecard approach will only have results when other mechanisms such as a well functioning Board and IT Steering Committee are in place.
  3. Consider the construction and implementation of an IT Balanced Scorecard as an evolutionary project. Constructing an IT Balanced Scorecard is not a one week project. It requires considerable time and other major resources. Moreover, it is a project that is to be matured over time and that is characterized by different stages, as is illustrated by the IT BSC Maturity Model introduced in this paper. This iterative approach is confirmed by this case. The described IT BSC began at a lower level with actions currently in place to reach a higher level where a more explicit connection exists between outcome measures and performance drivers, and where an explicit linkage is established with business requirements.
  4. Provide a formal project organization. Good project management is a critical success factor for effective construction and implementation of an IT Balanced Scorecard. IT management of the case company confronted with the question of how the IT BSC project was organized had to admit that in the beginning, there was no real formal organization in place and that this delayed the progress of its implementation. Currently, the sponsor of the IT BSC is the CIO, and one full-time project manager is assigned to the project. A group of 15 individuals have key roles and accountabilities for scorecard deliverables.
  5. Provide best IT practices. Introducing an IT Balanced Scorecard in an IT environment with poor management and IT practices is too large a challenge. The implementation of the IT BSC within the case company was certainly supported by practices already in place, such as ROI-evaluation of IT projects, the existence of IT steering committees, Service Level Agreement practices, etc. If it is decided to implement, e.g., the Information Economics approach to score and evaluate projects and to integrate this method within the IT BSC, this will take considerable time and is to be seen as a separate project.
  6. Revisit the dynamic measures. The implementation of the IT Balanced Scorecard requires the establishment and definition of a large number of metrics. As business requirements change, these metrics are dynamic and should be reevaluated on a regular basis. Most important in this regard is that it should always remain very clear why a certain issue is measured, i.e., what the value is of measuring it. When this value cannot be demonstrated any more, a measure should be challenged and changed or replaced by another one.


Conclusion

In this chapter, the development and implementation of an IT Balanced Scorecard within a large Canadian insurance group is described and discussed. It was shown that building and implementing such a scorecard is a project that needs substantial human and financial resources. Furthermore, setting up an IT BSC is a project that is characterized by different phases in time. The current status of the case scorecard is Level 2 of the IT BSC Maturity Model that is introduced in this paper. This implies that the case IT scorecard to date has to be linked with the business scorecard or at least the business objectives to support the IT/business alignment process and the IT Governance process. Currently, a plan for the next two years has been developed with the objective to build a mature IT BSC explicitly linked to the business. It is recognized within the case company that this will be a great challenge for both IT and business people.

The case under review illustrated one of the most crucial issues in building and implementing an IT strategic Balanced Scorecard: its required linkage with the business objectives. To create this link, a cascade of Balanced Scorecards has been established with the lower level unit scorecards for the operational and development services. The measures of these unit scorecards are rolled-up or aggregated in the IT strategic scorecard that ultimately realizes the link with the business objectives through its corporate contribution perspective. The precise articulation of the cause-and-effect relationships through the identification of outcome measures and their corresponding performance drivers seemed to be a critical success factor. These relationships are implicit in the current IT strategic Balanced Scorecard but are to be defined more explicitly.


References

Bensabat, I.,Goldstein, D., & Mead, M. (1987). The case research strategy in studies of information systems. MIS Quarterly, September, 369–386.

Gold, C. (1992). Total quality management in information services - IS measures: A balancing act. Boston, MA: Research Note Ernst & Young Center for Information Technology and Strategy.

Gold, C. (1994). U.S. measures - A balancing act. Boston, MA: Ernst & Young Center for Business Innovation.

IT Governance Institute. (2000). CobiT (Control Objectives for IT and related Technologies) 3e edition. Available online: www.isaca.org/cobit.htm.

Kaplan, R., & Norton, D. (1992). The Balanced Scorecard - measures that drive performance. Harvard Business Review, (January/Febraury), 71–79.

Kaplan, R., & Norton, D. (1993). Putting the Balanced Scorecard to work. Harvard Business Review, (September/October), 134–142.

Kaplan, R., & Norton, D. (1996a). Using the Balanced Scorecard as a strategic management system. Harvard Business Review, (January/February), 75–85.

Kaplan, R., & Norton, D. (1996b). The Balanced Scorecard: Translating vision into action. Boston, MA: Harvard Business School Press.

Paulk, M.,Curtis, B.,Chrissis, M.B., & Weber, C. (1993). Capability maturity model for software, version 1.1. Technical Report Software Engineering Institute, CMU/SEI-93-TR-024, ESC-TR-93-177.

Saull, R. (2000). The IT Balanced Scorecard - A roadmap to effective governance of a shared services IT organization. Information Systems Control Journal (previously IS Audit and Control Journal), 2, 31–38.

Van der Zee, J. (1999). Alignment is not enough: Integrating business and IT management with the Balanced Scorecard. Proceedings of the 1st Conference on the IT Balanced Scorecard, Antwerp (March, pp. 1–221).

Van Grembergen, W. (2000). The Balanced Scorecard and IT governance. Information Systems Control Journal (previously IS Audit & Control Journal), 2, 40–43.

Van Grembergen, W., & Timmerman, D. (1998). Monitoring the IT process through the balanced score card. Proceedings of the 9th Information Resources Management (IRMA) International Conference, Boston, MA (May, pp. 105–116).

Van Grembergen, W., & Van Bruggen, R. (1997). Measuring and improving corporate information technology through the Balanced Scorecard technique. Proceedings of the Fourth European Conference on the Evaluation of Information technology, Deflt, (October, pp. 163–171).

Venkatraman, N. (1999). Valuing the IS contribution to the business. Computer Sciences Corporation.

Willcocks, L. (1995). Information management. The evaluation of information systems investments. London: Chapman & Hall.

Yin, K. (1994). Case study research. Design and methods. Thousand Oaks, CA: Sage Publications.


Appendix Details on the Measures of the IT Strategic Scorecard

In this annex, the IT strategic Balanced Scorecard is discussed in more detail (see also Saull, 2000). In each of the four quadrants, the objectives, measures and benchmarks will be elaborated. Many of the measures are rolled-up or aggregated from the unit scorecards (e.g., data center scorecard) to metrics in the IT strategic scorecard. Some of the measures defined in the IT strategic scorecard are high-level but cover specific concrete metrics. At this moment, the collecting process of the data is often very laborintensive, but it is the belief of management that first the correct measures have to be defined before implementing tools that can automate the data collecting process.


Corporate Contribution Scorecard

The Corporate contribution perspective evaluates the performance of the IT organization from the viewpoint of executive management, the Board of Directors and the shareholders, and provides answers to the key questions of these stakeholders concerning IT Governance (cf., Figure 5). The key issues, as depicted by Figure 13, are business/ IT alignment, value delivery, cost management, risk management and inter-company synergy achievement. Benchmarks have been used where an objective standard was available or could be determined in most cases from external sources.

Objective

Measures

Benchmarks

Business/IT alignment

  • Operational plan/budget approval
  • Not applicable

Value delivery

  • Measured in business unit performance
  • Not applicable

Cost management

  • Attainment of expense and recovery targets
  • Attainment of unit cost targets
  • Industry expenditure comparisons
  • Compass operational 'top performance' levels

Risk management

  • Results of internal audits
  • Execution of Security Initiative
  • Delivery of disaster recovery assessment
  • OSFI sound business practices
  • Not applicable
  • Not applicable

Inter-company synergy achievement

  • Single system solutions
  • Target state architecture approval
  • Attainment of targeted integration cost reductions
  • IT organization integration
  • Merger & Acquisition guidelines
  • Not applicable
  • Not applicable
  • Not applicable


Figure 13: Corporate Contribution Scorecard

The main measurement challenges are with the areas of business/IT alignment and the value delivery.

Currently, business/IT alignment is measured by the approval of the IT operational plan and budget. Although not a discrete measure of alignment, the approval process within the Group is particularly thorough and as a result is accepted by business executives as a good indicator. All aspects of development, operations and governance/ support services are examined and challenged to ensure they are essential to achieving business objectives or supporting the enabling IT strategy.

In the value delivery area, the performance of a specific IT services group delivering to a specific business unit (e.g., 'group insurance' services) is measured. For each business unit, specific metrics are and/or will be defined. The ultimate responsibility for achieving and measuring the business value of IT rests with the business and is reflected in the business results of the individual lines of business in different ways, depending on the nature of value being sought.

Cost management is a traditional financial objective and is in the first place measured through the attainment of expense and recovery targets. The expenses refer to the costs that the IT organization has made for the business, and the recovery refers to the allocation of costs to IT services and the internal charge back to the business. All IT costs are fully loaded (no profit margin) and recovered from the lines of business on a fair and equitable basis as agreed to by the companies' CFOs. Comparisons with similar industries will be drawn to benchmark these metrics. Next to this, IT unit costs (e.g., application development) will be measured and compared to the 'top performing levels' benchmark provided by Compass.

The development of the risk management metrics are the priority for the upcoming year. At this moment, the results of the internal audits are used and benchmarked against criteria provided by OSFI, the Canadian federal regulator in the financial services sector. The execution of the Security Initiative and the delivery of a Disaster Recovery Assessment need to be accomplished in the upcoming year. This will enable the business to get an insight on how well it is prepared to respond to different disaster scenarios.

Synergy achievement is measured through the achievement of single system solutions, targeted cost reductions and the integration of the IT organizations. This measure is very crucial in the context of the merger of the three IT organizations in the sense that it enables a post-evaluation of this merger and demonstrates to management whether the new IT organization is effective and efficient. The selection of single system solutions was a cooperative effort between business leaders and IT staff, resulting in a "Target State Architecture" depicting the target applications architecture. The synergy targets were heavily influenced by the consulting firm (Bain & Co.) that was used to assist in evaluating the London Life acquisition and the tri-company IT merger potential. The consultants suggested specific dollar reduction targets for technology services (IT operations) and application delivery services (IT development), largely based on norms they had developed from their previous merger and acquisition work. The approval of the Target State Architecture plan and the attainment of the targeted integration cost reductions will be measured. The IT organization integration metric refers to the synergies within the IT organization, e.g., is there one single service desk for the three companies or are there three different ones?

Customer Orientation Scorecard

The Customer orientation perspective evaluates the performance of IT from the viewpoint of internal business users (customers of IT) and, by extension, the customers of the business units. It provides answers to the key questions of these stakeholders concerning IT service quality (cf., Figure 5). As shown in Figure 14, the issues this perspective focuses on are competitive costs, development services performance, operational services performance and customer satisfaction.

Objective

Measures

Benchmarks

Customer satisfaction

  • Business unit survey ratings:

    • Cost transparency and levels
    • Service quality and responsiveness
    • Value of IT advice and support
    • Contribution to business objectives
  • Not applicable

Competitive costs

  • Attainment of unit cost targets
  • Blended labor rates
  • Compass operational 'Top Level Performing' levels
  • Market comparisons

Development services performance

  • Major project success scores

    • Recorded goal attainment
    • Sponsor satisfaction ratings
    • Project governance rating
  • Not applicable

Operational services performance

  • Attainment of targeted service levels
  • Competitor comparisons


Figure 14: Customer Orientation Scorecard

In the Customer satisfaction area, the IT BSC of the merged IT organization is relying on annual interviews with key business managers. It is the intent to set up one generic survey, which can be re-used, with relevant questions that cover the topics mentioned in Figure 14.

Insight into the competitive costs area can demonstrate to the business how cost competitive the IT organization is compared to other (e.g., external) parties. This insight is realized by measuring the attainment of IT unit cost targets and the blended labor rate. This rate model provides an overall single rate for any IT professional who is appointed to the business. The competitive costs measures are benchmarked against Compass's operational 'Top Performing level' and against the offerings of commercial IT service vendors (market comparisons).

Development services performance measures are project-oriented, using attributes such as goal attainment, sponsor satisfaction and project governance (i.e., the way the project is managed). These data are mostly captured by interviews with key managers. The most effective time to establish the basis for these (project) development measures is at the point where business cases are being prepared and projects are evaluated. Each IT project initiative will be evaluated by the IS Executive Committee in which IT and business managers determine — based on the business drivers, budget and state architecture compliance — which projects need to be executed. When a project is approved, the project manager defines clear targets for cost, schedule, quality, scope and governance. The quantitative data (e.g., budget) are reported throughout the lifecycle of the project. After completion of the project, the quantitative and qualitative data are evaluated during the major project review and the main success drivers, delivery issues and lessons learned are documented.

In terms of operational service performance, IT management measures achievement against targeted service levels. For each operational unit (e.g., data center), average response time, service availability and resolution time for incidents are rolled-up to these service performance metrics in the strategic Balanced Scorecard. The results are benchmarked against the performance of competitors.

Operational Excellence Scorecard

The operational excellence scorecard provides the performance of IT from the viewpoint of IT management (process owners and service delivery managers) and the audit and regulatory bodies. The operational excellence perspective copes with the key questions of these stakeholders and provides answers to questions of maturity, productivity and reliability of IT processes (cf., Figure 5). The issues that are of focus here, as displayed in Figure 15, are development process performance, operational process performance, process maturity and enterprise architecture management.

Objective

Measures

Benchmarks

Development process performance

  • Function point measures of:

    • Productivity
    • Quality
    • Delivery rate
  • To be determined

Operational process performance

  • Benchmark based measures of:

    • Productivity
    • Responsiveness
    • Change management effectiveness
    • Incident occurrence levels
  • Selected Compass benchmark studies

Process maturity

  • Assessed level of maturity and compliance in priority processes within:

    • Planning and organization
    • Acquisition and implementation
    • Delivery and support
    • Monitoring
  • To be defined

Enterprise architecture management

  • Major project architecture approval
  • Product acquisition compliance to technology standards
  • 'State of the infrastructure' assessment
  • Not applicable


Figure 15: Operational Excellence Scorecard

In relation to development process performance, function point-based measures of productivity, quality and delivery rate such as number of faults per 100 installed function points and delivery rate of function points per month are defined. Benchmark data on industry performance will be gathered from a third party (e.g., Compass). In the operational process performance area, measures of productivity, responsiveness, change management effectiveness and incident occurrence level are benchmarked against selected Compass studies (e.g., on data centers, client server, etc.).

The process maturity is assessed using the CobiT (Control Objectives for IT and related Technology) framework and maturity models (ITGI, 2000). CobiT identifies 34 IT processes within four different domains (see Figure 15) and describes detailed maturity levels for each of these processes. The Group has identified 15 out the 34 priority processes that should have a maturity assessment in 2003 and the other processes will be measured later.

Enterprise architecture management deals with the IT responsibility to define an enterprise architecture which supports long-term business strategy and objectives and to act as a steward on behalf of business executives to protect the integrity of that architecture. Major project architecture approval measures the compliance of net new systems as they are proposed, developed and implemented. Product acquisition compliance technology standards measure our adherence to detailed technology standards which are at the heart of minimizing technology diversity and maximizing inter-company technology synergies. The "State of the Infrastructure" assessment measures the degree to which IT has been able to maintain a robust and reliable infrastructure as required to deliver effectively to business needs. It does so by comparing each platform area against risk-based criteria for potential impact to business continuity, security and/or compliance.

Future Orientation Perspective

The future orientation perspective shows the performance of IT from the viewpoint of the IT organization itself: process owners, practitioners and support professionals. The future orientation perspective provides answers to stakeholder questions regarding IT's readiness for future challenges (cf., Figure 5). The issues focused on, as depicted in Figure 16, are human resources management, employee satisfaction and Knowledge Management. The metrics that will appear in the future orientation quadrant of the IT strategic Balanced Scorecard are in many cases the aggregated results of measures used in the unit scorecards (e.g., career center).

Objective

Measures

Benchmarks

Human resource management

  • Results against targets:

    • Staff complement by skill type
    • Staff turnover
    • Staff 'billable' ratio
    • Professional development days per staff member
  • Not applicable
  • Market comparison
  • Industry standard
  • Industry standard

Employee satisfaction

  • Employee satisfaction survey scores in:

    • Compensation
    • Work climate
    • Feedback
    • Personal growth
    • Vision and purpose
  • North American technology-dependent companies

Knowledge management

  • Delivery of internal process improvements to 'Cybrary'
  • Implementation of 'lessons learned' sharing process
  • Not applicable
  • Not applicable


Figure 16: Future Orientation Scorecard

Human resource management is an objective that is tracked by comparing measures as described in Figure 16 against predefined targets: the staff complement by skill type (number of people with a certain profile, e.g., systems analyst), staff turnover, staff 'billable' ratio (i.e., hours billed/total hours salary paid; if this ratio can be increased, the IT organization can charge lower rates to the business for the IT assigned people), and professional development days per staff member.

Employee satisfaction is measured by using surveys with questions relating to compensation, work climate, feedback, personal growth, and vision and purpose. Benchmark data of North American technology-dependent companies are provided by a third party.

In the Knowledge Management area, the delivery of internal process improvements to the 'Cybrary' is very important. The 'Cybrary' refers to the intranet that all employees can access for seeking and sharing knowledge. To measure improvements, metrics (e.g., number of hits per day on the Cybrary) still need to be developed. Closely linked to this, Knowledge Management is also measured by the implementation of the 'lessons learned' sharing process. Here too, specific metrics still need to be developed.




Strategies for Information Technology Governance
Strategies for Information Technology Governance
ISBN: 1591402840
EAN: 2147483647
Year: 2002
Pages: 182

Flylib.com © 2008-2020.
If you may any questions please contact us: flylib@qtcs.net