Evaluating IS Efficiency and Effectiveness of Information Systems in Supporting Business Processes

The evaluation of the efficiency and effectiveness of an organization's IT program involves reviewing the IT governance structure as well as its alignment with the organization's strategy. The IT organization must also manage the risks associated with ongoing development and operations. The IT organization should have a risk-management program that utilizes internal controls and best practices to mitigate risks to an acceptable level. As a part of risk management, the IT organization should have formal documented methodologies for managing business process change to include organization, management, controls, and measurement. The IS auditor should ensure that IT is aligned with corporate goals and that the benefit of IT is maximized while risk is minimized.

Methods and Approaches for Designing and Improving Business Procedures

The standard approach to improving business processes is to identify specific areas to be reviewed, document the existing baseline process(s), and identify areas for improvement. After improvement areas have been identified, they should be presented to senior management for prioritization and implementation. Upon implementation of the business processes, the organization should monitor the new processes against the baseline and establish a continuous improvement process. Known as business process re-engineering (BPR), this usually successfully reduces manual interventions and controls within the organization.

Benchmarking

ISACA defines benchmarking as the continuous, systematic process of evaluating the products, services, and work processes of organizations, recognized as representing best practices for the purpose of organizational improvement. The purpose of identifying a benchmarking partner is to find a work process in your industry that is identified as having the qualities that your organization would like to re-engineer to (success, quality, excellence, and so on). ISACA outlines the following steps in a benchmarking exercise:

1. Plan. In the planning stage, critical processes are identified for the benchmarking exercise. The benchmarking team should identify the critical processes and understand how they are measured, what kind of data is needed, and how that data needs to be collected.

2. Research. The team should collect baseline data about its own processes before collecting this data about others. The next step is to identify the benchmarking partners through sources such as business newspapers and magazines, quality award winners, and trade journals.

3. Observe. The next step is to collect data and visit the benchmarking partner. There should be an agreement with the partner organization, a data-collection plan, and a method to facilitate proper observation.

4. Analyze. This step involves summarizing and interpreting the data collected, analyzing the gaps between an organization's process and its partner's process, and converting key findings into new operational goals.

5. Adapt. Adapting the results of benchmarking can be the most difficult step. In this step, the team needs to translate the findings into a few core principles and work down from the principles to strategies and action plans.

6. Improve. Continuous improvement is the key focus in a benchmarking exercise. Benchmarking links each process in an organization with an improvement strategy and organizational goals.

Benchmarking partners are identified in the research stage of the benchmarking process.

The IS auditor must ensure that the change efforts are consistent with the culture and strategic plan of the organization, and that the change efforts reduce negative impact on the organization's staff. In addition, the auditor must ensure that key controls, if required, are engineered into the new process. If key controls are removed as a part of the re-engineering effort, the IS auditor must ensure that all risks associated with these controls are communicated to and accepted by management.

This benchmarking methodology assumes that organizations will be able to find partner organizations that will agree to review and observation. In today's competitive market, most organizations turn to professional consulting companies that have performed business process re-engineering across industries and use the information gathered during those engagements to compare to their organization.

Business Process Re-engineering (BPR)

In today's competitive landscape, the continuous improvement of business processes no longer ensures an organization's survival. Business change is primarily driven by customer needs for new and improved products and services. If an organization cannot provide these products and services, customers have the option of turning to other organizations that can provide these products and services. Business process re-engineering (BPR) provides an accelerated means of process improvement by assuming that existing business processes do not work; therefore, the re-engineering effort can focus on a new processes by defining a future state (to be).

After the future state has been defined, the re-engineering team can create an action plan based on the gap between current processes and the future state. The re-engineering team and management then can create the transition plan and begin to implement the changes. To help ensure the success of the re-engineering effort, determining the scope of areas to be reviewed should be the first step in the business process re-engineering project. In defining specific areas for improvement, the organization can ensure that the effort focuses on value and customer requirements.

As organizations work to drive time and cost out of business processes, they often turn to technology as a solution. The advent of new technologies such as the Internet has allowed organizations to rapidly bring new capabilities that dramatically improve business processes. The availability of new technologies and the drive for rapid implementation could put the organization at risk by driving key controls out of improved business processes and lacking key controls in new business processes. An IS auditor should always make sure that a re-engineered business process has not inadvertently removed key controls from the previous control environment.

Whenever business processes have been re-engineered, the IS auditor should attempt to identify and quantify the impact of any controls that have been removed, or controls that might not work as effectively after a business process changes.

The implementation of BPR affects the culture, structure, and direction of the organization. Generally, the largest impact of re-engineering is on the staff. The organization should have a change-management process and teams that can evaluate possible issues or problems that might arise and that can provide solutions. The change-management team should monitor the re-engineering process to ensure that it is meeting the strategic plan and goals of the organization. As the re-engineering is implemented, the organization should see improvements in products, services, and profitability. The proper implementation of technology should reduce manual intervention and controls, producing an accelerated production and delivery of products and services.

Business process re-engineering often results in increased automation, which results in a greater number of people using technology.

A couple emerging business and technology trends illustrate these improvements. The first is customer relationship management (CRM), which focuses on managing detailed customer information. This might include previous transactions and customer requirements, allowing organizations to match customer needs to products and services. A CRM system usually integrates a database, web technologies, telephony, accounting, and fulfillment systems. This integration enables organizations to capture transaction data, customer preferences, order status, and demographic information. This gives an organization a complete view of its customers across all business units and product lines, and enables it to proactively identify which products or services the customer might need.

The second, supply chain management (SCM), is the improvement of an organization's product and service design, purchasing, invoicing, distribution, and customer service. The implementation of SCM involves streamlining the supply chain through the collaboration of entities in real time and the realization of just-in-time (JIT) delivery. JIT delivery reduces the overall cycle time associated with manufacture and inventory by creating products and services based on customer demand.

One of the technologies associated with SCM is the process of electronic funds transfer (EFT). EFT is an electronic payment process between buyers and sellers that is very efficient because it reduces paper transactions and manual intervention.

EFT systems are more efficient than traditional paper checks for accounts payable disbursements.

Business Performance Indicators

After an organization has developed a strategic plan and defined its goals, it must measure its progress toward these goals. Key performance indicators (KPI) are quantifiable measurements that are developed and accepted by senior management. Key performance indicators vary by organization but are created as long-term measurements of an organization's operational activities against its goals. The organization uses quantifiable measurements that ensure the measurement of expected outcomes as opposed to activities. As an example of a goal, the IT organization would expect to deliver services in accordance with service-level agreements (SLA). The IT organization would measure actual service levels against the SLA, identify gaps, and define controls to proactively reduce the service-level failures to meet the SLA.

Some organizations tend to measure things that are easy to measure instead of those that are critical to the organization meeting its goals. These types of measurements might include the number of events but not the expected outcome from the events. To ensure that KPIs are understandable and do not detract from the organization's mission, they should be kept to a minimum of three to five. The use of KPIs provides management with a compass that allows for course corrections in meeting organizational goals and a communication tool for the entire organization defining the importance of achieving these goals.

Another way to measure organizational performance is the balanced scorecard. The balanced scorecard is a management tool that clarifies an organization's goals, and defines actions and the measurement of those actions to meet goals. The balanced scorecard differs from previous methodologies, in that it combines measurement of all business processes. This allows managers to see the organization from many different perspectives and identify areas for improvement. The balanced scorecard incorporates measurements of financial performance, customer satisfaction, business processes, and the capability to improve business processes. ISACA defines the application of the balanced scorecard to IT as a three-layered structure that addresses the four perspectives through the following.

Mission:

• To be a preferred supplier of information systems

• To deliver effective and efficient applications and services

• To obtain reasonable business contribution of IT investments

• To develop opportunities to answer future challenges

Strategies:

• Use preferred suppliers of application and operations

• Foster user partnerships and greater customer service

• Pursue efficient and economical developments and operations

• Control IT objectives

• Provide business value to IT projects

• Provide new business capabilities

• Train and educate IT staff, and promote excellence

• Provide support for research and development

Measures:

• Provide a balanced set of metrics to guide business-oriented IT decisions

Table 7.1 integrates the ISACA example and shows some possible measures associated with a balanced scorecard.

Table 7.1. Balanced Scorecard Perspectives, Objectives, and Measures

Perspective	Objective	Measure
Customer	• Use preferred suppliers of information systems, applications, and operations	Client/customer satisfaction Service-level agreements met
	• Foster user partnerships and greater customer service	Systems, applications, and operations controls meet control objectives
Financial	• Deliver efficient and effective applications and services	Total operational and development costs
	• Obtain reasonable business contribution of IT investments	Commitments met, project estimates
	• Pursue efficient and economical development and operations	Requirements met, post-implementation review
Internal business processes	• Provide support for research and development • Control IT Objectives • Provide business value to IT projects • Develop opportunities to answer future challenges	Best practices adopted New opportunities realized IT strategic plan aligned with business strategy
Growth and learning	• Provide new business capabilities • Train and educate IT personnel, and promote excellence • Provide a balanced set of metrics to guide business-oriented IT decisions	Employees in developmental assignments Employee satisfaction Core competency profile IT mission and strategy clearly defined