Methodology

---

Data Collection and Analysis

A three-round, non-anonymous Delphi-type open survey was conducted, using personalized e-mail with attached survey instruments. Chang and Gable (2000) critique the Delphi method in the context of IS key issues studies and its application within the context of the current study. The study involved three survey rounds: (1) "inventory" round, (2) "confirmation" round, and (3) "weights" round. Round-One sought to inventory the morass of issues perceived by the contacts. Subsequent to Round-One a "tentative set" of issue categories was synthesized. In Round-Two, a "preliminary set" was confirmed with respondents. Having established a "master set," in Round-Three respondents were asked to score or weight the issues in the master set, indicating their perceived relative importance.

In coding and synthesizing the survey responses from Round-One, several potential coding schemes were examined and tested. Attempts to map the data onto existing models (e.g., MIT90s framework, ERP life cycle) failed to provide a satisfactory level of discrimination between substantive issues. The strengths and weaknesses of potential coding methods and synthesis procedures are discussed in Chang et al. (2000). Qualitative data analysis techniques (Gadamer, 1977, 1985; Husserl, 1985; Lacity and Janson, 1994; Ramm, 1970; Tesch, 1991; Winograd and Flores, 1986) also served as a guide to coding and synthesis (e.g., how to deal with a large amount of non-numerical, unstructured, and rich data; how to ensure that when synthesized, those issues accurately reflect the respondents' concerns) that confront Delphi method researchers. Ultimately, an open coding approach was adopted to structure the issues identified in Round-One. The major strength of the open coding approach is that it is data driven-the categories so formed reflect the range of issues that were collected, rather than some pre-defined scheme. Because the categories are determined from the data themselves, respondents should comprehend them more readily in subsequent survey rounds.

To support the interpretation of study findings, an understanding of the contextual background of the Queensland budget sector and the study organizations in relation to their SAP Financials project was essential (i.e., organizational nature/background, major services/roles/responsibilities of the agency, history/initiatives of the financial management system, overview of agencies' SAP project). Thus, a series of interviews and domain experts' workshops that involved senior staff members from the respondent groups were conducted before, during, and after the Delphi survey rounds.

Study Population

During 1998 and early 1999, the study case (a group of five government agencies) proactively moved as a team and implemented the SAP Financials. Individuals from the implementation partner (IP), a "big 5" Consulting Firm, and these five closely related government client agencies (agency A to E) were pre-identified and contacted for study participation. To qualify for study participation, they were required to possess substantial involvement with SAP Financials: at any level, in any role, in any phase of the life cycle, with any of the modules implemented. Employing formal "Survey Participants' Selection Guidelines," and through interviews of senior sponsors in each agency, 117 individuals were identified and included in the contact database. Note that the term "client" herein refers to employees of the agencies, who are "clients" of both the ERP vendor and the implementation partner. Owing to the full support of the Queensland Government in this study and to the assistance of key contacts in each organization, the 117 contacts selected approximate the 'population' of knowledgeable individuals (rather than a "sample").

---