Discussion

The evolution of GEM from a procurement system to a market that fulfils almost all of the criteria presented by both Bytheway and Nokkentved is of some significance in providing a degree of support for the validity of these models. It became clear early in the project that to achieve a real benefit from the implementation of the e-marketplace at the integration level, the transaction level must be satisfied first.

In this section of the chapter, we examine and classify the declared project objectives and benefits listed earlier by DoIT in using e-marketplaces in the government sector.

The established benefits evaluation frameworks are used to structure the examination of benefits. We also highlight a number of issues that are still to be resolved because of the strategy adopted in relation to e-marketplaces with some suggestions for organizations on how to deal with these.

Analysis Within the DeLone and McLean Framework

We did not find it surprising that most of the declared objectives and benefits fit this well accepted model, nor did we find it unusual that many of the benefits postulated by the model were not enunciated in the statement. The list obviously was not intended to be an exhaustive model of the benefits to be gained by the introduction of the GEM system, but specifically to address the perceived benefits from procurement perspective. We believe that each of the aspects of the DeLone and McLean model should be included in the evaluation of the benefits of GEM. The measurement of these constructs from each perspective (supplier, seller and policy maker) could be incorporated into a balanced score card for the long-term evaluation of the project.

What we consider of most interest are the perceived objectives and benefits that do not fit the model, viz:

Objectives

4.	To satisfy government buying and policy compliance.
7.	To support the Open Buying on the Internet (OBI) standard and Australian Procurement and Construction Council (APCC) Guidelines. *
9.	To provide a vehicle for procurement reform. *

Benefits

F.

Open trading model for transacting on a many-to-many basis.

Table 1: Objectives and benefits of electronic marketplaces categorized under MIS success measures (DeLone & McLean, 1992)

System quality	Information quality	Information use
Ease of use, Convenience of access System flexibility, Usefulness of features and functions, response time, Integration of systems Objectives: 6. Integration with FMIS 8. Confidential and secure infrastructure Benefits: None recorded	Current, Timely, Reliable, Useable Complete, Accurate, Free from bias Objectives: None recorded Benefits: A. Reduced time C. Reduced duplicate payments E. Improved mgt. reporting	Purpose of use Recurring use Motivation to use Objectives: End-to-end buying solution Benefits: None recorded

User satisfaction	Individual impact	Organizational impact
Overall satisfaction Information satisfaction Full information required Enjoyment Decision-making satisfaction Objectives: None recorded Benefits: E. Improved mgt. reporting	Problem identification, information awareness and decision effectiveness through provision Analytical capabilities for business analysis Market intelligence Forecasting Greater visibility of supply chain Project management collaboration Information sharing Interorganization connections Instant communications Mediated business processes Objectives: 2. Supplier maintained catalogues 5. Buyer-to-supplier efficiencies (through automated communication technologies) Benefits: G. Added value to supply chain management H. Supplier maintained catalogues	Operating cost reductions. Objectives: 6. Integration with FMIS Benefits: Reduced time Reduced real cost Overall productivity gains. Staff reductions Benefits D. Minimize data collection H. Supplier maintained catalogues Contribution to achieving goals through improved management of data and 'near perfect' information flow. Objectives: 3. Work flow management 5. Buyer-to-supplier efficiencies

Conceptually, these items may be grouped as extra-organizational impact (within the public sector, we would suggest the title of impact on society). This reflects the significant swing in emphasis from the internal focus of information technology in the early 1990s to a much wider approach a decade later as the introduction of technology and the consequent changes in process cross organizational boundaries. This seventh dimension may be particularly important for the evaluation and measurement of large IT projects in the public sector as they impact on the society which they serve in a way which may be separate from the public sector outcomes which they are initially designed to support.

It is interesting to see that Objective 4: "To satisfy government buying and policy compliance," might be the subject some considerable debate in terms of its impact on society. On the one hand, it may be argued that compliance with government policy is of benefit to society, allowing suppliers equal access to the public purse. However, as we have described earlier, this could also be interpreted as ensuring strict compliance with central purchasing contracts negotiated with a select few suppliers, excluding others form the market.

Analysis Within the Bytheway Evaluation Framework

In this case we have been able to categorize all of the published, perceived objectives and benefits within the framework. One category, the "defensive" is hardly used and this can be attributed to the dominant role of the government within the market of Western Australia. The framework is not as detailed as that proposed by DeLone and McLean and is not a substitute, yet they appear to be complementary, allowing for the categorization along different important dimensions.

• Efficiency: doing things right

  • Objectives:

    1.	End-to-end buying solution,
    2.	Supplier maintained catalogues.
    5.	Buyer-to-supplier efficiencies

  • Benefits:

    A.	Reduced time
    B.	Reduced real cost
    D.	Minimize data collection
    H.	Supplier Maintained Catalogues

• Effectiveness: doing the right thing

  • Objectives:

    6.	Integration with FMIS
    8.	Confidential and Secure

  • Benefits: A, C & G

    A.	Reduced time
    C.	Reduced Duplicate payments
    G.	Added value to supply chain management

• Evolution: doing things different

  • Objectives:

    3.	Work flow management
    7.	OBI/APCC

  • Benefits:

    E.

    Improved mgt. reporting

  • Defensive: reacting to moves made by competitors

  • Objectives:

    7.

    OBI/APCC (possibly)

  • Benefits:

    • None recorded

• Experimental: trying to understand you opportunities

  • Objectives:

    4.	Policy compliance
    7.	OBI/APCC
    9.	Procurement reform

  • Benefits:

    F.

    Open trading model

Limitations of Benefits Evaluation Frameworks

This study shows that the DeLone and McLean framework is robust from an internal perspective, but is incomplete in that this internal focus is a limitation where the effect of the introduction of a new IT system is such that it is designed to have an impact on the whole society in which it operates. This can be addressed by the extra dimension proposed as "impact on society," with the development of measurable constructs to reflect this dimension required as a matter of urgency.

The extended Bytheway framework proved robust within its limitations, and the case study provides evidence to support a significant investment by the Western Australian government on an experimental basis. It must be noted that as this framework addresses mainly organizational benefits, it does not stand alone as well as the extended DeLone and McLean framework. It assumes the realization of other benefits such as the technical integration in order to obtain the organizational benefits.

The Bytheway framework is organizational and should be used with a traditional framework such as DeLone and Mclean's, hence the constructs are sufficiently broad to be applied to benefits accruing to the community in general as well as the parties to the transaction.

Again, the Bytheway framework does not address the issues of measurement and evaluation of benefit when it has a community focus. These too, require urgent attention.

This case study and analysis highlights several issues of importance:

Accounting Information Systems texts generally (Bodnar & Hopwood, 1998; Gelinas Jr., Sutton, & Oram, 1999; Hall, 1998; Hollander, Denna, & Cherrington, 2000; Romney & Steinbart, 2000; Wilkinson & Cerullo, 1997) and established frameworks for e-marketplaces address the corporate AIS needs in the context of the major business cycles:

• Revenue and collection

• Acquisition and payment

This case study clearly shows that the implementation of a system to achieve only a portion of a cycle was not acceptable to the users of the system and that the proponents of the system modified the scope to meet this requirement. In planning IT initiatives, proponents need to address integration issues early in the process.

The case also provides strong evidence that the implementation of e-marketplaces by the Western Australian government has objectives and perceived benefits which fall outside of the traditional, internally focussed IT benefits framework dating from the early 1990s. A seventh dimension to the DeLone and Mclean framework "Impact on Society" is proposed to address this issue.

It is clear from the case that the implementation of e-procurement systems is still largely at an experimental stage and that the concept of running parallel systems within different organizational units is still a useful tool when technology is either unproven or immature.