Research Hypothesis

The aims of the research were to discover the difficulties of applying XP in a changing software requirements environment and to use qualitative methods to uncover these difficulties.

This research expects to find that traditional evaluation techniques presently adopted in the analysis of novel software development methodologies such as XP will be improved by qualitative research and analysis techniques. Particularly, this research is moving in the direction of "soft computing" techniques that may be better used to evaluate qualitative data returned, for example, from the complex relationships that exist between the 12 XP practices.

During the process of investigation, the qualitative research uncovers the difficulties of applying XP in an environment of changing software requirements. Successes, failures, and evolving practices are discovered how management and staff adapted during a period of rapid change, and why it was important to have a flexible management approach. The experiences of both management and staff were examined during a period when initial practices matured and new practices emerged.

The research project adopted some of the techniques historically developed from the social sciences [Gittins2000]: ethnography, qualitative interviews, and discourse analyses. An understanding of "grounded theory" was particularly important in this work, because grounded theory can provide help in situations where little is known about a topic or problem area, or to generate new ideas in settings that have become static or stale. Grounded theory was developed by Barney Glaser and Anselm Strauss in the '60s and deals with the generation of theory from data [Glaser+1967]. Researchers start with an area of interest, collect data, and allow relevant ideas to develop. Rigid preconceived ideas are seen to prevent the development of useful research. To capture relevant data, qualitative research techniques are employed [Gittins+2000], which include the immersion of the researcher within the developer environment. The use of guided interviews and the timely rotation of questionnaires was the chosen method of data collection.

Qualitative methods and data evaluation enabled the researcher to study selective issues in detail, without the predetermined constraints of categorized analysis. The researcher was instrumental in gathering data from open-ended questions. Direct quotations are the basic source of raw materials in qualitative research, revealing a respondent's depth of concern. This contrasts with the statistical features of quantitative methods, such as Goal Question Metrics [Basili+1984], recognized by their encumbrance of predetermined procedures.

In terms of qualitative interview procedures, Patton suggests three basic approaches to collecting qualitative data through open-ended interviews [Patton1990]. The three approaches are distinguished by the extent to which the questions are standardized and predetermined, each approach having strengths and weaknesses, depending on the purpose of the interview.

• "Informal conversation" interviews are a spontaneous flow of questions in which the subject may not realize that the questions are being monitored.

• The "general interview guide" approach, adopted extensively for this study, predetermines a set of issues to be explored.

• The "standardized open-ended interview" pursues the subject through a set of fixed questions that may be used on a number of occasions with different subjects.

In a series of interviews, data was collected using informal conversation and verbatim transcripts taken from general guided interviews.

To support data collected from the interviews, an extensive questionnaire (100 questions) was prepared to investigate the rules and practices of XP. Questions targeted the software development process, XP practices, and both managerial and behavioral effectiveness. Behavioral questions were based on Herzberg's "hygiene and motivation factors" [Herzberg1974], and ample provision was made for open comments on each topic. Because of the importance given to the XP working environment, a developer-area floor plan was provided with the questionnaire, giving respondents an opportunity to suggest improvements to the work area. Repeating the questionnaire every three months helped research and management by matching the maturing XP practices against developer responses.