Methodology
|
The Xavier/TDG Project Team performed a thorough literature review of English language periodicals, doctorate dissertations, and publicly available government reports from 1960 through 1999. We also included PMI Seminars & Symposium papers and easily available masters theses. A total of ninety-two people worked approximately 6,000 hours on this research project between September 1999 and April 2000. The project team is listed in Table 3. Timothy J. Kloppenborg, Ph.D., PMP, served as the principal investigator. Tim is an Associate Professor of Management at Williams College of Business, Xavier University. He led the effort, coordinated with all stakeholders, recruited the team, and was involved in making all major decisions. Warren A. Opfer, MBA, PMP, served as the co-principal researcher. Warren is a principal with The Dayton Group. He led the research on government sources, led much of the technical effort, and was involved in making many of the decisions. James M. Gallagher, PMP, served as contributing investigator. He is President of The Dayton Group. Jim recruited Warren and Tim, was instrumental in securing and organizing this research effort, and was involved in making many decisions. John Stemmer, MLS, is Xavier's electronic librarian. John was involved in making many of the technical decisions and led the day-to-day efforts of the students throughout the investigation. Rachana Sampat, MBA, was a graduate student at Xavier. Rachana was the principal assistant on all phases of the research and was involved in making many decisions.
 |
| Principal Investigator: | Timothy J. Kloppenborg, Ph.D., PMP | |
| Co-Principal Investigator: | Warren A. Opfer, MBA, CDP | |
| Contributing Investigator: | James M. Gallagher, PMP | |
| Electronic Librarian: | John Stemmer | |
| Principal Graduate Assistant: | Rachana Sampat | |
| Other Student Assistants: | ||
| Luis Barriga | Matthew O'Connor | |
| Julie Butz | Megera O'Donnell | |
| Christopher Debrunner | Priya Patel | |
| Kimberly Dulin | Michael Sigg | |
| Michelle Groetzinger | Greg Walker | |
| James Krallman | ||
| Knowledge Area Facilitators: | ||
| Bycio, Peter | XU, Management | Integration |
| Cagle, Julie | XU, Finance | Risk |
| Finch, Miriam | XU, Communication Arts | Communications |
| Clark, Thomas | XU, Management | Communications |
| Cunningham, Margaret | XU, Management | Quality |
| Petrick, Joseph | WSU, Management | Human Resources |
| Shami, Manar | UC, Construction Science | Scheduling |
| Surdick, John | XU, Accounting | Cost |
| Tadepalli, Raghu | XU, Marketing | Procurement |
| Tesch, Deborah | XU, Information Systems | Scope |
| Workshop Arrangements: | ||
| Debra Miersma, PMP | ||
| Lynn Mills | ||
| Alan Dulin | ||
| GroupWare Consultants: | ||
| Teresa Frank, PMP | ||
| Patrick Sepate, PMP | ||
| Ryan Belt | ||
| Workshop Volunteers: 51 Senior Project Executives (mostly PMPs) | ||
| Other Database Workers: | ||
| Cathy Heuing | ||
| Chris Schenck | ||
| Laura Opfer | ||
| Jennifer Voelker, PMP | ||
| Thomas Youger, PMP | ||
| PMI Research Conference 2000 Project Team: | ||
| Lewis Gedansky, Ph.D. | ||
| David I. Cleland, Ph.D. | ||
| Jeffrey Pinto, Ph.D. | ||
| Dennis Slevin, Ph.D. | ||
|
The deliverables included an annotated database of project management research and a final written report, resulting in the research conference paper and this chapter. This chapter includes trends found in the research, reasons for the trends," "potential best article candidates", and predictions for the future. The tasks accomplished on this project can be described in the following twelve areas:
-
Determine what project management research is.
-
Identify sources of data.
-
Determine formats for our outputs.
-
Acquire data.
-
Input data.
-
Identify records to be retained.
-
Plan and conduct the workshop.
-
Identify and interpret trends.
-
Identify "potential best article candidates."
-
Predict future directions.
-
Advise the French language team.
-
Construct the final database.
Determine What Project Management Research Is
Our first critical task was to define project management research. After much coordination and discussion among the Xavier/TDG Project Team and the PMI Research Conference 2000 Project Team, the following definition was agreed upon:
Project Management Research Definition: For the purposes of this study, project management research is defined to include published works that are based upon data (either primary or secondary) and that make generalizable conclusions drawn from the data where the data and conclusions are focused on either the project management context or the management activities (not the technical activities) needed to complete a project successfully; where:
-
Published means that the works are in public distribution or through publicly available government sources (i.e., DoD, NASA, and such).
-
Data means information organized for analysis.
-
Generalizable means able to draw inferences beyond the individual case.
-
Conclusions means judgments or decisions made after deliberation.
-
Project management context means the environment in which projects/programs operate.
-
Management activities mean all the processes used to manage the nine project management knowledge areas as described in A Guide to the Project Management Body of Knowledge (PMBOK Guide) and other information dealing with project management knowledge, skills, and attitudes.
-
Technical activities mean project specific tasks such as writing code, digging a foundation, and so on.
Research can include both theory and practice in the field of project/program management. Examples include:
-
Summaries and critical analyses of research results.
-
Surveys of current practices.
-
Critical analyses of concepts, theories, or practices.
-
Developments of concepts, theories, or practices.
-
Analyses of successes and failures.
-
Comparisons and/or analyses of case studies.
Identify Sources of Data
The second task was to identify the useful sources of data. We explored various commercial databases and selected those shown in Table 4 because collectively they covered the various disciplines (management, engineering, computers, and so forth) that may be interested in project management, covered many journals both from North America and elsewhere in the world, and covered all doctoral dissertations.
|
| ABI Inform (1971–present) |
| Provides coverage of business, management and related topics. |
| Includes over 1,000 leading business and management publications. |
| Includes over 350 English language titles from non-US sources. |
| Compendex (1980–present) |
| A comprehensive interdisciplinary, engineering database. |
| Covers engineering, manufacturing, quality control, and engineering management issues. |
| Infotrac General Business File (1980–present) |
| Includes business and management topics in over 900 journals. |
| Includes information on finance, trade, new technologies, trends, etc. |
| Digital Dissertations (1861–present) |
| Dissertations from over 1,000 North American graduate schools and European Universities. |
| Post 1980 includes abstract written by author. |
|
We also manually researched selected journals by either paper or microfilm, both to ensure we covered the time period before the electronic databases started and to include specific journals we felt were important but were not included in the electronic databases. These selected journals are shown in Table 5.
|
|
|
We initiated contact with various other professional associations to inquire about alternative sources of project management research. While this effort resulted in interesting phone calls and emails, the sources identified were already included. Examples of these organizations are Institute of Industrial Engineers, Academy of Management, Institute of Electrical and Electronic Engineers, and Construction Industry Institute.
We also initiated contacts with various government agencies to locate publicly accessible government reports that qualified as project management research. These sources are listed in Table 6.
|
| 1. US Air Force Institute of Technology | AFIT |
| 2. Scientific and Technical Information Network | STINET |
| 3. Defense Technical Information Center | DTIC |
| 4. US Defense Systems Management College | DSMC |
| 5. Research on Ongoing Acquisition Research (in DSMC) | ROAR |
| 6. Canadian Aeronautics and Space Institute | CASI |
| 7. US National Aeronautics and Space Administration | NASA |
| 8. Dudley Knox Library—Naval Postgraduate School (NPS) | DUDLEY |
| 9. The Rand Corporation | RAND |
|
Determine Formats for Our Outputs
The third major task was to determine the reporting formats for the project working documents and deliverables. These needed to be decided early so we did not lose time redoing work. Many of our project tasks were on the critical path and the ones that were not had very little slack time. We needed to select bibliographic software and design a database quickly. For this a product called Reference Manager (ISI ResearchSoft; previously Research Information Systems) was selected, as it appeared to have all the necessary features that are required to create a comprehensive database with the added advantage of allowing the user to create different databases based on various criteria. It also provided the option of exporting the database records to Microsoft Access for ease of use and portability.
The database design we developed is included as Table 7. We determined that we would use the abstract from any entry that had one, regardless of the length or format. These vary widely since they come from thousands of primary sources. We also decided that any abstracts we would write would be simple, descriptive, and approximately five lines in length. An example of a citation in the database format is included as Table 8.
|
| Reference Type: Generic |
| Reference ID: 1 |
| Authors Primary: Author Name |
| Title Primary: Sample Record |
| Author Secondary: if needed |
| Title Secondary: if needed |
| Periodical: Periodical Name |
| Volume: Volume Number |
| Issue: [Issue Number] |
| Starting Page: 1 |
| Ending Page: 999 |
| Date Primary: 12-3-1999 |
| Publication Place: City, State, and Country of publication (optional) |
| Publisher: Name |
| Title Series: Optional |
| Authors Series: Optional |
| Keywords: These are from the Citation |
| Reprint: Not in File |
| Abstract: From the resource database or written by us (initial ones we write) |
| Notes: As you need them. Add editors/reviewers comments here also |
| Knowledge Area Keywords: Knowledge Area Keyword |
| Application Area Keywords: Application Area Keyword |
| Process Area Keywords: Process Area Keyword |
| User Defined Field #4: Not Used |
| Status Codes: Status Field (blank/E-<initials>/?[flag for review]/R-<initials>) {entries are separated by;} |
| ISSN/ISBN: from the resource database |
| Availability: from the resource database |
| Source Code: resource database source name |
| Misc. Field #2: not used |
| Misc. Field #3: not used |
| Address: from the resource database—optional |
| Web URLs: from the resource database or entered by us (can be used to link to full text files stored on hard drive) |
|
|
| Reference Type: Generic |
| Reference ID: 26 |
| Authors Primary: Baumgardner, C.R. |
| Title Primary: An Examination of the Perceived Importance of Technical Competence in Acquisition Project Management |
| Author Secondary: AIR FORCE INST OF TECH WRIGHT |
| Starting Page: 243 PAGES Report Number: AFIT/GSM/LSY/91S-4 |
| Date Primary: 9-1-1991 |
| Keywords: *Program manager/*PROJECT MANAGEMENT/ACQUISITION/Acquisition managers/ADDITION/ADMINISTRATION/Attributes of project managers/DETERMINATION/LIFE CYCLES/LIMITATIONS/MANAGEMENT/Management skills/PHASE/Project management/PROJECT MANAGER/Project managers/RATES/SCHOOLS/SKILLS/TEAMS (PERSONNEL)/Technical competence/THESES/TRAINING/TRAINING |
| Reprint: Not in File |
| Abstract: This research examined the importance Air Force acquisition project managers attribute to technical competence, and several individual and situational factors which might affect their perceptions of the importance of technical competence: extent of acquisition experience, degree of technical academic training, level of project technology, caliber of technical project team, and acquisition phase. A review of the literature revealed that previous studies of project and program managers suffered from several limitations with regard to a direct examination of the importance of technical competence. No single study has empirically supported a clear determination on the importance of this attribute to project managers. Considering the nature of today's projects and the ongoing dramatic rate of technological advancement and change, an investigation of the potential importance of technical competence in managing projects seems particularly critical for prospective project managers and those with little experience. In a study of 228 acquisition managers, the findings show a majority consider technical competence as extremely important or absolutely essential, even though they may rank the attribute low in terms of use when compared to other skills. Further, the results indicate the importance of technical competence among acquisition project managers varies significantly as a function of technical project team capabilities and phase of the acquisition life cycle. In addition, the research suggests technical competence is an important contributor to project managers' ability to communicate. |
| Notes: AD Number: ADA246682 |
| Descriptive Note: Master's thesis |
| Monitor Acronym: XF |
| Monitor Series: AFIT |
| Imitation Code: APPROVED FOR PUBLIC RELEASE |
| Source Code: 012250 |
| Knowledge Area Keywords: 5;6;7;9 |
| Application Area Keywords: 9 |
| Process Area Keywords: 1;14 |
| Status Codes: E-LMO; R-WAO |
| Source Code: DTIC |
|
An analysis and research taxonomy based on the PMBOK Guide nine knowledge areas was developed. This provided the basis for the development of the search keywords and facilitated the categorizing and cataloging of the research, allowed for the verification of observed trends in the previous research, and assisted in the identification of major research opportunities for advancing the body of knowledge in project management.
We identified three sets of keywords that would be used as the taxonomy and would be used in the review, classification, and cataloging of the citations. These keyword sets are summarized below and described in Table 9.
|
| Knowledge Areas:
|
| Application Areas:
|
| Process Areas:
|
|
-
The first set of keywords relates to the nine knowledge areas of project management as defined in the PMBOK Guide. It has been useful in identifying citations relating to the different aspects of project management.
-
The second set of keywords relates to the application of project management to different industries.
-
The third set of keywords relates to the process aspect of project management. Fifteen process areas, including project life-cycle stages such as plan and close, as well as management processes, such as organize and motivate, have been established.
Acquire Data
The fourth major task was to actually acquire the data. This included selecting search terms, searching electronic databases, and downloading files. Our selection criteria for searching the online databases were based on the list shown in Table 10.
|
|
|
The commercial databases were downloaded en masse. Ensuring that individual citations met the research definition occurred later. Reports in the government databases were verified for inclusion first and then downloaded. Many entries from both traditional and government sources were only available in either paper or microfilm format. These needed to be copied individually. Some entries, such as PMI Seminars & Symposium papers, were available on CD-ROM.
Input Data
The fifth major task was to input the data into Reference Manager. To accomplish this we first had to write a unique filter for each electronic database from which we wanted to import files so they would be compatible in Reference Manager. Even with these filters, some files needed to be manually manipulated for correct format. We then needed to develop Reference Manager procedures and train everyone who used it to maintain consistency. We wrote abstracts when necessary and typed entries that came from old paper and microfilm sources. At this point, we imported files into Reference Manager.
Identify Records to be Retained
The sixth major task was to apply the research definition to all of the citations and eliminate those that did not meet the criteria. Due to the volume of citations, that was a very substantial task. The first step in this was to remove duplicates from the imported records. As records often appeared in more than one source, duplicate citations could be deleted. Since records were identified through queries of commercial databases, there were often citations and papers included that did not apply. When counting all sources, over 100,000 total entries were considered for inclusion. Very quickly it became apparent that it would be impossible to locate all 19,000 books that were identified using our keywords. Therefore, the assumption was made that most research that is included in books has also appeared in an article or paper somewhere, and books were deleted. We noticed certain publications were not research oriented and, as such, we deleted them en masse electronically. A list of these publications appears in Table 11.
|
| Advertising Age | Marketing |
| American Medical News | Marketing News |
| Automotive News | New York Times |
| Business Insurance | PC Week |
| Business Marketing | PC World |
| Computer Technology Review | PC Computing |
| Computer World | PC Magazine |
| Financial Times | Publishers Weekly |
| InfoWorld | Wall Street Journal |
| Other items were deleted because of a readily identifiable indication that they were not research such as: | |
|
|
|
Other items were deleted because of a readily identifiable indication that they were not research. Examples of these indications are also contained in Table 11.
Based on further review of the materials kept, additional items that were quickly deleted included authorless citations, abstractless citations, certain proceedings, and systems and computer type journals.
Finally, the primary method of deleting unneeded records was to manually look at the abstract of every single entry and make decisions on whether it met the definition of project management research stated above and whether it was really on topic. Many, many articles, papers, and government reports were deleted because they did not qualify as research. The most common reason was that the entry was based on opinion instead of data. More than half of the identified dissertations failed the test of being on topic. Most of these included the term project management in their abstract to describe the efforts of their dissertation as a project such as "this butterfly-collecting project." While they are research, they are certainly not project management.
Plan and Conduct the Workshop
The seventh major task was to plan and conduct a workshop so we could utilize the collective judgment of nine additional professors and the experience of over fifty project management practitioners to assist in identifying and interpreting the trends and identifying areas for future research. Each professor acted as a facilitator and was an expert in one of the nine knowledge areas; however, not all were experts in project management. These professors were from Xavier University, Wright State University, and the University of Cincinnati. These professors are listed in Table 3.
We assembled a book for each workshop participant that included an agenda, a table of contents, a summary of the trends uncovered in each PMBOK Guide knowledge area, a list of approximately twenty-five "preliminary best article candidates" from each PMBOK Guide knowledge area, and a special section on government reports. We arranged for a computer network with groupware software to support the workshop. This included loading a great deal of information into the groupware, conducting a technical setup with the software consultants, and conducting a dry run with the facilitators and online scribes.
The actual workshop started with several people representing PMI setting the stage. Then the workshop facilitators of each group quickly overviewed the trends discovered in the nine PMBOK Guide knowledge areas, as well as in the government reports. The practitioners were divided into the nine PMBOK Guide knowledge areas based on their specialization and preferences. The professors facilitated breakout sessions for each of the nine PMBOK Guide knowledge areas asking the practitioners to comment on why they felt each identified trend in research developed the way it did. The next activity in the workshop was for the practitioners and professors to vote on the "top ten" articles in their respective knowledge area. That was followed by a prediction within each knowledge area of research trends likely to be seen in the future. The practitioners were then challenged to identify what potential research they would find most useful in the future in each knowledge area. Finally, all workshop participants identified best practices they would like to see repeated in future workshops and lessons learned to improve future workshops. The groupware downloaded workshop results into several documents totaling fifty-five pages of text. Since this was so voluminous, each professor wrote a two to three page summary of the results of his respective knowledge area.
Identify and Interpret Trends
The eighth task was to identify and interpret trends in the research. To accomplish this, each of the professors was furnished with their respective portion of the working database (this ranged from several hundred to several thousand citations), the results of thirty-six simple queries (each of the knowledge area, application area, and process area keywords defined above), and a student to assist them with more complex queries.
Once the professor identified the citations of interest, she reviewed their abstracts to identify trends. We then imported these trends into the groupware software for the workshop. The professor briefly described the trends to all workshop participants, and then asked the experts in her area detailed questions concerning the trends. The answers to these questions were captured electronically. Finally, when the workshop was over the professors each wrote a summary of the results of their respective areas. We then compared these results and have included an overall summary in the results section of this paper. The working database at that point in time did not include PMI Seminars & Symposium Papers and a few articles prior to 1980 that were manually added later. This working database did include some articles that were later deleted because they did not satisfy our research definition.
Identify "Preliminary Best Article Candidates"
The "best article candidates" list made up the ninth task. Each professor also identified (just from abstracts) approximately twenty-five citations that they thought had important research findings and/or ideas. The citation and abstract of each of these was printed and included in the workshop book. The professors used these abstracts to facilitate discussion aimed at interpreting the identified trends. After a morning of discussion on the trends identified and some time to review the abstracts, the participants voted on up to ten citations that they thought made the most meaningful contribution to their personal understanding. These lists of "preliminary best article candidates" are listed in the results section of this chapter. They are not meant to be an authoritative list. They are meant to generate discussion among project management researchers concerning what is important project management research.
Predict Future Directions
The tenth task was to predict future directions for project management research. The professors and practitioners did this for each knowledge area toward the end of the workshop. One interesting note is that while there is some overlap between what the practitioners predict will happen and hope will happen, there are many differences. A summary of both of these lists is included in the results section. We believe researchers should make careful note of the areas in which these senior project management experts say research is needed.
Advise the French Language Team
The eleventh task was to provide guidance to the French language team to ensure that both efforts would yield results that could be consistently compared. The English language team started first and had many more records to consider. As we developed procedures, we wanted to ensure the French team knew what we were doing.
Construct the Final Database
The twelfth, and final task, was to construct the final database. For this, we constructed individual, "clean" databases of journal citations from electronic database sources, PMI Seminars & Symposium papers, dissertations, government reports, and a miscellaneous database of journal citations that came from paper and microfilm. These were then combined and one final check for duplicated citations was performed.
|
EAN: 2147483647
Pages: 207
- An Emerging Strategy for E-Business IT Governance
- A View on Knowledge Management: Utilizing a Balanced Scorecard Methodology for Analyzing Knowledge Metrics
- Measuring ROI in E-Commerce Applications: Analysis to Action
- Technical Issues Related to IT Governance Tactics: Product Metrics, Measurements and Process Control
- Governance Structures for IT in the Health Care Industry