2.3 Self-fulfilling prophecies


2.3 Self-fulfilling prophecies

The multiproject system described above displays the characteristics of a complex system in the following respects:

  • High coupling: Any intervention is likely to affect something somewhere else.

  • Time lagging and nonlinear responses: The results of an intervention take time to materialize and when they do, they do not materialize at a constant rate of progress.

  • The true state of the system is unknowable: The current state of the system can only be inferred, its most likely evolution only guessed at. The presence of many actors, each with his or her own agenda and opinion about what should be done and when, and whose behavior is conditioned by the behavior of others, contributes to the fact that the system resists reductionistic analyses.

Thus, the multiproject system is inherently unstable; the best advice one can give on how to manage it is to avoid getting into trouble in the first place, because once one gets trapped in the vicious circle it is very difficult to get out.

Repenning, Gonçalves, and Black [6] argue that every organization has a tipping point, a threshold that determines how much development and how much problem fixing an organization can handle, which once crossed causes fire fighting to spread rapidly from a few isolated projects to the entire organization. The cornerstone of their theory is that the more up-front work done in a project, the less difficulties encountered downstream (see Figure 2.4). Based on this model, Repenning, Gonçalves, and Black produced the chart shown in Figure 2.5. In this figure we see that the development system behaves either like a virtuous or a vicious circle, depending on how much up-front work is done in the current year in support of next year's projects.

click to expand
Figure 2.4: Feedback structure of multiproject product development. (After: [6].)

click to expand
Figure 2.5: Execution modes in a multiproject environment. (After: [6].)

So other things being equal, in a fire-fighting situation desperate interventions lead to more desperate measures, which justify bringing in more firefighters, which leads to more fire fighting, which leads to more extreme measures.



2.4 Common responses to project delays

The following sections provide the factual basis for the claims made in the preceding section. Much of the data used has not been drawn from the project environment—such data is either nonexistent or difficult to access. However, to jump from the original fields of research where the data was collected to project work is not too much of a stretch. For example, if it has been established that air-traffic controllers make more mistakes when they are tired, doesn't that corroborate the contention that people working in projects under high-stress conditions make more mistakes? If psychological studies show that students forget half of what was said in class after only a day or two, can't we make assumptions about what happens to an engineer's train of thought when she is interrupted to work on a more pressing task?

2.4.1 Cutting back or eliminating low-visibility tasks

When a project begins to fall behind schedule, one of the first things to suffer are low-visibility activities, such as tradeoff studies, inspections, and reviews, which are curtailed or dropped altogether in favor of high-output activities, such as coding and integration. In general, people in situations in which they must achieve multiple goals, such as meeting a deadline and achieving a certain level of quality, tend to sacrifice the least visible goal when they perceive that satisfying both would be difficult. In a study conducted by Gilliland and Landis [7] which attempted to evaluate the tradeoffs between quality and quantity when both goals were perceived as being difficult to attain simultaneously, participants gave up the less visible quality goal (see Figure 2.6) and allocated their efforts toward the more achievable quantity goal. Weinberg and Schulman [8] came to a similar conclusion (see Table 2.1). In their experiments, five teams were given the same programming assignment but different goals to achieve. The experiment showed two remarkable results:

  1. Four of the five teams excelled with respect to the objective they were given. One finished second.

  2. None of the teams performed consistently well with respect to all of the objectives.

So if the organization, explicitly or implicitly, favors one goal over another, when something has to give, rest assured it will be the less favored, less visible goal.

click to expand
Figure 2.6: Quantity over quality. (After: [7].)

Table 2.1: Results of Weinberg-Schulman Experiment

Team's Objective

Ranking with Respect to All Objectives


Effort To Complete

Number of Statements

Memory Required

Program Clarity

Output Clarity

Effort to complete

1

4

4

5

3

Number of statements

2–3

1

2

3

5

Memory required

5

2

1

4

4

Program clarity

4

3

3

2

2

Output clarity

2–3

5

5

1

1

In a larger setting, the Software Engineering Institute (SEI), in its 2001 report on process maturity [9], stated, "Software Quality Assurance is the least frequently satisfied Level 2 Key Process Area among organizations assessed at Level 1" (see Figure 2.7). In Figure 2.7, we can see that peer review, an effective but low-profile activity, is seldom practiced even among those organizations looking to be assessed at Level 3 of the SEI maturity scale. In its study of NASA's project management practices (see Table 2.2), the Mars Climate Orbiter [10] Mishap Investigation Board traced many of the mission's problems to the nonperformance of reviews and risk-management activities.

click to expand
Figure 2.7: Software quality assurance and peer reviews are among the least satisfied key process areas of the Software Engineering Institute's Capability Maturity Model. (After: [9].)

Table 2.2: Recurring Themes from Failure Investigations and Studies at NASA

Project


Theme

Mars Climate Orbiter

Wide-field Infrared Explorer

Lewis

Boeing MAR

Faster, Better, Cheaper

Solar Hellospheric Observatory

LMA IAT on Mission Success

Space Shuttle IA Team

Frequency

Reviews

X

X

X

X

X

  

X

6

Risk management

X

 

X

X

X

X

 

X

6

Testing, verification, validation

X

X

 

X

 

X

X

X

6

Communications

X

 

X

  

X

X

X

5

Health monitoring during critical ops

X

X

  

X

   

3

Safety and quality culture

X

  

X

  

X

 

3

Staffing

X

   

X

   

2

Continuity

X

   

X

   

2

Cost and schedule

  

X

 

X

   

2

Engineering discipline

  

X

X

     

Government/contractor roles and responsibilities

  

X

    

X

2

Human error

      

X

X

2

Leadership

X

    

X

  

2

Mission assurance

X

   

X

   

2

Overconfidence

X

      

X

2

Problem reporting

X

      

X

2

Subcontractor, supplier oversight

   

X

  

X

 

2

System engineering

X

  

X

    

2

Training

X

     

X

 

2

Configuration control

     

X

  

1

Documentation

    

X

   

1

Line organization involvement

X

       

1

Operations

X

       

1

Project team

    

X

   

1

Requirements

  

X

     

1

Science involvement

X

       

1

Technology readiness

    

X

   

1

Workforce stress

       

X

1

After: [10].

2.4.2 Effects on product quality and decision making

Many factors affect the quality of work, but here we are interested in the drop in quality resulting from the abandonment of reviews, inspections, regression testing, and other error-identification activities, and from the deterioration in the quality of the decisions made as a result of increased time pressure.

We saw in the previous section that low-visibility activities such as inspections and risk management tend to suffer when they conflict with other tasks of a more visible nature. To understand the negative impact this has in the project workload, it is first necessary to understand the value of inspections, code reviews, and regression testing. Table 2.3 shows the effectiveness of various quality activities.

Table 2.3: Contribution of "Low Visibility" Quality Activities in the Software Industry

Quality Activity

Defect-Identification Effectiveness (%)


Informal design reviews

25–40

Formal design inspections

45–65

Informal code reviews

20–35

Formal code inspections

45–70

Unit test

15–50

New function test

20–35

Regression test

15–30

Integration test

25–40

System test

25–55

Low-volume beta test (10 clients)

25–40

High-volume beta test (1,000 clients)

60–85

After: [11].

When these activities are abandoned, the errors that they could have detected will move to later stages in the product life cycle, where they will have to be fixed at a higher cost. Of course, the most expensive errors are those uncovered only after the product has been released for general use—or in the case of NASA, after the spacecraft has been launched.

Similarly, time pressure affects the quality of the decisions made. In a study conducted by Kim, Wilemon, and Schultz [12] about the consequences of stress on new-product development projects, time pressure was found to be one of the main stressors, and its effect on the quality of decisions and interpersonal relationships mostly negative. When asked to what extent stress affected their work performance, 69% of the participants responded that it adversely affected them to some extent or to a great extent (see Table 2.4).

Table 2.4: Impact of Stress on Performance

Impact of stress on performance

Number of respondents

Percentage


Very great extent

  5

  8.6

Great extent

11

19.0

Some extent

24

41.4

Small extent

16

27.6

Not at all

  2

  3.4

After: [12].

In a survey of 1,414 employees (641 managers and 773 hourly workers) conducted by the consulting firm Kepner-Tregoe [13], the most common barriers to thinking reported by the respondents (see Table 2.5) were organizational politics, time pressure, and lack of involvement in decision making.

Table 2.5: Common Barriers to Thinking

Ranking by Workers

Ranking by Managers


Organizational politics

Time pressure

Time pressure

Organizational politics

Lack of involvement in decision making

Lack of involvement in decision making

After: [13].

Another study conducted by the same firm [14] shows that one of the ways in which employees deal with time pressure is by making less quality decisions (see Figure 2.8).

click to expand
Figure 2.8: Strategies for dealing with time pressure. (After: [14].)

2.4.3 Rework

Many studies have been conducted that compare the cost of fixing or avoiding a problem early on versus fixing it at a latter time. Although the results are somewhat disparate, they are consistent in finding that the cost of a late fix is higher than that of an earlier fix. Figure 2.9 summarizes the results of some well-known studies. Comparable results (see Figure 2.10) have been reported in fields other than software.

click to expand
Figure 2.9: Relative costs of fixing problems when originated during formal testing and in the field.

click to expand
Figure 2.10: Relationship between timing of changes and rework costs in a mechanical engineering project for three different components. Numbers in circles correspond to order of changes. (After: [15].)

The extra cost of fixing a late error reflects the cost of locating and undoing the work in all parts or subsystems where the error might have propagated, correcting the original mistake, redoing the work, and finally verifying the correction. So only if the cost of preventing errors were more expensive than the sum of all the other costs would it make sense to eliminate defect-prevention activities in order to recoup project delays.

2.4.4 Overtime

Another typical response to a late project is to "encourage" people to put more hours into the task. We are not talking here about spending a weekend or a late night at the office in order to fix a problem or to prepare an urgent report, but of those projects that require people to put in long hours, week after week, weekend after weekend, which has a detrimental effect on their social and private lives. From an organizational standpoint, there are limits to worker productivity no matter how many hours employees put in. In a study conducted by Nevinson [16] that used the results of other such studies, despite the number of hours people stayed at the office, 50 hours of work per week was the maximum in terms of worker productivity (see Figure 2.11). Furthermore, after four consecutive 50-hour weeks, people began to experience burnout, and productivity dropped to less than 35 hours per week.

click to expand
Figure 2.11: Hours at the office versus productive working hours. (After: [16].)

2.4.5 Fatigue

Tired people not only produce less, they make more mistakes. Human fatigue has long been recognized as a contributing factor in transportation accidents. The Federal Aviation Administration in its analysis of human factors in aviation accidents (see Figure 2.12) attributes 13.6% of accidents due to human error to "adverse mental states" accounting for loss of situational awareness, mental fatigue, circadian dysrhythmia, and pernicious attitudes such as overconfidence, complacency, and misplaced motivation.

click to expand
Figure 2.12: Analysis of aviation accidents due to human factors. (After: [17].)

In his summary of over 20 years of research studies into operator efficiency as a function of stress and fatigue (see Table 2.6), Hockey [18] reports that fatigue and loss of sleep affect several performance indicators related to the quality of a person's decisions.

Table 2.6: Effects of Fatigue on Performance
 

Performance Indicators


Stressors

Alertness

Attention Breadth

Speed

Accuracy

Short-term Memory Capacity

Noise

+

0

Anxiety

+

0

Incentive

+

+

+

+

Stimulant drugs

+

+

0

Later time of day

+

?

+

Heat

+

0

0

Alcohol

Depressant drugs

+

Fatigue

0

Sleep loss

+

0

Earlier time of day

?

+

+

The table summarizes the typical outcomes of several studies performed independently over several years. A plus (+) sign indicates an increase in the variable measured, a zero (0) either no change or no consistent trend across studies, a minus (–) a decrease in the variable measured. A question mark (?) is used when there is insufficient data to draw conclusions.

2.4.6 Management attention

Figure 2.13 shows the pattern of funds spent, the pattern of funds committed [19], and the pattern of management attention [20] in a typical new-product development project.

click to expand
Figure 2.13: Spending, commitment, and management intervention patterns.

What these patterns clearly show is that management attention increases near the final phases of the project, when the wishful thinking, denial, and optimism that reigned over the previous phases must be abandoned, but when there is little possibility of influencing the outcome of the project within original budgeting and schedule constraints. This type of behavior is confirmed by research done by the firm PRTM, which shows that one of the characteristics that distinguishes best-in-class companies is the timing of management intervention, as suggested by the negligible number of projects canceled by these companies in late phases of execution, as compared with project cancellations at other firms (see Figure 2.14).

click to expand
Figure 2.14: Projects canceled after detailed design work in best-in-class firms as compared with other corporations. (After: [21].)

2.4.7 Multitasking

As the workload increases beyond the organization's capacity to absorb it, resources tend to be assigned to more than one project at a time in an attempt to keep everything moving and the sponsors happy, but while employees attempt to juggle many activities at once, some "balls" are inevitably dropped. The resulting loss of productivity only makes a bad situation worse. Again, we are not talking here about the sporadic distractions that occur naturally in human endeavors—those involving unexpected problems or personal interactions—but rather the chronic fire-fighting behavior that prevails in many organizations.

Although responsible for some loss of productivity [22] and for some very serious accidents, the multitasking that concerns us is not that which arises from doing many things at once, like having a telephone conversation while reading e-mails or speaking on a mobile phone while driving. I refer here to multitasking that results from individuals being "partially" allocated to several projects at one time. Clark and Wheelwright [20] (see Figure 2.15) detected a sharp drop in value-adding activities as engineers were assigned to more than two projects. This drop is consistent with the results of experiments about learning and forgetting conducted by Spitzer [23] as early as 1939. In these studies Spitzer tested 3,605 students to see how much material was forgotten as a function of time; he discovered that after a day or two, the students had forgotten almost half the material they had read (see Figure 2.16). This loss is even more acute [24] when the subject is simultaneously involved in other learning or problem solving activities. By drawing a parallel between the students in Spitzer's experiments and an engineer who is interrupted during a design project to work on another task, we can conclude that there is a substantial time requirement in switching tasks, due to the relearning process that must take place when work on the original project is resumed.

click to expand
Figure 2.15: Loss of productivity due to task switching. (After: [20].)

click to expand
Figure 2.16: Spitzer experiments showing student retention over time. (After: [23].)

But knowledge retention is not the only problem associated with multi-tasking. If a resource is not ready when required, the associated delay will be passed down the "critical chain" of project activities [25]. Although such disruptions are difficult to quantify in a general way, Figures 2.17 and 2.18show how delays affect an employee's work life, to the extent that some time is inevitably allocated to "waiting" and opportunistic work.

click to expand
Figure 2.17: Time allocation in project work. (After: [26].)

click to expand
Figure 2.18: Work trajectory for an engineer assigned to a new-product development project. (Source: [27].)

2.4.8 Head count increases

Back in 1975, F. Brooks [28] enunciated his famous law: "Adding people to a late project makes it later". Although today we know that this law should not be applied indiscriminately since the actual outcome of the intervention depends on circumstances such as what stage the project is at, how many people are currently involved, the type of work that must be done, the system architecture, and so on—it is true nonetheless that adding people to a project adds to the existing workload.

First we need to consider the division of work. If the work to be done must be further subdivided to take advantage of the newly added resources, then there will be some added workload to account for the decomposition and later integration of the new parts which were not included in the original plans. Second, the assimilation of the newcomers naturally adds to the workload. During the learning period, which generally ranges from 4 to 8 weeks (see Figure 2.19), not only are the new additions not fully productive, but they take away time from the more senior members of the team, who must coach the newcomers in the intricacies of the project. Third, as the size of the team increases, so does the communication overhead among project members (see Figures 2.20 and 2.21). Larger teams need more time to communicate, achieve consensus, and coordinate their work. This adds to the already overtaxed members of the team, so the immediate consequence of the increase in head count is to slow the project even further. This in turn might lead the inexperienced manager to oversteer the project by adding yet more people.

click to expand
Figure 2.19: According to a majority of Project Managers it takes at least four weeks to get a new team member up to speed. (After: [29].)

click to expand
Figure 2.20: Communications path in an organization. Although the original study refers to a functional organization, the same pattern is likely to be found in a project whose sections correspond to subsystems and section heads to team leaders or subproject managers. (Source: [30].)

click to expand
Figure 2.21: Effect of team size in communications. (After: [31].)

2.4.9 Scope reductions

When all other interventions fail, the only practical thing left to do is to reduce the project's scope. Changing the schedule, the other alternative, is usually not an option, as it is perceived as having too great an impact on the portfolio.

How common is this practice? Figure 2.22 shows the results of a 1995 survey, known as the CHAOS report [321], conducted by the Standish Group, which shows that 53% of the projects were completed over budget, late, and with fewer features than originally specified. Half of this 53% delivered less than 50% percent of the functionality planned.

click to expand
Figure 2.22: Prevalence of scope reductions to recover from a late project. (After: [32].)