Project Risk Management

IT and business project risk management is a subset of the broader processes of business risk management. However in the past, the process of project risk assessment in business and IT projects has tended to be intuitive and hidden.

As shown in Figure 12.3, whenever a software or business guru is asked to estimate how long a project will take, he or she undertakes a most amazing process.In essence, the expert intuitively assesses factors that he or she has learned from bitter experience will influence the length of the project, assesses the impact of those factors depending on the specific tasks , adjusts the estimates by the relative weights of the factors, assesses the probability of the factors actually occurring, and then calculates an adjusted guesstimate . No wonder project people are considered clever!

Figure 12.3. Risk assessment: Mysterious and covert

In effect, project risk management is the formalization of a process that has been covert and subjective . As a result, it has been poorly understood and practiced. (Most contemporary project management texts do not even mention risk management.) Worse, this approach is totally dependent on the project manager's experience.

A Little Risk Test

You are having your first meeting with your project sponsor. Your sponsor says, "We are very excited about this new product. It is going to kill our competitors and they will take so long to match us that, by the time they have it out, there'll be no market left. As you are probably aware, I am very busy with a takeover of another company, but I'll be right behind this product. Our suppliers have promised us some new supersecret technology to help us. So, the sooner you get started the better."

How would you feel about your new project?

Of course, because the process of risk assessment and risk management has remained intuitive and covert, most business people and senior management have no idea of project risk assessment and the need for managing risks. The fact that project risk assessment has been a covert process is one of the major reasons so few organizations have adopted a formal approach to project risk management ”they just don't know about it.

Project Risk Assessment

The risk of a project can be assessed by considering the following broad risk categories:

• System or product complexity (risk),

• Client or target environment (risk), and

• Team environment (risk).

Each of these categories considers different risk factors and should be treated as independently as possible. The overall project risk is the aggregation of all three risk categories.

Risk Test Answer

If you have any experience in projects at all, the words of your sponsor should have struck fear in your heart. Your intuitive "risk antenna" should have gone wild when you read the test. You have a project with an aggressive sponsor who is busy on other projects, you'll probably have a fixed deadline, and you'll be using new and probably unproven technology. Get another job (unless you're a masochist).

Product or System Risk

It has long been understood that the complexity, size , degree of innovation, and other variables of the system or product being developed are a key factor in risk, estimation, and sizing. For software projects the complexity and therefore the risk of a system can generally be evaluated by considering its data complexity; that is, how many inputs, outputs, enquiries, logical internal files, and shared files are involved in the system. For business products, the internal complexity of the product and the size or organizational impact would be indicators of risk.

Other factors that affect system and product complexity are as follows :

• Function, process, and algorithm complexity;

• Complex control, decision exception, policy considerations, and mathematical operations;

• High performance requirements;

• High quality expectations;

• Innovative technology or processing requirements; and

• The degree of change implicit in the product.

A Museum of Wonder and Horror

It is useful for you to imagine a Museum of Modern Products that displayed all systems and products, teams , and client groups in Perspex cubes for public appreciation (or horror!). In this context, consider your system or product in comparison with others that you or your organization has undertaken. Then compare your team and your client group with others that you and your team have had experience with. Putting it all together gives you the overall project risk.

By evaluating the intrinsic system or product complexity, the person or team undertaking risk assessment can predict the risk associated with the product: Is it low, medium, or high risk? As mentioned earlier, it is important that this assessment is undertaken only by considering the software or product, not the team or target environment categories.

Client or Target Environment

For many projects, the most difficult area of risk is the target area or client group for whom the product is being developed. Many of the risks associated with this category are beyond the team's scope of control and, as we discuss later, many of the risk factors in this category defy any form of measurement.

The complexity and risk of the target or user environment are related to the following factors:

• The number of sections and departments, agencies, branches and installations (client sites) involved in developing, implementing, and using the system;

• The level of client knowledge and participation in the application and the project development process;

• The degree of support for the project from the business groups;

• The priority and impact of the application within the client area; and

• The need for physical restructuring of offices, development of new sites, and so on.

Again, as for the other risk categories, when considering the risks associated with the client area, it is important for the project team to consider the client area as independently as possible from the product risks and team risks.

Team Environment

The final category of risk involves the intrinsic risks associated with the project team. As documented by Larry Putnam and Ware Myers (1992), as well as by Boehm (1989) and Jones (1994), the most significant factors affecting project productivity, risk, and effectiveness are the capability, morale , and experience of the team members .

The complexity or risk of the team environment is related to the following factors:

• Schedules, whether fixed or flexible;

• The experience and likely stability of the project team;

• The development and estimated time frame of the project;

• Use of outside vendors or contractors; and

• The physical team and project environment.

Subjective Versus Objective Risk Assessment

Given that there are at least 100 factors that can be included in a risk model, it is difficult to gather objective measures for many of these factors. In the subjective approach, the complexity of the software is determined as simple, average, or complex, depending on the group's experience. However, for example, in a software project, the use of function points (this is a technique for estimating the size of software based on counting inputs, outputs, interfaces, enquiries, and internal files) can provide an objective measure of software complexity where there is an agreement that simply means < 100 adjusted function point (AFD); average means 100 to 1,000 AFP and complex is > 1,000 AFP.

The Dangers of Statistics

In his book, Programming Productivity (1994), Capers Jones identifies 45 risk factors, which he claims account for 90% of the variance between projects. This appears impressive. However, as you probably know, there are a couple of animals that share just over 95% of the same genetic stuff as us humans . So next time you have a look at a pygmy chimpanzee or an orangutan, think about 10% variance.

Metrics ”the Holy Grail

Computing has been measuring various risk factors (e.g., the difference between the productivity of highly experienced and inexperienced programmers) for over 30 years . Significant progress has been made in the objective measurement of these factors. The work of Capers Jones and his group (op cit) is particularly impressive. Put simply, there are good measures for system and team categories of risk. However, the target or client risk factors remain difficult to objectively measure. Worse, in the area of business projects, almost no measures are available.

Unfortunately, many of the risk factors are not practically measurable. For example, the risks associated with developing software for four different client groups who are engaged in advanced political warfare are clearly much higher than those associated with a product for one supportive client. To quantify such factors and their impact is beyond the practical limit for most organizations.

A sensible approach is to quantify the factors that are easily measured and use subjective group agreements for the remaining factors. We have found that this combining of subjective and objective assessment works well.

As Bernstein (1996) stated, "Both objective measurement and subjective degrees of belief are essential; neither is sufficient by itself" (p. 100).

The Risk Assessment Process

The process of risk assessment involves the same participative process as we use in all our planning and project management processes. You as the project manager, your team members, and your critical project stakeholders complete a risk questionnaire and, through a series of open discussions, you discuss the ranking of each risk factor and achieve an overall risk assessment for the project.

What is important is the discussion undertaken during the risk assessment process between team members and stakeholders. It is a powerful process for bringing into the open assumptions and different views on the project.

In our experience, it is unlikely that a team will agree on the risk ranking of all factors. Depending on their skills and background, different team members will see the project differently. If after discussion, there is still no agreement, then a voting technique where the majority wins is the best approach. If the votes are tied, the worst case is used for the risk assessment.

When planning a project, it pays to be paranoid .

Ideally within each organization, there should be a standard risk assessment questionnaire for each class of project (e.g., in-house software, package, communications, and operations). However, if there is no standard risk model for your organization, it is reasonably simple to develop one by gathering your most experienced project managers for a brainstorming session in which all factors that have caused their projects trouble are listed, discussed, ranked, and scored by level of impact. The resultant questionnaire can then be compared to the various factors discussed by Jones, Boehm, and others for confirmation of impact.

A simple and very useful risk questionnaire is shown in Figure 12.4.

Figure 12.4. Project risk assessment: Short model

It is very important that you use the short risk assessment questionnaire only for small projects (i.e., shorter than three months). For larger projects, use this model as a starting point and brainstorm with the RAP participants any additional risks that may apply to your project. ^[5]

^[5] There is an online version of this questionnaire available on our Web site (www.Thomsett.com.au).

Open or Closed Voting?

If there is a high degree of trust among you, your team, and your stakeholders, you can undertake the evaluation and ranking of each risk factor in a open, free-wheeling discussion. If not, then each participant in the planning session completes the questionnaire privately and then you have the open discussion. This generally reduces the likelihood of groupthink .

Zip Your Lips

It is pretty likely that during the risk assessment process with your team and your stakeholders you'll personally disagree with many of their rankings. Just zip your lips. Remember, you will be more experienced than your team, so what they see as a high risk you may see as low (because it's easy, right?). Well you must ask yourself this question: Who is doing the project? You have a right to see if there is anything that can be done to help the team lower the risk but, if not, you must go with their assessment.

Overall Project Risk Assessment

At the conclusion of the risk assessment process, there will be an assessment of the risk level associated with the product, the team, the target environment, and the overall project:

• Product: Low risk

• Team: Medium risk

• Target: High risk

• Overall project: Medium risk

In addition, the team will have identified a number of high risk factors that place the project at risk. It is here that the second process of risk management, risk control (or containment), is undertaken.