MSF Development Process Model Principles

The MSF Development Process Model fulfills a key function of project development by specifying which activities should be performed and when. The model has two other important aspects: its close relationship with the MSF Development Team Model and the benefits to the organization of using them together; and the MSF Development Process Model's underlying practices and principles. The latter include:

• Using versioned releases.
• Creating living documents.
• Scheduling for an uncertain future.
• Managing tradeoffs.
• Managing risks.
• Maintaining a fixed ship date mindset.
• Breaking large projects into manageable parts.
• Performing daily builds.
• Using bottom-up estimating.

Using Versioned Releases

We recommend a product development strategy that divides large projects into multiple versioned releases, with no separate product maintenance phase. After the project team establishes a pattern of making good tradeoff decisions and shipping the right products at the right time, it's important to begin cycling through versioned releases as rapidly as possible. Versioned releases enable the project team to respond to continuous changes in scope, schedule, and project risk. In the process of frequently updating the product, communication is maintained with the customer, and the customer's suggestions for future releases of the product can be taken into consideration.

The team should deliver a core set of features in the first release and add features incrementally in later releases until the full vision for the product is achieved. For later versions, the product vision can be reshaped as business requirements change, and the product can be updated accordingly.

To summarize, using versioned releases has the following benefits:

• Communication Promotes frequent and honest communication between the team and the customer. Each release reflects the best ideas of everyone involved.
• Earlier delivery Enables the project team to deliver critical functionality earlier and to obtain feedback from the customer for future releases. When the customer knows (or senses) that future product releases will be delivered in a timely manner, the customer is much more receptive to deferring features to later releases.
• Closure Forces closure on project issues. Using a versioned release allows the team to deal with a manageable number of issues during the Stabilizing Phase and to address all the issues before release.
• Goals Sets clear, motivating goals for all team members. The team can easily manage each version's scope and quickly achieve results, so team members see rapid progress. Their role in determining the schedule helps ensure that their tasks are manageable, specific, and associated with a tangible result.
• Freedom and flexibility Allows freedom and flexibility in the design process, enabling the team to be responsive to changes in the business environment. This freedom and flexibility reduces uncertainty and helps to manage the changes in project scope by allowing the team to vary features and schedules in relation to the overall plan. Features that become critical as a result of business changes can be designated as high priority for the next release. The early release becomes stable as the team starts work on the next one.
• Continuous and incremental feature delivery Dictates a new set of features immediately following the release of the completed set. As a result, the team continues to add value for the project's customer and users.

In Rapid Development, Steve McConnell explains:

One of the keys to users and customers agreeing to the version-2 approach is that they have some assurance that there will in fact be a version 2. If they fear that the current version will be the last version ever built, they'll try harder to put all their pet features into it. Short release cycles help to build the users' confidence that their favorite feature will eventually make it into the product.

Another way to reassure the project's customer and users is to create a multi-release product plan from the beginning of the project. This plan involves the articulation of both current and future versions of the product so that the team and the customer can trust in the product's future.

Creating Living Documents

Although a sufficient amount of good planning is crucial to project success, too much planning is harmful. As we've said, an over-emphasis on planning can create "analysis paralysis." To avoid endless spinning in the Planning Phase, the team needs to establish a baseline in its planning efforts as early as possible, so that it can move on to developing the solution even if some questions are left unanswered. On the other hand, because of the ongoing need for change, planning and other documents should be frozen only when leaving them unfrozen poses an unacceptable project risk.

This concept of "baseline early, freeze late" is the essence of creating living documents that change and grow throughout the development process.

One mark of a mature project team is that it admits when a document needs to be changed to reflect new or updated information. Another mark is that it has implemented a good change control process, so that documents are changed only when necessary.

Scheduling for an Uncertain Future

The future is inherently uncertain. Teams need to address future uncertainties in project scheduling and management by planning for them, using two primary approaches: adding buffer time and using risk-driven scheduling.

Buffer Time

Typically, the project team determines the product's ship date by simply adding up all the time estimates along the critical path. In some cases, the team then increases all the estimates by a given percentage in an attempt to allow for delays. This practice is not the same as buffer time.

To use an analogy, buffer time is what a military commander gains by holding back reserve troops to account for variations in the results of an attack plan. In the world of software development, buffer time is a period added by Program Management to the end of a project timeline. Program Management owns the scheduled buffer time and applies it as needed. As shown in Figure 4.8, this period is not factored into the individual tasks, and the goal is still to complete the tasks within the time allotted for them. Buffer time is not an allowance for poorly defined tasks. In almost all cases, having to use the buffer period comes at a cost, even if it is nothing more than a required explanation and justification for using the buffer and a plan for avoiding the problem that caused the buffer use in the future.

Figure 4.8 Critical-path timeline

The addition of buffer time to the timeline creates two ship dates: an internal ship date and an external ship date. The internal ship date is the summation of the critical-path time estimates, and the external ship date is the internal ship date plus the buffer time.

Risk-Driven Scheduling

Risk-driven scheduling assigns a high priority to high-risk tasks and takes into account risk priorities assigned by the customer. If the high-risk tasks require more time than planned, risk-driven scheduling increases the amount of required reaction time. Risk-driven scheduling:

• Encourages early proof-of-concept prototypes.
• Determines which features will be shipped and when.
• Prioritizes tasks based on technical and business risk.
• Encourages developers to aggressively shoot for the early deadline.
• Signals a warning if the early deadline is missed, pinpointing the need to make adjustments and tradeoffs earlier.
• Gives the customer a better view of the riskier areas of the project and manages customer expectations in a more productive manner.

Managing Tradeoffs

In Dynamics of Software Development, Jim McCarthy confirms that every project balances three critical elements:

…you're working with only three things: resources (people and money), features (the product and its quality), and the schedule. This triangle of elements is all you work with. There's nothing else to be worked with. And changing one side of the triangle has an impact on at least one other side, usually two.

The relationship between these three variables tends to be hazy at the beginning of the development process. At that point, the team has a rough idea of what to build, an estimate of available resources, and an approximate target delivery date. During the Planning Phase, the project elements represented in the triangle become more distinct. By the time the Planning Phase is complete, the team knows the nature of available resources, the product features, and the fixed ship date.

It's important to keep in mind that the three variables are interrelated. Changes on one side of the tradeoff triangle affect the other two sides. If the team understands and utilizes this concept, the team has both the rationale and the motivation to take corrective action as changes occur during development.

For example, suppose that a triangle shows that 10 resources will deliver 20 features by June 1. During the development process, the customer discovers a new critical feature that was not included in the original Functional Specification. Adding this new feature to the triangle creates an imbalance in the other sides. As noted in Figure 4.9, the team must correct this imbalance by dropping features, adding resources, changing the ship date, or some combination of all three actions.

Figure 4.9 Unbalanced tradeoff triangle

The power of the triangle lies in its simplicity. It's simple enough that it can be drawn on a napkin during lunch with a customer to explain the types of tradeoffs that must occur in order for the project to succeed.

Although the triangle is a simple and effective tool, it does not convey the project's priorities with regard to the three variables. One way to document these priorities and to manage the expectations of the team and the customer is to create a tradeoff matrix like the ones shown in Table 4.1 for the project variables and the levels of constraint. This matrix allows the team and the customer to indicate the manner in which tradeoffs should occur.

Table 4.1 Sample Tradeoff matrixes

Project 1				Project 2
	Optimize	Constrain	Accept		Optimize	Constrain	Accept
Resources		X		Resources	X
Ship Date	X			Ship Date			X
Features			X	Features		X

Working together, the team and the customer select an exclusive level of constraint for each of the project variables. No row or column in the project tradeoff matrix may have more than one check mark, because hybrid combinations pose serious risks to the project and must be accounted for explicitly in the risk management plan. The columns are defined as follows:

• Optimize The optimized variable is the one that should be as good as possible at the end of the project. To optimize resources is to seek the lowest possible allocation of resources (minimum cost strategy). To optimize the ship date is to seek the earliest possible ship date (early to market strategy). To optimize features is to seek the most complete product possible (maximum benefit strategy).
• Constrain The constrained variable has been assigned a fixed value. To constrain resources is to set a ceiling for them (not-to-exceed strategy). To constrain the ship date is to time-box the project (not before and not after strategy). To constrain features is to ship at least the essential set of functionality (minimal acceptable benefit strategy).
• Accept When one variable has been constrained and another optimized, the third variable must simply be accepted. To accept resources is to acknowledge that the product will take whatever resources it takes (time and materials strategy). To accept the ship date is to acknowledge that the product won't ship before it's finished (it's done when it's done strategy). To accept features is to achieve one or more of the other project tradeoffs by dropping features immediately before the ship date (it's done when it ships strategy). The check mark in the "accept" column designates the variable that is owned by the team and ensures that the team is empowered to manage change and risk, and is therefore positioned to succeed, not fail. For example, if the customer says that the feature set must be as rich as possible (features are optimized) and that the project must be done by a certain date (the ship date is constrained), the customer needs to give the team whatever resources are necessary to meet those two conditions (resource costs are accepted).

The team should use the tradeoff matrix as a reference when making decisions. The matrix is not intended to show absolute priorities; it is merely a tool to facilitate communication and understanding. Most important for the project team is that the matrix defines areas in which the customer is willing to compromise.

Managing Risk

For most projects, the ability to manage risk is the key to project success. For a project team to be successful, it must:

• Learn from its environment.
• Adapt to the environment rapidly.
• Predict accurately what will happen next.
• Take actions based on the above.

When the team understands and implements actions that minimize uncertainty and maximize stability and predictability, it can operate equally well in either a volatile or a stable environment. Preparedness for uncertain events is the goal of risk assessment and management.

There are two inherently different approaches to managing risk. Most teams practice reactive risk management; that is, they react in some way after the risk has already caused a problem. We advocate practicing proactive risk management, which we discuss in detail in Chapter 5. Proactive risk management means that the team has a visible process for managing risks before they are realized.

The process should also be measurable and repeatable. With proactive risk management, risks are assessed continuously, and this assessment information is used to make decisions in all phases of the project. The risks are then carried forward until they are resolved or, if they emerge as problems, until they are handled.

Maintaining a Fixed Ship-Date Mindset

A fixed ship-date mindset means that the team treats its projected ship date as unchangeable. Essentially, the team builds the project schedule and then agrees that the schedule side of the triangle is fixed in place. Once the ship date is fixed, the team cannot use this side of the triangle for making corrective decisions unless there are no other options available.

Adopting a fixed ship-date mindset has the following advantages:

• Forces creativity Creativity is necessary to implement features in as timely a manner as possible, because the option of delaying the ship date has been removed.
• Prioritizes tasks according to importance Features are prioritized so that lower priority features can be dropped if necessary to ship on time. If features must be dropped in order to make the ship date, the features that are most important to the customer will be delivered.
• Empowers the team by providing an effective decision-making tool The team makes decisions on the basis of how they will affect the ability to deliver on the fixed ship date.
• Provides a motivational goal for the team A constantly slipping ship date creates team morale problems and can ultimately lead to a developmental "death march," in which team members lose interest in a project because it seems as if the product will never ship.

The team arrives at a fixed ship date through bottom-up estimation and the use of buffer time. This estimation process is fundamental to the success of implementing a fixed ship date mindset, because the team must be willing to commit to the date as a realistic and achievable goal.

Breaking Large Projects into Manageable Parts

For large projects or complex projects, the feature set should be broken up into smaller, somewhat independent pieces. These pieces should then be treated as internal releases or subprojects. This process can be thought of as versioned releases within a single project, where only the final version is released at the end of the project life cycle.

Teams can spend approximately two to four months on an internal release. Each release has time allocated for feature development, optimization, testing, and stabilization. In addition, approximately one-third of the total development time is added as buffer time for unplanned contingencies.

For each internal release, Development delivers a cluster of features for testing. Assuming that the release is testable, the team goes through a full test/debug/retest cycle, as if it was going to ship the product with just these features. When the code meets or exceeds the quality bar for the internal release, the team can proceed to develop the next set of features. This internal-release approach also helps solve the significant problems that can arise when applications are integrated only in the late stages of product development.

Breaking a large project into subprojects:

• Allows the team to focus on delivering a smaller and more easily definable aspect of the project.
• Provides a sense of completion for the team as it achieves each internal release milestone.
• Provides early warning signs of project health, because each internal release milestone is an assessment point with its own postmortem review. If internal releases slip, then the team can take corrective action earlier to keep the larger project on track.
• Increases the overall quality of the product, because each internal release has its own quality bar.
• Allows the team to practice shipping with each internal release so that the actual shipping of the product at the end of the project is more predictable.

In Debugging the Development Process (Microsoft Press, 1994), Steve Maguire states:

It's not the two-month period alone that creates the wins and fosters enthusiasm. It's the thrill of finishing an interesting subproject.

"Finishing all top-priority items" may be important, but the top-priority items don't make up a subproject. They're just a random list of things that happen to be important. There's no motivating theme behind such a list.

For example, "Implementing the charting subsystem" is a subproject. All of the tasks that would be involved would relate to that common theme. You might use a task list to remind people of the known charting issues they'd have to handle, but ultimately the theme of the subproject would drive development. The goal wouldn't be for the team to finish 352 unrelated tasks. The goal would be to do everything necessary to fully complete—to "ship"—the charting subsystem, regardless of whether the tasks it would take were on a list somewhere. The subproject would be in "ship mode" from the outset.

Performing Daily Builds

A typical software development project involves "building" an executable program from up to thousands of different files. Some software development teams practice the "daily build and smoke test" process in which they compile every file, combine them into a single executable program, and put the program through a smoke test, or simple coverage test, to see if it runs. The smoke test is a quick-pass test of the entire system to expose any major problems. The daily build is not valuable unless accompanied by a smoke test. We discuss coverage testing further in Chapter 12.

Performing daily builds and smoke tests provides a number of important benefits including:

• Minimizing code integration risk by identifying incompatible code early and allowing the team to make debugging or redesign decisions.
• Supporting improved defect diagnosis, making it easier to pinpoint why the product may be broken on any single day.
• Reducing the risk of low quality.

The daily build and smoke test must be performed each day—not weekly or monthly—to produce the greatest benefits. The software being built must work; otherwise, the build is viewed as broken, and it must be fixed. Performing daily builds and smoke tests is like trying to ship a product every day, which enforces a sense of discipline.

Standards for daily builds and smoke tests vary from project to project, but at a minimum the standards should include:

• Compiling all files and components successfully.
• Linking all files and components successfully.
• Finding no "showstopper" bugs that would make the program hazardous to operate or prevent it from launching.
• Passing the smoke test.

Using Bottom-Up Scheduling

Simply put, those who will be doing the work should schedule the work.

Two fundamental benefits result from requiring the individuals who actually perform the work to develop their own work estimates are:

• Accuracy The estimates tend to be more accurate because they are based on experience. A person asked to perform a particular task is expected to have previous experience in executing this or other similar tasks.
• Accountability Because team members have developed their own estimates, they are more accountable for the success of meeting those estimates. Initial estimates may be high, but with milestone reviews, project knowledge, and also estimating practice, the estimates will begin to more accurately reflect the tasks and will provide the right task-level motivation.

As shown in Figure 4.10, the estimating process is the responsibility of the entire team. As the low-level estimates are rolled up into the Master Project Schedule, buffer time is added to ensure that the schedule is attainable. It's this technique that makes the fixed ship-date mindset possible.

Figure 4.10 Bottom-up scheduling