MSF for Agile Software Development

On February 13, 2001, a group of high-profile developers met in Snowbird, Utah to share new lightweight approaches in developing software. There was consensus that traditional, waterfall-based approaches actually hindered how developers write code. They discussed how Extreme Programming (XP) related to other light-development processes and they tried to come up with a unification of concepts. It is at that meeting that the term Agile was adopted and the Manifesto for Agile Software Development was written.

Important

You can view the full text of the Manifesto for Agile Software Development at the following link: agilemanifesto.or/.

Here follows a general overview of the Agile approach to software development. To begin, the process and tools should not take precedence over the developers. In contrast, large projects (such as government or military projects) require extensive documentation. It is an environment where Big Requirements Up Front (BRUF) are needed to an extent. It is quite clear, however, that the process of gathering reams of documentation will slow down the developer. The Agile methodology espouses the importance of working code over documentation. In contrast to Agile methodologies, many larger projects focus on the project specification as the be-all-and-end-all. In a waterfall approach, once you have laid out your plans, a large vehicle is set in motion; good or bad, your project will lumber forward. If there are flaws in your approach, all elements of your project will be affected by the flaws - kind of like a bunch of lemmings falling off a cliff. In Agile, what's important is the end customer and their wants and needs. During the course of developing software, you may find that a particular architecture doesn't work, or a problem comes up that wasn't accounted for in the initial design. An Agile developer can adapt to change for the benefit of the project and the benefit of the customer.

There are many misconceptions surrounding the Agile approach to software development. Some companies construe themselves as agile simply because they are small and don't have a process in place. (They develop software completely ad-hoc.) However, a true Agile approach is quite regimented. It involves more than just doing a bit of unit testing. Agile is a very specific test-first development approach with frequent refactoring, high iterations, frequent (if not continuous) builds, and a close connection with the customer.

Important

There are others that believe that they are working with absolutely no processes in place. This is also untrue - when you write and check in code, even if the process isn't codified, it is still considered a process!

MSF for Agile Software Development was developed by Randy Miller, a strong proponent of the Agile development community. This new process framework was developed with the goal of changing MSF to incorporate modern Agile principles. On his blog (http://blogs.msdn.com/randymiller/), Randy describes how the Agile movement has evolved; rather than focus on a specific approach (such as Scrum or XP), Randy says that proponents are now cherry picking and combining the elements that work. Randy calls this "Agile 2.0" (http://blogs.msdn.com/randymiller/archive/2006/03/23/559229.aspx).

Let's look at how Team System and MSF for Agile Software Development measure up to the Manifesto for Agile Software Development:

• One of the key principles of Agile is that people are more important than process or tools. In MSF for Agile Software Development, the team-of-peers concept comes into play. Each role has the equal responsibility of representing their constituencies. In other words, each person on the team has an equal say and an equal stake for the success or failure of a project. In this worldview, the thought leaders will emerge from the group no matter what their role is, and they will drive the development of the project (unlike a more formalized project manager/developer relationship). In Team System, the process is enacted in the tools, which allows the developer to be more Agile.

• In the manifesto, there is a greater emphasis placed on working code rather than documentation. Team System automanages a lot of the documentation, leaving the developer to be more productive. For example, a rich history of changes in work items is automatically maintained for auditability. The developer doesn't have to do anything apart from day-to-day work. Work items provide a lightweight way of creating and maintaining documentation. The individual work items may not look like much, but once you aggregate them and present them in a report format, what was done for the project and the progress so far emerge with great clarity.

• The Agile methodology maintains that the customer is more important than a set of specifications or contract provisions. MSF enacts this by incorporating the customer in the Team Model. The product manager advocates for the customer and has equal visibility and participation in the development process. Personas, as described later in this section, also play a role in representing the needs and wants of a target audience.

• Finally, the manifesto advocates being responsive rather than following a plan. MSF is built around overlapping iterations with incremental improvements in all stages of development. Because you are working within a team of peers, all members have a say at any stage of the development process, which in turn allows you to improve quality on all levels.

The incorporation of architecture as a component of MSF for Agile Software Development is controversial. The reason the architecture role is in there is because it accurately reflects of the environment of many Microsoft customers. There is also recognition that some planning before implementation isn't necessarily a bad thing, even in an Agile project. In his book Extreme Programming Explained, Kent Beck substitutes the concept of architecture with the concept of the "metaphor" to provide a clear vision of what you are trying to build. Some Agile proponents are slowly starting to advocate techniques such as Enough Design Up Front (ENUF) and continuous architectural refactoring as part of their development cycles.

Team System makes architecture Agile through tooling. It allows you to easily and quickly validate whether your designs work against an existing infrastructure, which allows you to refactor as needed. The Microsoft product teams have also devised a technique using Shadow Modeling, which allows you to apply lightweight architectural design on larger applications. You can read more about shadow modeling in this whitepaper written by Randy Miller: stsc.hill.af.mil/crosstalk/2005/12/0512Miller.html.

One of the traditional project management biases is that developers must be closely managed to make sure they deliver. However, the developers aren't the problem - the root of the problem for not shipping on time and cost overruns can be attributed directly to inaccurate estimation. In the real world, most developers have a tendency to overdeliver features, often trying to incorporate features they think will add value to the product or be needed in future versions. Agile methodologies such as XP and Scrum compensate for bad estimation techniques by taking the brunt of responsibility off the developers and sharing it with the entire development team. XP developers are "allowed" to be off by their estimates. In Scrum, the customer's expectation are managed to allow the developer to focus on a predefined set of features during a sprint, which allows a developer to work within a normal nine-to-five schedule, to focus on the code, and deliver on time.

Therefore, the key question is, How can we improve estimation? If the features keep changing and the complexity is hard to pin down, how can we assign timeframes to the tasks needed to do the job? When you deal with people instead of machines, it increases the complexity of variables you have to consider.

Important

Barry Gervin, one of my colleagues, shared one of the ways that Agile development teams can get a handle on time and cost factors using arbitrary estimation units, specifically called NUTs (Nebulous Units of Time - a term and concept coined by Joshua Kerievsky). Essentially, you estimate the complexity of each feature using a scale of one to four, one being the simplest and four being the most complex. (This is very similar to the concept of Rough Order of Magnitude used as an estimation technique within MSF for Agile Software Development.) By measuring the time required to complete each task, you can profile your team and measure the amount of time required to complete one NUT of work, for example.

To improve estimation on a software development project, you have to "unprogram" yourself from trying to guess time-to-task and substitute empirical principles that are more scientific. Each team will have its own dynamic and velocity - it is up to you to measure and profile your team in controlled circumstances to measure the true velocity of your team. Once you have gathered the necessary data and reverse engineered how your team works, it will then be possible for you to provide concrete realistic estimates. One of the issues with statistical analysis is that you need someone who is well versed in statistical analysis to correctly interpret the data. For example, if you look at lines of code, you may find that less code is being developed from one month to the next. On the surface, the natural tendency is to react negatively and interpret the results as a work slow down. The reality of the situation may be that the level of work is still within normal bounds and the code is being refactored and written more efficiently. Fortunately, Team System simplifies the statistical information by providing default templates that deliver easy-to-interpret results.

One of the characteristics of Agile software development is close customer interaction. When you deal with a large complex distributed development project, you can't always benefit from having the customer onsite. If you think of a large-scale project like building the Windows operating system, you could easily fill a huge metropolitan city with all the target customers! So, how to do deliver on a customer's needs, yet deal with scale? Microsoft has worked out an effective technique using personas - in fact, they played a key role in the development of Visual Studio. Personas are used as an interaction tool - to design one, you must create fictional character representations of your customers with different skill levels and goals. If you are familiar with actors and roles la Unified Modeling Language (UML), you can think of personas as an instantiation of an actor in a role. A persona provides more fine-grained control as you define the wants, needs, and abilities of a user. If a persona is well designed, it will provide the essence of a customer, which can easily be imparted to developers. The concept of personas was popularized by Alan Cooper's book The Inmates Are Running the Asylum. You can learn more about the fascinating history of personas in Alan Cooper's newsletter hosted on his Web site at cooper.com/content/insights/newsletters/2003_08/Origin_of_Personas.asp.

Important

In designing Visual Studio, Microsoft created three personas to target different developer types: Mort (a VB developer), Elvis (a C# developer), and Einstein (a C++ developer). These personas have been controversial in the Windows developer community, as they highlight language, work, and feature differences between C# and VB.

Working with MSF for Agile Software Development

So where is the starting point for MSF for Agile Software Development? First, as explained previously, MSF for Agile Software Development is a framework. If you have a process you are now using, your best bet is to customize MSF to make it fit nicely in your environment. But what if you don't have a satisfactory process? What if you simply would like to take MSF for Agile Software Development for a spin? Well, you can create an instantiation of MSF for Agile Software Development using the default template and the New Project Creation Wizard. The two process templates that ship with Team System have been road tested by many companies, including a council comprising of global systems integrators (GSI).

What you won't find here is a treatise on how to manage your project using Agile processes - there are literally thousands of books on the market on that particular topic! What you will learn here is how the basic template works and what the process is like. You can find the first hint of a development process in the work item instance (in other words, the list of work items that appear by default when you create a new team project using MSF for Agile Software Development). The requirements include:

• Create vision statement

• Create personas

• Brainstorm and prioritize quality of service requirements list

• Create iteration plan

The team project comes with work products (in other words, templates, documents, and spreadsheets) to help you manage your project. If you look at the Requirements folders in the documents related to your team project, you will notice a template for the vision statement called Vision.doc. What's more, if you open up the Vision Statement work item, you will find in the description a link to vision statement documentation within the MSF process guidance.

Important

A huge part of the information you need to use the MSF for Agile Software Development process framework is included in the process guidance and associated templates. The Project Checklist.xls file contains a list of all the preliminary tasks that are required for implementation and is a recommended starting point.

You can start working on an MSF for Agile Software Development based project by defining your vision statement and creating personas. It should be a straightforward process because the templates are available as a starting point. Once you have completed that process, the next step is to define what quality standards are needed for the project to be signed off. Quality-of-service (QOS) requirements define constraints on functionality such as performance and security. A well-written QOS requirement includes empirical, reproducible details. For example, "Make the application secure" is an example of a bad QOS requirement since it is vague. A better QOS requirement is "Code must pass managed code analysis (FxCop) security rules." You can document the backlog of QOS requirements on the provided QOSRequirements.xls spreadsheet. What is great is that you can then easily synchronize the requirements on Team Foundation Server as work items. However, there is more work to be done before we do this. (In case you are interested, work item management is documented in Chapter 13.)

Important

Microsoft has created a Threat Analysis & Modeling tool to figure out attack vectors within your service-oriented application. Once the tool analyzes the architectural design of your application (designed using the Team Edition for Software Architect designers), it provides a threat list that can be uploaded as a set of QOS requirements in Team Foundation Server. You can learn more about this tool at http://blogs.msdn.com/threatmodeling/.

Once you have completed these preliminary steps, it is time to organize and structure your project. MSF for Agile Software Development comes with an iteration plan already baked in. It's basically Iteration0, Iteration1, and Iteration2. A traditional approach in setting up iterations is breaking it down into predictable steps. For example: planning, development, testing and quality assurance, and production. An agilist will handle iterations quite differently - iterations are scopes of work including a list of high priority scenarios, tasks, and requirements. In a test driven development (TDD) approach, testing occurs before development; therefore, we can see right away that the traditional iteration structure doesn't quite work. Using an Agile methodology, rather than deal with large units of work passed from team to team, the work is distributed and broken down in smaller, easier to handle component parts. Admittedly, Iteration0 is quite vague as an iteration name - you may want to personalize it to fit your needs and circumstance. One example of an Agile iteration plan is Scrum's Sprint Backlog.

Working with Areas, Scenarios, and Tasks

Areas provide another way of organizing your project. Think of an area as a category, or a logical grouping. If you are working on a commercial product, you can use the areas to represent different customers. You can also use areas to represent features or even groups that are working on the project. You can configure the areas and iterations for any given team project by right-clicking on it and selecting, Team Project Settings⇨Areas and Iterations.

In most software development projects, there is testing involved. Agile processes use continuous testing; more traditional, formalized processes designate a specific test phase. No matter what process you use, you will have to manage bugs at the end of an iteration (or project). A test plan is often necessary for working out a budget and pricing out bug fixes. You can work out how you will test a project using the test approach (which describes the tests in detail) and test development plan (which allows you to organize and prioritize your different tests). If bugs appear during any part of the process, you can add them to the bug triage list (Triage List.xls). This list allows you to sort out and prioritize your unchecked bugs and defects before assigning them within each iteration.

Next comes the scenario. A scenario differs from a use case, as a persona differs from an actor and a role. A use case indicates a generic list of occurrences with a generic actor/role. For example: "Customer A types in credit card number in form, clicks on the submit button, system responds with approval or rejection." A scenario uses a persona and personalizes the description: "Bob (the persona) accesses the Web form and types in '1234 5678 9012' then clicks on the submit button. Since the credit card number is invalid, Bob receives an error message and a link back to the entry form." The scenario provides a smart way of creating rich descriptions of expected behavior and features in your software. A well-written scenario not only shows a specific use, but also an important pattern of use.

As you might have guessed, there is a Scenario template (Scenario Description.doc) to assist you in writing the scenarios, and a scenario list (Scenarios.xls) to organize (and upload) scenarios once written.

Now that you've gathered your vision statement, personas, scenarios, QOS requirements, and tests, it is now time to open up Microsoft Project and develop your project plan (Development Project Plan.mpp). It is important to get the entire software development team involved in breaking down the scenarios and requirements into smaller component tasks. You first need to evaluate the architectural impact of each scenario. You can then break down each scenario and requirement into smaller, more granular tasks. According to MSF for Agile Software Development, each task must be design-oriented and should be easy to complete in one to three days by a single team member. In Microsoft Project, the layout will look something like Figure 6-1.

Figure 6-1

The scenarios and tasks should be ordered by priority - the really important ones should be assigned to the first iteration. If you take a Scrum-like approach, you may want to solidly scope out the amount of work you are responsible for in each iteration.

Test Cases

Tests must also be organized. You should figure out which ones are ongoing test cases and which ones can be automated (as a managed code analysis rule or as part of a test list associated to the build process). You are now ready to assign the work items (scenarios and associated tasks) to your team members by synchronizing your list with Team Foundation Server.

During the course of the project, you may encounter blocking issues that will prevent certain tasks from being completed. In fact, it's inevitable. For example, you may find that you can't test a mobile application because of delays obtaining the actual mobile device. The tester who is responsible for testing the application on the device should create an issue work item, and then associate the work item to the blocked task work item. You incorporate this process in your day-to-day work. It will make it easier for a project manager to track, isolate, and manage the issues. Another mechanism for handling potential blocking issues in a project is the risk work item. At the beginning of your project, you can identify all risk within your project and then assign tasks to mitigate them.

Once the developer has completed all tasks related to a scenario, the scenario work item can then be assigned to the tester. This allows you to apply tests that can't be automated. Once testing is complete, the scenario can be wrapped up.