Practice 13. Organize Around the Architecture

Per Kroll

Organizing large projects around the architecture enables more effective communication and collaboration.

Problem

As projects grow in size, communication among team members becomes increasingly complex. The cost of communication increases, and it is hard for team members to know enough about the overall system not to step on each other's toes and overlap their work. One solution would be to have project members to talk to most other project members to make sure they know enough to do a good job. This process soon becomes burdensome, however, and results in reduced efficiency.

This practice describes how you can address that problem by organizing your teams in such a way that you minimize the need for project members to talk to large groups of people to do their job. By organizing around the architecture, you can significantly reduce the number of project members who need to talk to each other.

Background

Say there are fifty people who will work on a major landscaping project. They will install a deck, a swimming pool, a rose garden, a front lawn, a back lawn, and a children's playhouse. As a project manager, how would you organize those fifty people? To make sure that the whole project fits together, you need somebody to "own" the overall architecture and somebody else to own each of the major componentsdeck, pool, rose garden, and so onto ensure the coordinated execution of each of these major items. In addition to giving you and the architect a single point of contact for each major component, this strategy makes it unnecessary for each of the fifty project members to know what each other project member is doing. If you are on the pool team, you clearly need to know mainly the specifics of the pool. However, to ensure that everything will adhere to the style of, say, a classical English country estate, you may also have to understand some of the project's overall themes. Now let's see how this example relates to running a software project.

One of the many benefits of a robust software architecture is that it clearly divides the system responsibilities into well-defined subsystems with well-defined interfaces. The architect or architecture team worries about the architecture and how it all ties together. Individual developers still need to understand the overall system, but they can focus primarily on a subset of a system, that is, one or several subsystems assigned to them. Organizing around the architecture reduces the risk of people stepping on each other's toes and duplicating work.

Organizing around the architecture also improves communication.^[10] Generally, face-to-face communication is most effective, except in the case of large projects, which have too many communication paths. The upper half of Figure 5.2 shows how many possible communication paths there are among all team members: note that it grows geometrically with the size of the team. For a team of size n, the number of communication paths = n * (n-1)/2. This means that a two-person team has 1 communication path, a three-person team has 3 communication paths, but a six-person team has 15 communication paths.

^[10] Kroll 2003.

Figure 5.2. Organizing Around Architecture Minimizes Communication Overload.

The number of possible communication paths among team members grows geometrically with the team size. Organizing around the architecture radically reduces the number of communication paths within a team. Issues regarding subsystem interaction are resolved by the architecture team, which owns the interfaces between subsystems.

(Reprinted from Kroll 2003.)

An increase in communication paths destroys project team efficiency, and you need to find a better method than having everybody communicating with everybody else. You can solve the problem by making one team responsible for the architecture and several small teams each responsible for one or several subsystems. Communication among these few teams is channeled through the architecture team to resolve issues around the overall solution and interfaces between subsystems. As you can see from the lower half of Figure 5.2, this approach leads to simplified and effective communication even in large projects. You typically also need to facilitate other types of communication by having coordinating teams deal with issues related to testing, scheduling, resources, process, and requirements.

Also note that iterative development, with its continuous integration and testing, is the best means of exposing issues to the extended team. If various pieces of your system do not fit well together, the integration and testing will point out what issues you are facing, allowing the relevant people to collaborate to fix them. Following this practice depends on the use of effective configuration and change management environments.

Applying the Practice

As your team grows, you need to find smart ways to divide the work among subteams. In this section we discuss the responsibilities of the architecture team and suggest a few alternative team structures for development teams, based on the overall architecture and size of project. Next we look at the responsibilities of teams owning various subsystems. We then offer guidance on collective code ownership and show how your decisions around who can change what code impact how you do use-case-driven development. Finally, we take a quick look at other teams that are typically not organized around the architecture. Let's have a look at each of these items in turn.

The Architecture Team

Earlys in the project, designate responsibility for the overall architecture.

Early in the project, you should designate a person or team to be responsible for the overall architecture. The architecture team must not work in isolation from the rest of the development team. Instead, architects should work with the entire project team to drive the architecture work, including identification of required subsystems. Architects should also make sure to detail the requirements that will have a fundamental impact on the architecture and design, implement, and test the aspects of the system that constitutes the architecture.^[11] If you are using the Unified Process lifecycle, the architecturally significant aspects of your system should be implemented and tested by the end of the Elaboration phase.^[12]

^[11] See Ambler 2002.

^[12] Kroll 2003.

Later in the project, the architecture team will ensure that each subsystem is developed according to the agreed-upon architecture, that is, that subsystem interfaces are not changed without discussions with the architecture team and other affected teams; that architectural patterns are properly leveraged and not compromised; and that architectural constraints, such as requirements around stability, load, and performance, are adhered to.

As you develop various subsystems, you will need to modify the architecture. As an example, subsystem interfaces may need to change. The architecture team is responsible for understanding the impact such a change will have on other subsystems and for bringing concerned parties together to discuss the impact and resolution of issues. You may also need additional or modified architectural mechanisms,^[13] such as an additional mechanism for dealing with persistency. The architecture team needs to assess the need for an additional mechanism; determine whether it can be resolved by modifying an existing mechanism; and, if not, ensure that all team members understand the availability of the new mechanism and when it should be used.

^[13] A mechanism is a standard software capability that is needed by the system, such as mechanisms for persistency, interprocess communication, message routing, or security.

In summary, the architecture team will be active throughout the project to ensure that the appropriate team members are involved in decision making, that architectural risks are mitigated, and that changes to the architecture are communicated to all concerned parties.

Structure of Development Teams

There are many different ways in which you can divide a large application into subsystems. The right subsystem structure for your project will depend, among other things, on the type and size of system you are building.

Smaller systems may have an architecture consisting of a number of subsystems, without any clear organizational structure between one and another. An alternative would be to organize subsystems around major business functions, in what is often referred to as vertical organization, or organization around feature sets (see Figure 5.3).

Figure 5.3. Teams Organized Around Vertical Business Functions or Feature Sets.

Smaller teams may choose to organize around vertical business functions or feature sets, such as the mortgage or derivatives in this example. Each team would build the user interface, business logic, domain, and persistency capabilities it needs.

As systems grow, organize subsystems in distinct layers.

As systems grow and become more complex, you may use a third method, namely, organizing subsystems in distinct layers, in which the lower layers deal with infrastructure, such as subsystems for persistency, reusable user-interface components, distributed computing, and so on. On top of these, you may have a layer providing domain-specific components, such as components for the financial domain. On top of these, functional subsystems provide the business functions required by end users, such as mortgage, derivatives, personal banking, financial reporting, or investment account management (see Figure 5.4).

Figure 5.4. Teams Organized Around Horizontal Layers and Vertical Business Functions.

When building large and complex systems, it is often advisable to organize around horizontal layers for infrastructure and domain-specific layers, while making separate feature teams responsible for vertical business functions.

Subsystem Teams

Each major subsystem should have a team responsible for it.

Each major subsystem should have a team responsible for it. Depending on the size of each subsystem, one team may be responsible for several. A team responsible for the persistency layer may, for example, have several subsystems handling the interfaces to each of the supported databases. Subsystem teams are responsible for understanding the architectural constraints they have to live within, the subsystem interfaces, permissible technology choices, architectural patterns to adhere to, architectural requirements, and so on.

As mentioned above, subsystem teams typically need to make changes to the architecture, that is, the interfaces or behavior to their subsystem or other subsystem(s), so that they can implement their subsystem in a reasonable way. They should then work with the architecture team to ensure that the changes they want to make are acceptable, and that the architecture team and impacted subsystem teams fully understand what changes need to be made. The appropriate subsystem team(s) then implement the changes and properly communicate them to all project members. This process of managing the subsystem interfaces is what Grady Booch refers to as "managing by the seams."^[14]

^[14] Booch 1996.

It is important to point out that although some people in your team will be responsible for the architecture and others will own a subsystem, this division does not excuse working in isolation from other teams. All team members have a continuous responsibility to do what it takes to build a successful productsee Practice 7: Everyone Owns the Product! As an architect or a developer responsible for a subsystem, you should help out wherever necessary and always look at the bigger picture: are you building an application that will address your business needs? Having responsibility for a certain subsystem, however, means that you are expected to be the primary point person for discussions and resolution of issues related to that subsystem, and you own the integrity of that subsystem.

Can Anybody Change Any Code?

XP ^[15] and Agile Modeling ^[16] both promote collective ownership, that is, the notion that anybody should be able to change any code or model. We believe that such a practice works primarily for small projects with limited system complexity, for which team members can easily communicate with each other to understand what changes have been made, and why; or for slightly larger teams with very skilled team members. If, for example, you are organized in vertical teams as in Figure 5.3, several vertical teams may be changing shared domain components. This way of working is typically fast and effective if effective configuration management practices are used.

^[15] Beck 2004. Corollary practice Shared Code.

^[16] Ambler 2002. Core practice Collective Ownership.

For larger projects, at a minimum you need to make sure that the responsible subsystem team is notified of any changes, so that they can review and approve the change. Otherwise, you run a big risk of having code changes introduce unexpected problems, and you can waste a lot of time trying to find defects introduced by somebody who had only limited appreciation for the complexity and interdependencies of the code within a subsystem. In our experience, the best approach for large projects is to have subsystem team members implement changes to their own subsystem.

For large projects, team members implement changes to their own subsystem.

Use-Case-Driven Development

We recommend using use-case-driven development,^[17] that is, identifying requirements in the form of use cases and scenarios,^[18] making somebody responsible for the design and implementation of each use case and scenario, and also designating somebody to test each one. (Note that for the rest of this section, all discussions about use cases also apply to scenarios.)

^[17] Jacobson 1992.

^[18] See Practice 9: Describe Requirements from the User Perspective for more information.

When designing each use case, describe how various components interact to provide the functionality of the use case. Typically, a use case will leverage capabilities in several different subsystems. It is a good idea to have somebody responsible for each use case who will work with the people involved to ensure that the functionality captured in the use case is delivered.

For smaller systems with collective ownership of the code, the person responsible for a use case also implements it. For larger systems, the person responsible for a use case works with the developers responsible for individual components to specify jointly which capabilities various components are to deliver, and the developers responsible for each component then implement and unit-test the agreed-on capabilities.

For smaller systems, the person responsible for a use case also implements it.

For extremely large systems, use cases may cut across several major subsystems, and the person responsible for a use case distributes only the different chunks of a use case among the involved subsystems; this is called "requirements flowdown."^[19] The subsystem team agrees to implement a certain aspect of a use case, and the team then needs to design and implement how that should be done.

^[19] See the RUP product for more information, especially the plug-in for Systems Engineering.

Is Everybody in the Team Organized Around the Architecture?

So, will everybody on the team belong either to the architecture team or to a subsystem team? No. Larger projects usually have many other teams that you may consider including in your project:

• A team owning system-level requirements.

• A team owning integration/builds.

• A team owning system-level testing.

• Various teams with functional specialties, such as a user-centered design team, or teams responsible for standards compliance, such as accessibility (to ensure that applications can be used by people with disability).

Other Methods

There are at least two essential differences between our application of iterative development and what XP suggests. The first difference concerns whether to focus on architecture early on. XP explicitly states that you should develop only for what you need today. According to this method, therefore, you should not expend any effort predicting what architecture you need to support capabilities you plan to develop later in the project, or in future projects. If you find that you need to change the architecture later in the project, you do it through refactoring. By keeping the code simple and using test-first design, refactoring can be done at a reasonable cost. Our experience is that later rework of the architecture is often costly, especially when teams expand and applications become more complex. Another reason RUP, OpenUP/Basic, and Agile Modeling recommend that you focus on architecture early on is to enable you to align the organization of a larger project with the architecture. To do that, however, you need to baseline a stable architecture early on. This practice may hence be difficult to leverage when using XP.

The second major difference is that XP emphasizes shared code; anybody can change any code, and nobody should own subsystems or major components. We believe, on the other hand, that you need to make somebody responsible for each subsystem and major component, but that people can make updates to other team members' subsystems and components as long as they keep the owner up to date with the work done.

Scrum provides guidance for scaling development teams. For smaller teams (up to twelve people) have a daily short meeting with all team members, a so-called "scrum." As your team grows, have a representative from each of the subteams meet daily to understand what each team is doing, to resolve issues, and to ensure smooth collaboration across subteams ("scrum-of-scrums"). Scrums and scrum-of-scrums are great complements to the above suggestions on how to organize your teams. It is, however, important that this cross-team coordination not only involve scrum masters but also require similar coordination with those who are dealing with testing, requirements coordination, and common infrastructure.

Levels of Adoption

This practice can be adopted at three levels:

• Basic. The team is organized around feature sets or vertical business functions, as in Figure 5.3. You assign owners to the vertical subsystems/feature sets and apply collective code ownership. An architect oversees the architecture and makes sure that architectural issues are resolved as necessary. Tight communication among all developers minimizes overhead.

  The basic practice of organizing around feature sets with collective code ownership is appropriate for small teams. It provides a minimum of ceremony and allows you to have short iterations.

• Intermediate. The project is organized around a combination of vertical and horizontal teams, as in Figure 5.2. Changes to the architecture take place after the architecture team agrees on proposed changes.

  The intermediate practice of having a combination of vertical and horizontal teams is appropriate for medium-size teams. You are now adding more complexity in communications and should consider using scrum-of-scrums. The added complexity tends to drive you toward longer iterations.

• Advanced. Same as intermediate, but change requests are formalized toward the latter part of the project. You need to formally document a suggestion to change the architecture, and an Architecture Control Board (the architecture team with representatives from various subsystem teams) needs to approve the request. You then communicate the decision on supporting material to all concerned teams.

  The advanced practice of formal handling of changes to architecture is appropriate for large development teams. It adds a lot of ceremony, and you need to consider whether that approach is right for your project and, if so, when it is appropriate to add this level of ceremony. If you are using the Unified Process lifecycle, you should not introduce this level of formality until some time in the Construction phase.

Related Practices

• Practice 7: Everyone Owns the Product! discusses how to collaborate across the entire team to develop a high-quality product and to avoid having issues fall between the cracks. Achieving the right collaboration is a key parameter to consider when organizing your team.

• Practice 16: Architect with Components and Services describes the benefits of developing with components and explains how to develop a component-based architecture. A component-based architecture provides clean interfaces between subsystems, making it easier to organize around the architecture.

Additional Information

Information in the Unified Process

OpenUP/Basic describes how an architect coordinates many of the technical issues that need to be resolved. These issues often center on the architecture. RUP adds guidance on layered and other architectural paradigms, how to organize the team around different architectural paradigms, and how to evolve the team structure as you progress through the project phases.

Additional Reading

Philippe Kruchten. "The Software Architect and the Software Architecture Team." In Software Architecture, P. Donohue, ed. Kluwer Academic Publishers, 1999.

Kent Beck with Cynthia Andres.Extreme Programming Explained: Embrace Change, Second Edition. Addison-Wesley, 2004.

Scott Ambler, John Nalbone, and Michael Vizdos. The Enterprise Unified Process: Extending the Rational Unified Process. Prentice Hall, 2005.

Ken Schwaber and M. Beedle. Agile Software Development with SCRUM. Prentice Hall, 2002.

Walker Royce. Software Project Management: A Unified Framework. Addison-Wesley, 1998.