Architectural Evolution and Maturation : Features versus Capabilities
Although much emphasis is placed on the initial creation and early versions of an architecture, the fact is that most of us will be spending the majority of our time working within an existing architecture. How a given architecture evolves can be more fascinating and interesting than simply creating its initial version. It is through evolution that we know where we have success, especially when the evolution is based on direct customer feedback. On a personal note, I've faced my greatest managerial challenges and had my most satisfying accomplishments when I've been able to work with software teams in modifying their existing architecture to more effectively meet the needs of the marketplace and position their product for greater success.
This process, which involves both evolution and maturation, is driven by actual use of the system by customers. Many companies claim to continually request customer feedback, but the reality is that customer feedback is most actively sought, and most thoroughly processed , when planning for the system's next release. The next release, in turn , is defined by its specified required functionality as expressed in the features marketed to customers. Whether or not these features can be created is dependent on the underlying architecture's capabilities. The interplay of requested or desired features and the underlying capabilities required to support them is how architectures evolve over time.
What makes this interplay so interesting is that it takes place within continual technological evolution. In other words, customer feedback isn't always based on some aspect of the existing system, but can be based on an announcement about some technology that could help the customer. In fact, the source of the announcement doesn't have to be a customer, but can quite often be the development team. Regardless of the source or context, it is useful to think of architectural evolution and maturation in terms of features and capabilities.
A feature, or function (I use the terms synonymously but prefer feature because features are more closely related to what you market to a customer) defines something that a product does or should do. The entire set of requested features defines the requirements of the product and can be elicited, documented/captured, and managed through any number of techniques. In case you're wondering what a feature is, here is a time- tested definition from Exploring Requirements: Quality Before Design [Weinberg and Gause, 1989]: "To test whether a requirement is actually a [feature], put the phrase 'We want the product to ' in front of it. Alternatively, you can say, 'The product should '" This approach shows that a wide variety of requirements qualify as features. Here are a few examples:
Note that, as descriptions of features, use cases have an advantage over other forms of documentation because they can put the desired feature into a specific context. The most useful context is based on the goals of the actors involved in the use casewhat they are trying to accomplish. Once you know this, and you know that the system can fulfill these goals, you have the data that product management needs to create the value proposition, which informs the business, licensing, and pricing models.
Features are most easily managed when clearly prioritized by marketing, and they are best implemented when the technical dependencies between them are made clear by the development team. This is because features are usually related, in that one often requires another for its operation. As stated earlier, because the system architecture determines how easy or hard it is to implement a given feature, good architectures are those in which it is considered easy to create the features desired.
Capability refers to the underlying architecture's ability to support a related set of features. The importance of a capability emerges when marketing is repeatedly told that a class of related features, or a set of features that appear to be unrelated on the surface but are related because of technical implementation, is difficult or impossible to implement within the given architecture. Examples in the sidebar on page 12 illustrate this point.
The interplay between architectural maturation and evolution is a function of time and release cycle. It is uncommon for the development team to implement most or all of the desired features in the very first release, especially if the product manager specifies requested features in a well-defined , disciplined manner. If all of the desired features aren't implemented in the first release, they often come along shortly thereafter. Because there aren't a lot of customers to provide feedback, the team is likely to continue working on the leftover features from the first release and complete them in the second. The focus of the team is on completing the total set of desired features, so there is usually little time or energy spent on changing the architecture. Instead, the initial architecture matures. Certainly some changes are made to it, but they are usually fairly tame.
After the system has been in continuous operation for three or more release cycles, or for two or more years , the initial features envisioned by its creators have typically been exhausted and product managers must begin to incorporate increasing amounts of direct customer feedback into the plans for future releases. This feedback, in the form of newly desired features or substantial modifications to existing features, is likely to mark the beginning of architectural evolution, as the development team creates the necessary capabilities that provide for these features.
Of course, not all architectural evolution is driven by customer demand. Companies that proactively manage their products will look for new technologies or techniques that can give them a competitive edge. Incorporating key new technologies into your evolving architecture can give you a sustained competitive advantage. Chapter 3 discusses this in greater detail, and Appendix B includes a pattern language that shows you one way to organize this process.
Architectural maturation and evolution are cyclical, with each phase of the cycle building on the previous one. New capabilities are rarely introduced in a mature state, as it is only through actual experience that can we know their true utility. Through feedback, these capabilities mature. In extremely well-tended architectures, capabilities are removed.
There is a special situation in which the maturation/evolution cycle must be broken and where capabilities dominate the development team's discussions. This is when you must undertake a complete redesign/rewrite of an existing system. Although many factors motivate a redesign/rewrite, one universal motivator is the fact that the existing system does not have the right capabilities and adding them is too costly, either in terms of time or in terms of development resources. In this situation, make certain your architect (or architecture team) clearly captures the missing capabilities so that everyone can be certain that you're going to start with a solid new foundation.
The motivation for redesign/rewrite (the lack of underlying capabilities) often finds its roots in the delivery of new features to a growing customer base as quickly as possible. Quite simply, success can breed failure when is not properly managed.
Architectural degradation begins simply enough. When market pressures for key features are high and the needed capabilities to implement them are missing, an otherwise sensible engineering manager may be tempted to coerce the development team into implementing the requested features without the requisite architectural capabilities. Usually many justifications are provided by everyone involved: Competitors will beat the team to an important customer, or an important customer won't upgrade to the next version, delaying critically needed revenue. Making these decisions is never easy.
The almost certain outcome of this well-intentioned act is that the future cost of maintaining or enhancing new features will be quite high. Additionally, because the underlying architectural capabilities have not been added to the system, any other new features depending on them will find themselves in the same predicament. Worse yet, the cost to morale will likely reduce the effectiveness of the team, further compounding the problemespecially if the alleged "market pressures" turn out not to be so pressing.
The result is that implementing new features results in the team taking on a technical "debt" that must be repaid in the future.  The principal of this debt is the cost associated with creating the right underlying capability. It simply won't go away. The interest is the additional burden of sustaining a feature in an architecture that can't support it. This interest increases with every release and increases again as customers continue to demand new features. If things get bad enoughand they can and dothe team might eventually have to scrap the entire architecture and start over. Servicing the debt has become too high.
While sometimes there really is a critical market window and a shortcut must be taken, or you simply must implement a given feature for an important customer, more often this is just an illusion fostered by impatient and harried executives looking for a supposed quick win.
As people mature, they often became wary of taking on unnecessary debt. You cannot wish away debt but have to pay it. When a given set of desired functions require a significant new or modified architectural capability, avoid debt whenever you can. When it comes to implementing needed capabilities, the phrase "Pay me now or pay me later with interest and penalties" truly applies.