Why (and How) Not to Reinvent the Wheel

So far we've considered design and coding standards that help us write quality, maintainable code. Professional enterprise architects and developers not only write good code; they avoid writing code they don't have to write.

Many common problems (beyond those addressed by J2EE application servers) have been solved well by open source or commercial packages and frameworks. In such cases, designing and implementing a proprietary solution may be wasted effort. By adopting an existing solution, we are free to devote all our effort to meeting business requirements.

In this section we'll look at issues in using third-party frameworks to reuse existing investment.

Help! API Overload

Today, there are many API and technology choices for most problems in J2EE.

Even Sun now seems to be at the point where pulling it all together is so complex that we're seeing significant duplication of effort. For example, JDO and EJB 2.0 entity beans with CMP seem to overlap significantly.

Ultimately, we all pay for duplication of effort in increased effort and decreased quality. At least we can do our utmost to control it within our organization. I believe that code reuse impossible, and we should do our best to achieve it.

There are many ways to avoid duplication of effort and leverage existing code. I suggest the following practices as a starting point:

• Adopt existing frameworks where possible. For example, use a standard logging framework and an existing framework for web applications. However, don't force developers to use organization-wide standard frameworks if it seems that they're not proving a good fit to the problem in hand. Where multiple alternative frameworks exist, survey the options. Don't automatically assume that the first product you look at, or the most popular, will best meet your needs.

• Have zero tolerance for code duplication. This indicates the need for generalization: try to avoid code duplication in the first place, but refactor it out of the way as soon as it appears.

• Ensure good communication amongst developers. For example, have developers give presentations on modules they've recently completed, so that other developers know what common needs are emerging or have already been met. Encourage developers to encourage other developers to use the infrastructure components they've implemented.

• Develop and maintain some simple infrastructure packages that implement functionality that's widely used. Document them well and ensure that all developers are aware of them.

• Adopt standard architectural patterns, even where it's not possible to share code. It's much easier to avoid duplication of effort when working with familiar patterns.

• Use code reviews. This not only helps to boost quality, but also spurs communication within a team.

Using Frameworks

One particularly valuable way of leveraging existing components, whether third-party or developed in-house, is to build within a framework. A framework is a generic architecture that forms the basis for specific applications within a domain or technology area.

A framework differs from a class library in that committing to a framework dictates the architecture of an application. Whereas user code that uses a class library handles control flow itself, using class library objects as helpers, frameworks take responsibility for control flow, calling user code (we've already talked about inversion of control and the Hollywood principle ("Don't call me, I'll call you")). This takes the same approach as the Template Method design pattern, but applies it on a much larger scale.

Frameworks differ from design patterns in that:

• Frameworks are concrete, not abstract. While design patterns are conceptual, you can take an existing framework and build an application with it by adding additional code. This normally takes the form of implementing framework interfaces or subclassing framework classes.

• Frameworks are higher-level than design patterns. A framework may use several design patterns.

• Frameworks are usually domain-specific or technology-specific, whereas design patterns can be applied to many problems. For example, a framework might handle insurance quotations, or provide a clean separation of logic from presentation for web applications. Most design patterns can be used in just about any application.

Adopting a good framework that is a good fit can slash a project's development time. The toughest design problems may have been solved, based on recognized best practices. Much of the project's implementation will be devoted to filling in the gaps, which shouldn't involve so many difficult design decisions.

On the other hand, trying to shoehorn a project into using a framework that is a poor fit will cause serious problems. The problems will be much worse than choosing an unsuitable class library. In that case, the library can be ignored: application developers will simply have to develop their own, more suitable, library functionality. A poorly fitting framework will impose an unnatural structure on application code.

The performance and reliability of the resulting application can also be no greater than that of the framework. Usually, this is not a problem, as an existing framework is likely to have been widely used in earlier projects and its reliability and performance characteristics are known, but in all cases it justifies a thorough quality check of a framework before making a commitment

What Makes a Good Framework?

Good frameworks are simple to use, yet powerful.

The Scylla and Charybdis of framework design are excessive flexibility and irritating rigidity.

Note 

In Greek mythology, Scylla was a sea monster that lived on one side of the Strait of Messia, opposite the whirlpool Charybdis. Sailors had to chart a course between the two.

Excessive flexibility means that the framework contains code that will probably never be used, and may be confusing to work with (it will also be harder to test, as there are more possibilities to cover). However, if a framework isn't flexible enough to meet a particular requirement, developers will cheerfully implement their own way of doing things, so that the framework delivers little benefit in practice.

Good framework code is a little different to good application code. A good framework may contain complex code: this is justified if it conceals that complexity from code that uses it A good framework simplifies application code.

Benefits of Using Existing Frameworks

Generally, it's better to avoid building any but simple frameworks in-house. Open source has flowered over the past few years, especially in Java, and there are many existing frameworks. Developing good frameworks is harder than developing applications.

The main benefit of adopting an existing framework is the same as that in adopting J2EE itself: it enables an organization's development team to focus its effort on developing the required product, rather than concerning itself with the underlying infrastructure. If the third-party framework is popular, there is also a potential advantage in the availability of skills working with that framework.

As usual, there's a trade-off: the learning curve in adopting the framework, and a continuing dependency on the framework. The more complex the project, the easier it is to justify the initial investment and ongoing dependency.

Evaluating Existing Frameworks

Adopting a framework is a very important decision. In some cases, it can determine whether a project succeeds or fails; in many cases, it will determine developer productivity. As with choosing an application server, it's important to conduct a thorough evaluation before making a commitment Remember that even if choosing a framework involves no license costs (in the case of an open source framework) there are many other costs to consider, such as the impact of a learning curve on developer productivity and the likely cost of dealing with any bugs in the framework.

I apply the following criteria to evaluating existing frameworks. Applying them in this order tends to limit the amount of time spent evaluating unsuitable products:

• What is the quality of the project documentation?

• What is the project's status?

• Is the design sound?

• What is the quality of the code?

• Does the release include test cases?

Let's look at each criterion in turn.

What is the Quality of the Project Documentation?

Is there a coherent – and persuasive – overview document that explains the framework's rationale and design? Are there Javadocs for all the classes, and do they contain meaningful information?

What is the Project's Status?

If the product is commercial, the main considerations will be the status of the vendor, the place of this product in the vendor's strategy, and the licensing strategy. There is a real danger in adopting a commercial, closed source, product that the vendor will shut shop or abandon it, leaving users unsupported. Clearly this is less likely to happen with a large vendor.

However, large companies such as IBM initiate many projects that don't fit into their longer-term strategy (consider many of the projects on the IBM Alphaworks site). The viability of the vendor is no guarantee that they will continue to resource and support any individual project. Finally, especially if the product is commercial but currently free, does the small print in the license agreement imply that the vendor could begin to charge for it at any time? Is youi organization prepared to accept this?

If the product is open source, there are different considerations. How live is the project? How many developers are working on it? When was the last release, and how frequently have releases been made? Does the project documentation cite reference sites? If so, how impressive are they? How active are the project mailing lists? Is there anywhere to go for support? Are the project developers helpful? The ideal is to have both helpful developers responding to newsgroup questions and the existence of paid consulting.

Sites such as SourceForge (http://www.sourceforge.net) have statistics on project activity. Other indications are active mailing lists and searching with your favorite search engine for material on the product.

Many managers have reservations about adopting open source products. Although the quality of projects varies widely, such reservations are becoming less and less rational. After all, Apache is now the most widely deployed web server, and has proven very reliable. Several open source Java products are very widely used: for example, the Xerces XML parser and Log4j. We're also seeing interest from major commercial players such as IBM in open source. Xalan and Eclipse, for example, are two significant open source projects that were initially developed at IBM.

Is the Design Sound?

The project's documentation should describe the design used (for example, the design patterns and architectural approach). Does this meet your needs? For example, a framework based entirely on concrete inheritance (such as Struts) may prove inflexible. Not only might this pose a problem for your code, but it might necessitate radical changes in the framework itself to add new functionality in the future. If your classes are forced to extend framework classes, this might require significant migration effort for your organization in future.

What is the Quality of the Code?

This may be time-consuming, but is very important, assuming that the source code is available. Assuming that the product has satisfied the previous criteria, the investment of time is justified.

Spend half a day browsing the code. Apply the same criteria as you would to code written within your organization, and look at some of the core classes to evaluate the cleanliness, efficiency and correctness of the implementation. As an incidental benefit, your team will end up understanding a lot more about the technology in question and, if the framework is well written, may see some useful design and coding techniques.

Does the Release Include Test Cases?

There are challenges developing reliable software with a community of geographically dispersed developers communicating via e-mail and newsgroups. One of the ways to assure quality is to develop a test suite. Successful open source products such as JBoss have large test suites. If an open source product doesn't have a test suite, it's a worrying sign. If you commit to it, you may find that your application breaks with each new release because of the lack of regression tests.

Implementing your own Framework

The first rule of developing frameworks in-house is: don't. In general it's better to adopt existing solutions.

However, there are situations where we have unusual needs, or where existing frameworks don't meet our needs. In this case, it will be better to develop a simple framework than to use an unsuitable existing product or to code haphazardly without any framework.

Even in this case, it's not a good idea to jump in early. Attempt to design a framework only after you understand the problem, and then try to design the simplest possible framework. Don't expect that your first design will be perfect: let the design evolve before making too big a commitment.

Learn from Existing Frameworks

As writing frameworks is hard, successful frameworks are among the most valuable examples of real world design. Take a close look at successful frameworks in your domain and others, the design patterns they use and how they enable application code to extend them.

Implementing a Framework

When implementing a framework, it's vital to have clear goals up front. It's impossible to foresee every requirement in the framework's future, but, unless you have a vision of what you want to achieve, you'll be disappointed with the results.

Probably the most important lesson of scoping a framework is to deliver maximum value with minimum complexity. Often we find a situation where the framework can solve most, but not all, of the problems in a domain fairly easily, but that providing a complete solution is hard. In this case, it may be preferable to settle for a simple solution to 90% of problems, rather than seek to force a generalization that covers the remaining 10%.

Important 

Apply the Pareto Principle if designing a framework. If a particular function seems particularly hard to implement, ask whether it's really necessary, or whether the framework can deliver most of its value without tackling this issue.

Writing a framework differs from writing application code in several ways:

• The XP advice of "Writing the simplest thing that could possibly work" isn 't always appropriate
  It's impossible to refactor the interfaces exposed by a framework without breaking code that uses it and severely reducing its usefulness. Even within an organization, the cost of incompatible changes to a framework can be very large (on the other hand, it is possible to refactor the internals of a framework). So the framework must be designed upfront to meet reasonably anticipated needs. However, adding unneeded flexibility increases complexity. This balance calls for fine judgment.

• Provide different levels of complexity
  Successful frameworks provide interfaces on several levels. It's easy for developers to become productive with them without a steep learning curve. Yet it's possible for developers with more complex requirements to use more features if they desire. The goal is that developers should need to handle no more complexity than is required for the task in hand.

• Distinguish between framework internals and externals
  Externals should be simple. Internals may be more complex, but should be encapsulated.

• It's even more important than usual to have a comprehensive test suite
  The cost of framework bugs is usually much higher than the cost of application bugs, as one framework bug may cause many flow-on bugs and necessitate costly workarounds.

An excellent article by Brian Foote and Joseph Yoder of the University of Illinois at Urbana-Champaign entitled "The Selfish Class" uses a biological analogy to characterize successful software artifacts that result in code reuse. It's particularly relevant to framework design (see http://www.joeyoder.com/papers/patterns/Selfish/selfish.html). See http://c2.com/cgi/wiki?CriticalSuccessFactorsOfObjectOrientedFrameworks for a discussion from an XP perspective.