XP Design Mantra: Refactor Mercilessly

This mantra describes the XP practice of constant refactoring. Refactoring, without a doubt, improves the design of existing code, assuming the code was bad in the first place. It takes the approach of many short steps, carefully nudging the code into better shape.

Smelling Better

(Sing to the tune of Getting Better by The Beatles)

It s smelling better all the time
Better, better, better

You have to admit the code smells better
It s smelling better all the time
(It couldn t smell much worse )
You have to admit it s smelling better
It s smelling better
On rewrite nine

My code is smelling better all the time

It s smelling better all the time
Better, better, better

I used to design like a fool
But refactoring is so cool
We don t write it down
Just smash it around
And ship it as fast as we can

You have to admit it s smelling better
It s smelling better all the time
(It couldn t smell much worse)
You have to admit it s smelling better
It s smelling better
On rewrite nine

Smelling so much better all the time

In XP, refactoring is also an important method of preventing software entropy from creeping in ”that is, preventing the design from becoming more complex as new functionality is added. By applying standard refactoring techniques repeatedly to the code, the code is kept simple and hence easy to extend with new functionality.

When Refactoring Is Useful

Refactoring ”used in the right context ”is a valuable practice. In particular, refactoring is useful on legacy systems that need to be extended with new functionality. If the new functionality requires a change in design, then refactoring (with a full suite of unit tests) is a safe technique for making the change. ^[4]

As described in Martin Fowler s book Refactoring: Improving the Design of Existing Code , refactoring consists of specific practices and techniques that can be applied to existing code, usually with an aim to reach a certain design pattern. In fact, the book can be thought of as Design Patterns in Motion. It includes such refactorings as Substitute Algorithm, Move Method, Replace Type Code with Subclasses, and Decompose Conditional.

The prescribed refactorings in Fowler s book on the subject are pretty much all sensible techniques. ^[5] In addition, the advice about refactoring first (making the code clearer, simpler, and more malleable) and then optimizing afterward if the code needs it is profoundly important for most projects.

Given a project that is in trouble (e.g., collapsing under the weight of its unwieldy codebase ), refactoring can have an almost miraculous effect, unraveling code that previously seemed locked forever in its spaghetti-like embrace. That is the sort of situation where merciless refactoring is useful.

Conversely, if your project is starting afresh with a new design, then you should find that it s possible to apply the same sound principles (and the same Gang of Four design patterns that Fowler s book espouses) to the new system. It s all common sense, and it all represents good design.

You should also find that refactoring a class model prior to writing code is much faster than refactoring source code, simply because there is less to change. So refactoring can prove to be very useful before coding even begins.

Repeated refactoring of particularly badly written code (by applying and fully unit-testing one refactoring at a time) will eventually get you to a nice, clean codebase.

As we just mentioned, refactoring involves cushioning all of your code with unit tests. The tests should be sufficiently comprehensive that you can run them with the confidence that they will catch most problems, even after a small change to the code.

Refactoring isn t the most efficient way to design software. A more efficient approach, we ve found, is to sketch some lines and boxes, quickly knock together a few throwaway prototypes , and then go with the design that you prefer. ^[6] However, if you need to fix some badly written code (or even to tidy up some code that isn t too scruffy, if you have the time), then refactoring is the safest ” possibly even the quickest (in the long run) ”way of doing it.

Note that if you re approaching a body of code because it contains a nefariousbug that needs to be rooted out and crushed, but the code is badly written, then refactoring the code first might seem like a good idea. One of the key principles of refactoring, however, is that you shouldn t make any further changes until all the unit tests pass. If the code contains a known bug, then presumably it s causing a unit test to fail (and if it isn t, a test should be quickly added in so that it does fail). At this stage, whether or not to refactor first is really a judgment call. You might just find it easier (and safer) to root out the bug first.

Afterward, you might decide whether to go ahead and refactor the code anyway. This is another judgment call. It depends on if the area of code has been identified as a known hot spot for bugs ”in which case, spending a little time improving the code will probably save a lot of time later on.

A code hot spot is described in Steve McConnell s book Rapid Development as an error-prone module :

An error-prone module is a module that s responsible for a disproportionate number of defects. On its IMS project, for example, IBM found that 57 percent of the errors were clumped in 7 percent of the modules. ^[7]

Tracking and analyzing defects can help to identify areas of code that would benefit hugely from some heavy refactoring. It s a fair bet that clean, welldesigned code will contain fewer defects than hastily designed spaghetti code. So the time spent refactoring error-prone modules produces a better return than refactoring code that is relatively defect-free.

When Refactoring Falls Short

We have worked at some of the big refactorings for months or years on running systems. When you have a system and it s in production and you need to add functionality, you ll have a hard time persuading managers that they should stop progress for a couple of months while you tidy up. Instead, you have to make like Hansel and Gretel and nibble around the edges, a little today, a little tomorrow. ^[8]
”Kent Beck and Martin Fowler

So, in other words, not getting the design right early costs big-time later on. Contrast that with this quote from the same source:

HYPE!  

With refactoring the emphasis changes. You still do upfront design, but now you don t try to find THE solution. Instead all you want is a reasonable solution. You know that as you build the solution, as you understand more about the problem, you realize that the best solution is different from the one you originally came up with. With refactoring this is not a problem, for it no longer is expensive to make the changes. ^[9]

Light Bulb  

This strikes us as circular, to say the least. Refactoring is time consuming and may involve guerilla tactics to get it done under the nose of management, who doubtless want you to spend time producing new functionality, yet it is also not a problem and no longer expensive. ^[10]

Normally, we view software development as problem ˜ solution. The XP approach appears to be solution ˜ problems.

Now it s time to push the controversy dial up to ten, with this rather vilifying statement: In a software project that is following sound design principles, constant refactoring is the ultimate time- waster . Countless programmer-hours have been lost to perfectionists tinkering away at their finished code, polishing it unnecessarily.

Outside the XP world, constant refactoring (as opposed to occasional refactoring, which is useful) just doesn t make sense, because you don t have to support XP s other practices. The Extremos would argue that constant refactoring is necessary because you re evolving your design as you go along. So if you begin by producing your design, you don t need to spend all that time constantly refactoring your code.

The Extremo justification for constant refactoring (in the context of emergent design) is that it allows you to get functionality into the hands of the users early. We discuss this further in Chapter 12 (and also in Chapter 15).

 

FANGS  

Visualize the Smell and You Will Know When to Stop, My Son

The danger (sorry, one of the dangers) with constant refactoring is that it s difficult to know exactly when to stop doing it. Depending on which Extremo you listen to, you must either listen to the code or smell the code. As we discuss in Chapter 16, this is like listening for the sound of one hand clapping. It s easy to just keep going, because it feels like you re doing good work (even though the project, from an observer s point of view, isn t making any progress) ”if you see a better design, then why not go ahead and change the code? Anyone can change anyone else s code because of the XP practice of collective ownership.

SOLUTION  

Actually Knowing When to Stop

If you see a better design, why wouldn t you change the code to use it? If the code works and has a low (or zero) defect count, and your architecture tells you that this section of code is essentially complete, then there s no need to modify the code any further. It s finished. A detailed architectural model makes it easy to identify dependencies across the overall design, so that we can identify when a piece of code really is complete (i.e., doesn t warrant further refactoring).

If, later on, further defects are discovered in that area of code or additional functionality needs to be designed in, then the code can be revisited ”refactored into a better design if needed.

Refactoring should be approached on the basis of necessity ”identifying what s needed to complete the project. This strikes us as a more effective way of tackling design and code quality than simply sniffing out code smells and attacking any crufty code that you find (regardless of whether redesigning it will benefit the rest of the project).

Refactorin

(Sing to the tune of Taxman by The Beatles, with apologies to George for the attempt at rhythm guitar in [])

[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]

I m happy as a man can be
My client thinks that change is free
So I m refactorin
Yeah, refactorin

[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]

I rewrite my code all day long
First it seems right, then it seems wrong
So I m refactorin
Yeah, refactorin

[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]

My project is so very small
That I don t have much work at all
But I ll have a job for years to come
Because software is never done
When you re refactorin
Yeah, refactorin , yeah

[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]
[dumb, dumb, da dumb dumb dumb]

GROUCHO  

Subsequent launches of additional pay populations were wanted by top management within a year. The team thought that was possible, though I can t remember why. ^[11]

The temptation is always to keep improving the design, perfecting the code, turning that search function into a class, extracting methods here, combining methods there, and so on. This is a dangerous trap to fall into. Once a piece of code works, leave it well enough alone. It works. Once a piece of code is fit for purpose, then it s good enough (albeit with the caveat in the next paragraph). Any more time spent on it is time that could be spent elsewhere on the project.

It s possible to take this good enough approach because we produced a design up front, and spent time prototyping and thinking about the design before we began coding. Therefore, it s a fairly safe bet that for any part of the system, we ll only need to write the code once and only once.

^[4] In fact, XP s practices could have been tailor-made for maintaining legacy systems (possibly because XP aims to put the project into maintenance mode as early as possible, as we discuss elsewhere).

^[5] Except, perhaps, for Introduce Null Object, in which an object that represents null is introduced so that null values don t need to be given special consideration. This has a very limited application, because the whole point of null is that it is special, and therefore should be treated differently from non-null values. Introducing objects that represent null could be storing up trouble, allowing special cases to pass through your code unchecked.

^[6] The key point here being that you design the system in detail, leaving no stone unturned. Taking the right approach (following a logical design process, as we discuss in Chapter 8) can work wonders in uncovering design errors (or preventing them from occurring at all) before any production code has been written.

^[7] Steve McConnell, Rapid Development: Taming Wild Software Schedules (Redmond, WA: Microsoft Press, 1996), p. 72.

^[8] Martin Fowler, Refactoring: Improving the Design of Existing Code (New York, NY: Addison-Wesley, 1999), p. 359.

^[9] Ibid., p. 67.

^[10] In fact, suggesting that it is no longer expensive to make the changes isn t circular, it s just plain wrong. It s an assertion made without proof ”and it s wrong.

^[11] Chet Hendrickson as quoted on the C2 Wiki page Chrysler Comprehensive Compensation, http://c2.com/cgi/wiki?ChryslerComprehensiveCompensation.