Documenting Requirements As Acceptance Tests

˜ Documenting Requirements As Acceptance Tests

A requirement is something that a system has to do. An acceptance test can verify whether a requirement is satisfied. And we applaud the effort to verify requirements with tests. But to equate tests with requirements is completely fallacious. Traffic laws exist to govern the behavior of drivers on the road. Police officers (traffic cops) enforce the traffic laws. But the traffic cop isn t the traffic law.

Trying to relate this to software, the first reason that comes to mind that exposes this fallacy is a very simple one: Requirements have to be something that can be understood by nonprogrammers. Customers, goal donors, gold owners , marketing folks ”all of these people have to determine the required behavior of the system, and these requirements need to be reviewable by the folks that are responsible for them. XP makes the presumption that customers can write acceptance tests. If your customer works in the marketing department, are you sure that s the person you want defining the acceptance tests for the system?

See Chapter 5 for more about the perils of the on-site customer.

 

Generally, the first cut at defining these requirements isn t going to be correct. This is why they need to be reviewed. Extremo philosophy is that source code is progress, therefore (having discussed the requirements with the customer and perhaps made a first-pass effort to validate their correctness) we should jump to code whether or not the requirements are known to be correct (in fact, with requirements that are almost guaranteed to be incorrect ”for the reasons that we discussed earlier in this chapter). Then we write acceptance tests to verify these incorrect requirements. Then we code the incorrect behavior, show it to the users, refactor everything, and do it again (and again, and again). Lather, rinse, repeat.

Does this make sense? Not to us. Can we eventually converge on an answer this way? Possibly. But it certainly seems like a daft way to go about it. It is, to be blunt, ridiculously difficult. It does, however, explain how a team of programmers could spend 4 years building one-third of a payroll system.

The term agility has inherent connotations of fast. You hear agile and you think fast. But going through the overhead of writing unit tests, writing code, building, and integrating over and over again because the only technique deemed acceptable for improving the quality of requirements is working, released software is anything but fast. When pressed on the subject, Extremos will sometimes back off on claims of fast and only claim constant velocity. For example (we present both of these quotes elsewhere in the book, but they bear repeating here):

GROUCHO  

. . . XP doesn t claim ˜fabulous productivity gains . We claim to tell you where you are. Have I published something to the contrary? Let me know, I ll correct it. ^[15]
”Ron Jeffries

GROUCHO  

Once you accept that scope is variable then suddenly the project is no longer about getting ˜done . Rather its [sic] about developing at a certain velocity. And once you establish a velocity then the schedule becomes the customer s problem. ^[16]
”Robert C. Martin

Well, constant velocity on a project isn t a great achievement if the velocity achieved is a snail s pace.

So, let s be clear about where we stand:

• Unit tests: Very important.

• Acceptance tests: ^[17] Also very useful.

• Writing the test before coding the method: Fine by us.

• The tests are the design: Nope.

• The tests are the requirements: Nope.

Fangs for the Memories

FANGS  

The stories and tests snake is driven by these issues:

• User stories have a totally amorphous definition.

• Coding everything for the sunny-day scenario will come back to bite you with both fangs.

• Good luck finding a customer who can code executable acceptance tests.

Regarding Robert C. Martin s quote at the start of the chapter, we could say, with tongue placed firmly in cheek, that XP prioritizes stories with the just slightly arbitrary mechanism of whether the customer can remember each story or not. So if the on-site customer wants his stories to be taken seriously by the team, he needs to have a pretty good memory.

 

User Stories Defanged

SOLUTION  

To refactor XP s approach to project requirements, we need to address these three issues:

• User stories do not define requirements in sufficient detail.

• With the greatest respect to all customers everywhere, human memory is neither reliable nor machine-like in consistency, and customers memories are no exception.

• Acceptance tests are not the ideal medium for capturing and validating customer requirements. We can think of many customers we have dealt with who would simply refuse to work this way.

So we need a better way of capturing requirements than 2-bit user stories, customer memories, and executable tests. Quite simply,

• Define the requirements in detail. Treat an incomplete requirement as a risk ”something that could delay the project because it may hide many other requirements, which in turn could result in a rethink of the architecture.

• Write 4-bit use cases (see the definition earlier in this chapter) to define the behavioral requirements. Capture all of the alternative flows. Leave no stone unturned.

• Write acceptance tests to verify that the requirements have been implemented, not that they have been thought of.

Automated acceptance tests are still useful for proving that the requirements have been correctly implemented, but they should not be confused with actual customer requirements. Acceptance tests bring the requirements too close to the solution, the implementation, to be useful as a specification. Requirements are there to define the problem rather than the solution.

 

^[15] Ron Jeffries posting to OTUG ( http://www.rational.com), subject: C3 Project Terminated, October 10, 2000.

^[16] Robert C. Martin posting to OTUG ( http://www.rational.com), subject: Scope Creep, October 11, 2000.

^[17] You can find some examples of acceptance tests at http://c2.com/cgi/wiki?AcceptanceTestExamples.