Acceptance tests should be kept under version control, just like unit tests and the system code, and usually they can be placed into the same repository. For the most part, you can use the same methods for checking acceptance tests in and out as for unit tests and system code, but you may want different rules for integrating and refactoring them. One of the cardinal rules for code and unit tests is that no integration of new or refactored code is complete until 100% of the unit tests pass. You can't require that 100% of all acceptance tests pass all the time, or you'd be back to the old "big-bang" integration, where the system isn't integrated until the very end when everything is done (and nothing works). We recommend the following rule:
In other words, once an acceptance test passes, it can never be allowed to fail. This is true when integrating both new and refactored code and when integrating refactored acceptance tests. |