Checks and Balances

The business world turns on checks and balances. Accountants don't audit their own books. Builders and architects have their plans reviewed by structural engineers. Contracting work is always inspected and approved by building inspectors. There is a reason for this: call it selective focus, separation of concerns, or just plain human nature. We all focus on some things and block out others, depending upon the task at hand.

For example, consider why building inspectors find code violations when they inspect a remodeling job. Sometimes it may be because the carpenter was lazy or incompetent, but usually it's because he was focusing on the construction: is the frame plumb and square, is the miter clean with no gaps, are the nails set properly under a shaving, so they can't be seen? All accomplished, but when the inspector arrives: whoops code requires a minimum width of 34 inches for a doorway providing egress from a room below grade, and this doorway is only 32 inches wide.

It's true that by simply focusing on finding the defects rather than building the system, the tester can be much more effective at finding defects. It would be a mistake, though, to conclude that this is the main way a tester benefits the team. In fact, putting too much emphasis on this aspect can lead to a dysfunctional team, where responsibility for testing and quality are placed solely on the tester.

Sure, the tester is focused on the testing and finding defects and quality. But so is everyone else on the team. For instance, one hallmark of XP is the "test-first" pattern, in which programmer pairs write the unit tests first, before they write one line of code. This focuses the programmers on testing and finding defects before they become so involved writing the code that testing becomes an afterthought. These automated unit tests must pass 100% before the code goes into the repository and then for each subsequent integration.

So it's not just the focus on finding defects that provides the benefit. After all, the carpenter who built the wrong-size doorway in our remodeling example was no less focused on finding defects than the inspector; he was just looking for a different type of defect.

The tester will be looking for a different type of defect too. Instead of defects in the function of individual programming objects and methods, which are the focus of the programmers' unit tests, the tester will be looking for disconnects between what the system is and what the customer wants it to be. This is acceptance testing, as opposed to unit testing.

Is this really a benefit? If the unit tests are comprehensive, automated, and pass 100%, doesn't this guarantee that the system is what the customer wants?

No. No. And No. Traditional software development, where unit testing is so haphazard that the systems are literally teeming with low-level bugs that show up in user-apparent problems, might leave room to think so. When you eradicate these, as you can by using XP practices, you're still left with the user-apparent problems. In fact, you may have more, because all those low-level bugs can mask the real gaps between what the customer wants and what the system does.