QA Must Test with Debug Builds

If you follow my recommendations in Chapter 3, you'll have some excellent diagnostics in your code base. The problem is that, generally, only the developers benefit from the diagnostics. To better help debug problems, the quality engineers need to be using the debug builds as well. You'll be amazed at how many problems you'll find and fix when the QA folks do their testing with debug builds.

One key point is that any assertions you add to the code can have their output disabled so that they do not mess up any automated tests the QA department runs. In the next chapter I discuss assertions for managed and native code. Both the managed code and my SUPERASSERT for native code have ways of turning off any popup message boxes or other interrupting output that cause automated tests to fail.

In the initial stages of the product cycle, the quality engineers should be alternating between debug and release builds. As the product progresses, they should gradually start concentrating more on the release builds. Until you reach the alpha release milestone, at which point you have enough of the features implemented to show customers the product, you should have the quality engineers use the debug build two to three days a week. As you approach beta 1, they should drop to two days a week. After beta 2, when all features and major bugs are fixed, they should drop to one day a week. After the release candidate milestone, they should be on the release build exclusively.