What to Look for When Reviewing Documentation

Testing the documentation can occur on two different levels. If it's non-code, such as a printed user's manual or the packaging, testing is a static process much like what's described in Chapters 4, "Examining the Specification," and 6, "Examining the Code." Think of it as technical editing or technical proofreading. If the documentation and code are more closely tied, such as with a hyperlinked online manual or with a helpful paper clip guy, it becomes a dynamic test effort that should be checked with the techniques you learned in Chapters 5, "Testing the Software with Blinders On," and 7, "Testing the Software with X-Ray Glasses." In this situation, you really are testing software.

NOTE

Whether or not the documentation is code, a very effective approach to testing it is to treat it just like a user would. Read it carefully, follow every step, examine every figure, and try every example. If there is sample code, type it in and make sure it works as described. With this simple real-world approach, you'll find bugs both in the software and the documentation.

Table 12.1 is a simple checklist to use as a basis for building your documentation test cases.

Table 12.1. A Documentation Testing Checklist

What to Check	What to Consider
General Areas
Audience	Does the documentation speak to the correct level of audience, not too novice, not too advanced?
Terminology	Is the terminology proper for the audience? Are the terms used consistently? If acronyms or abbreviations are used, are they standard ones or do they need to be defined? Make sure that your company's acronyms don't accidentally make it through. Are all the terms indexed and cross-referenced correctly?
Content and subject matter	Are the appropriate topics covered? Are any topics missing? How about topics that shouldn't be included, such as a feature that was cut from the product and no one told the manual writer. Is the material covered in the proper depth?
Correctness
Just the facts	Is all the information factually and technically correct? Look for mistakes caused by the writers working from outdated specs or sales people inflating the truth. Check the table of contents, the index, and chapter references. Try the website URLs. Is the product support phone number correct? Try it.
Step by step	Read all the text carefully and slowly. Follow the instructions exactly. Assume nothing! Resist the temptation to fill in missing steps; your customers won't know what's missing. Compare your results to the ones shown in the documentation.
Correctness
Figures and screen captures	Check figures for accuracy and precision. Do they represent the correct image and is the image correct? Make sure that any screen captures aren't from prerelease software that has since changed. Are the figure captions correct?
Samples and examples	Load and use every sample just as a customer would. If it's code, type or copy it in and run it. There's nothing more embarrassing than samples that don't workand it happens all the time!
Spelling and grammar	In an ideal world, these types of bugs wouldn't make it through to you. Spelling and grammar checkers are too commonplace not to be used. It's possible, though, that someone forgot to perform the check or that a specialized or technical term slipped through. It's also possible that the checking had to be done manually, such as in a screen capture or a drawn figure. Don't take it for granted.

Finally, if the documentation is software driven, test it as you would the rest of the software. Check that the index list is complete, that searching finds the correct results, and that the hyperlinks and hotspots jump to the correct pages. Use equivalence partition techniques to decide what test cases to try.