Where to Begin

for RuBoard

Ah, where to begin. How do you begin to test a body of code? Start by analyzing assumptions. Bugs tend to result from two kinds of errors: errors in logic and errors in assumptions. Every time you call a function, you make assumptions about what the function's valid parameters are and the valid contexts in which it may be called. This is especially true when you're calling code you didn't write such as operating system or library routines. Contrary to what may seem obvious, the most challenging aspect of working through a large body of code is not figuring out the logic behind each line of code. The most challenging aspect is understanding the assumptions in place when the code was written. You have to analyze the assumptions being made about and by the code you're testing.

For example, in the interest of performance, it's perfectly valid for a deep subroutine not to thoroughly validate the parameters passed to it based on the assumption that some higher level routine is handling parameter validation. However, if you later exposed the subroutine to the outside world, this assumption would no longer be valid. The best approach here would be not to expose the subroutine itself, but to construct a wrapper routine that handles parameter validation before calling the subroutine.

Errors in assumptions tend to be more elusive than errors in logic. They're sometimes difficult to track down, yet you must do so to test the software thoroughly.

Testing your own code is challenging because it's tough to be objective about your own work. Testing the code of others is challenging in that you must deduce the assumptions and requirements underlying the code without actually having written it. Both types of testing present unique challenges.

Usually, the author of a piece of code is best qualified to detail the assumptions behind it. She should know the thought process that went into writing the code. This thought process gives rise to assumptions about the code, the way in which it will be used, and the environment in which it will run. These assumptions form the basis of formal requirements or preconditions, so knowing them is crucial to knowing the code. When you begin trying to understand code you didn't write, start by talking to its author and gaining an understanding of the assumptions that went into it.

Once you have the assumptions in hand, attempt to simplify them. By simplifying the assumptions underlying a body of code, you gradually narrow the problem space down to just the problem at hand. Sometimes this amounts to redefining the problem to allow greatly simplified solutions; sometimes it means merely focusing on a particular aspect of the problem.

Understand that the author of a piece of code is often the worst person to test it objectively. Of necessity, the person who builds a piece of software spends much of her time thinking about how it should work. A tester, on the other hand, must focus on how the software does work. Sometimes familiarity with how the software should work blinds us to ways in which it fails to measure up. Ideally, someone intimately familiar with the assumption and requirement underpinnings of the software, but not its implementation, should be ultimately responsible for testing it. You have to be careful to keep the focus on meeting the requirements, not on implementation details, and you have to be careful not to allow the requirements to be skewed to match the implementation. Your job as the tester is to check the logic of the system as well as the assumptions underlying it.

for RuBoard