| The rest of this chapter on static white-box testing covers some problems you should look for when verifying software for a formal code review. These checklists[1] are in addition to comparing the code against a standard or a guideline and to making sure that the code meets the project's design requirements. [1] These checklist items were adapted from Software Testing in the Real World: Improving the Process, pp. 198-201. Copyright 1995 by Edward Kit. Used by permission of Pearson Education Limited, London. All rights reserved.
To really understand and apply these checks, you should have some programming experience. If you haven't done much programming, you might find it useful to read an introductory book such as Sams Teach Yourself Beginning Programming in 24 Hours by Sams Publishing before you attempt to review program code in detail. Data Reference Errors Data reference errors are bugs caused by using a variable, constant, array, string, or record that hasn't been properly declared or initialized for how it's being used and referenced. Is an uninitialized variable referenced? Looking for omissions is just as important as looking for errors. Are array and string subscripts integer values and are they always within the bounds of the array's or string's dimension? Are there any potential "off by one" errors in indexing operations or subscript references to arrays? Remember the code in Listing 5.1 from Chapter 5. Is a variable used where a constant would actually work betterfor example, when checking the boundary of an array? Is a variable ever assigned a value that's of a different type than the variable? For example, does the code accidentally assign a floating-point number to an integer variable? Is memory allocated for referenced pointers? If a data structure is referenced in multiple functions or subroutines, is the structure defined identically in each one?
NOTE Data reference errors are the primary cause of buffer overrunsthe bug behind many software security issues. Chapter 13, "Testing for Software Security," covers this topic in more detail.
Data Declaration Errors Data declaration bugs are caused by improperly declaring or using variables or constants. Are all the variables assigned the correct length, type, and storage class? For example, should a variable be declared as a string instead of an array of characters? If a variable is initialized at the same time as it's declared, is it properly initialized and consistent with its type? Are there any variables with similar names? This isn't necessarily a bug, but it could be a sign that the names have been confused with those from somewhere else in the program. Are any variables declared that are never referenced or are referenced only once? Are all the variables explicitly declared within their specific module? If not, is it understood that the variable is shared with the next higher module?
Computation Errors Computational or calculation errors are essentially bad math. The calculations don't result in the expected result. Do any calculations that use variables have different data types, such as adding an integer to a floating-point number? Do any calculations that use variables have the same data type but are different lengthsadding a byte to a word, for example? Are the compiler's conversion rules for variables of inconsistent type or length understood and taken into account in any calculations? Is the target variable of an assignment smaller than the right-hand expression? Is overflow or underflow in the middle of a numeric calculation possible? Is it ever possible for a divisor/modulus to be zero? For cases of integer arithmetic, does the code handle that some calculations, particularly division, will result in loss of precision? Can a variable's value go outside its meaningful range? For example, could the result of a probability be less than 0% or greater than 100%? For expressions containing multiple operators, is there any confusion about the order of evaluation and is operator precedence correct? Are parentheses needed for clarification?
Comparison Errors Less than, greater than, equal, not equal, true, false. Comparison and decision errors are very susceptible to boundary condition problems. Are the comparisons correct? It may sound pretty simple, but there's always confusion over whether a comparison should be less than or less than or equal to. Are there comparisons between fractional or floating-point values? If so, will any precision problems affect their comparison? Is 1.00000001 close enough to 1.00000002 to be equal? Does each Boolean expression state what it should state? Does the Boolean calculation work as expected? Is there any doubt about the order of evaluation? Are the operands of a Boolean operator Boolean? For example, is an integer variable containing integer values being used in a Boolean calculation?
Control Flow Errors Control flow errors are the result of loops and other control constructs in the language not behaving as expected. They are usually caused, directly or indirectly, by computational or comparison errors. If the language contains statement groups such as begin...end and do...while, are the ends explicit and do they match their appropriate groups? Will the program, module, subroutine, or loop eventually terminate? If it won't, is that acceptable? Is there a possibility of premature loop exit? Is it possible that a loop never executes? Is it acceptable if it doesn't? If the program contains a multiway branch such as a switch...case statement, can the index variable ever exceed the number of branch possibilities? If it does, is this case handled properly? Are there any "off by one" errors that would cause unexpected flow through the loop?
Subroutine Parameter Errors Subroutine parameter errors are due to incorrect passing of data to and from software subroutines. Do the types and sizes of parameters received by a subroutine match those sent by the calling code? Is the order correct? If a subroutine has multiple entry points (yuck), is a parameter ever referenced that isn't associated with the current point of entry? If constants are ever passed as arguments, are they accidentally changed in the subroutine? Does a subroutine alter a parameter that's intended only as an input value? Do the units of each parameter match the units of each corresponding argumentEnglish versus metric, for example? If global variables are present, do they have similar definitions and attributes in all referencing subroutines?
Input/Output Errors These errors include anything related to reading from a file, accepting input from a keyboard or mouse, and writing to an output device such as a printer or screen. The items presented here are very simplified and generic. You should adapt and add to them to properly cover the software you're testing. Does the software strictly adhere to the specified format of the data being read or written by the external device? If the file or peripheral isn't present or ready, is that error condition handled? Does the software handle the situation of the external device being disconnected, not available, or full during a read or write? Are all conceivable errors handled by the software in an expected way? Have all error messages been checked for correctness, appropriateness, grammar, and spelling?
Other Checks This best-of-the-rest list defines a few items that didn't fit well in the other categories. It's not by any means complete, but should give you ideas for specific items that should be added to a list tailored for your software project. Will the software work with languages other than English? Does it handle extended ASCII characters? Does it need to use Unicode instead of ASCII? If the software is intended to be portable to other compilers and CPUs, have allowances been made for this? Portability, if required, can be a huge issue if not planned and tested for. Has compatibility been considered so that the software will operate with different amounts of available memory, different internal hardware such as graphics and sound cards, and different peripherals such as printers and modems? Does compilation of the program produce any "warning" or "informational" messages? They usually indicate that something questionable is being done. Purists would argue that any warning message is unacceptable.
|
