The Formal Test Cycle

Before we get our teeth too deeply into the Visual Basic way of programming, I think it's worth reviewing the different levels of testing that apply to all software development projects regardless of the programming language or target platform.

The nature of testing is so varied in its requirements that it is difficult to give generalized definitions. What is appropriate for a small (one- or two-person) application is totally unsuitable for a large (twenty-person) development, whereas the amount of formality that accompanies a large project would slow down the delivery of a small application by a wholly unreasonable amount. With this in mind, I have tried where appropriate to illustrate the relative scale that is necessary at each stage.

Unit/Component Testing

Unit testing is a test of a simple piece of code—in our case a subroutine, a function, an event, a method, or a Property Get/Let/Set. In formal terms, it is the smallest piece of testable code. It should be nonreliant on other units of code that have been written for this development project because they will almost certainly be only partly tested themselves. However, it is acceptable to call library routines (such as the Visual Basic built-in functions) since you can be highly confident that they are correct. The idea is to confirm that the functional specification of the unit has been correctly implemented. An example of a unit would be a single user-defined calculation.

TIP

---

Sometimes it is necessary to comment out one piece of code to get another piece to work. This might be necessary during the main development cycle when, for example, the underlying code might be dependent on something that has not yet been written or that contains a known bug. If you have to comment out a piece of code, add a Debug.Print statement just before or after it to highlight the fact that you have done so. It's inevitable that you'll forget to remove the leading apostrophe from time to time, and adding a Debug.Print statement should save you from having to find out the hard way.

Component-level testing is the next level up from unit testing. A component can have fairly straightforward functionality, but it is just complex enough to warrant breaking down the actual implementation into several smaller units. For example, a logical process could be specified that calculates the monthly salary for an individual. This process might consist of the following operations:

• Extract from the database the number of hours worked in the month.

• Calculate the amount of gross pay.

• Add a bonus amount (optional).

• Make all standard deductions from this amount.

Each operation will probably have different requirements. For example, the database extraction will need error handling to allow for the usual group of possibilities (user record not found, database service not available, and so on). The calculations will need to prevent numeric type errors (divide by zero, mainly), and if they are remote components, they will have to raise fresh errors. Therefore, the entire component (for example, CalcMonthlySalary) will consist of four smaller units (GetHoursForEmployee, CalcGrossPay, GetBonusAmount, and CalcDeductions), but CalcMonthlySalary will still be small enough to qualify as a unit (for testing purposes).

To test a defined unit, a series of scenarios should be devised that guarantees every line of code will be executed at some time (not necessarily in the same test). For example, if a function includes an If..Then..Else statement, at least two test scenarios should be devised, one to cover each path of execution. If it is a function that is being tested, defining the expected result of the test is generally easier because the return value of the function can be tested for correctness or reasonableness. However, if you are testing a subroutine, you can check only the effect(s) of calling the routine because there is no return value. I generally have a bias toward writing routines as functions where this is reasonable. For some operations, particularly GUI manipulation, it is not so necessary or beneficial because an error should generally be contained within the routine in which it occurred.

In a small system, the developer would likely perform this level of testing. In a larger system, the developer would still perform the initial test, but a separate individual would most likely conduct a more formal version of the test.

Integration Testing

This is the next level up and is concerned with confirming that no problems arise out of combining unit components into more complex processes. For example, two discrete functions might appear to test successfully in isolation, but if function B is fed the output of function A as one of its parameters, it might not perform as expected. One possible cause might be incorrect or insufficient data validation. Using the previous example of the calculation of the single net salary figure, the actual system might implement a menu or command button option to calculate the salaries for all employees and produce a report of the results. It is this entire routine that qualifies as an integration test.

As with unit testing, it is important to write test plans that will execute along all conceivable paths between the units. Integration testing, by its nature, will probably be performed by a dedicated tester—except for small projects.

System Testing

System testing is concerned with the full build of an application (or application suite). At this level, the emphasis is less on bug hunting per se, and more on checking that the various parts of the system correctly interact with each other. The level of testing that would be conducted at this phase would be more systemwide. For example, it could include correct initialization from the Registry, performance, unexpected termination of resources (for example, database connections being terminated when other parts of the system still expect them to be there), logon failures, error recovery and centralized error handling (if appropriate), correct GUI behavior, and correct help file topics, to name just a few.

A system test is conducted on a complete build of the application under construction or at least on a specified phase of it. Ideally, it should be in the state in which the end user will see it (for example, no test dialog boxes popping up and no "different ways of doing things until we code that part of the interface"). Therefore, it should be as complete as possible. In my opinion, the testing cycle should also include the system installation task and not just the execution of the application. If you are developing software for corporatewide use, it is highly unlikely that you will be performing the installations. Most large corporations have dedicated installation teams, and these people are still end users in that they will be running software that you have generated. On the other hand, if you are developing commercial software, the setup program is the first thing the "real" user will see. First impressions count. The Setup Wizard has matured into a very useful tool, but you should still test its output.¹

User Acceptance Testing

User acceptance testing happens when a tested version of the specified deliverable is made available to a selected number of users who have already received training in the use of the system. In this scenario, the users chosen to perform the tests will be expected to give the system the kind of usage that it will receive in real life. The best way to perform this testing is to get the users to identify an appropriate set of data for the system test and to enter it into the system themselves. This data is most useful if it is real rather than hypothetical. Whatever kind of processing the system performs can then be instigated (for example, printing reports) and the results scrutinized carefully. Ideally, the development and testing team will have little or no input into this process, other than to answer questions and to confirm the existence of any bugs that crop up. Apart from this careful input of prepared data, the system should also be used "normally" for a while to determine the level of confidence that can be attributed to the system. If this confidence level is satisfactory, the system can be signed off and a system rollout can commence. If possible, a partial rollout would initially be preferable—not only for prolonged confidence tests, but also to ease the burden on the support team. These people will encounter more queries as to the use of the system during these early days than at any other time during its lifetime, so if the volume of queries can be spread out, so much the better. It also gives them an idea of the most frequently asked questions so that they can organize their knowledge base accordingly.

Regression Testing

Regression testing is the repetition of previously run tests after changes have been made to the source code. The purpose is to verify that things in the new build still work according to specification and that no new bugs have been introduced in the intervening coding sessions. Although it is impossible to quantify precisely (some have tried), figures that I have come across from time to time suggest that for every ten bugs that are identified and cleared, perhaps another four will be introduced. This sounds like a realistic figure to me, although I would apply it more to process code rather than event handlers, which are more self-contained (which, of course, is a benefit of the object-based model that Microsoft Visual Basic employs). As you continue each test/debug iteration, the overall number of bugs in the code should decrease correspondingly until a shippable product exists.

Code Reviews

The code review (or inspection) process is a major part of the software quality cycle, and it is also one of the most important. It is an acknowledgment that the creation of test scripts or the use of automated testing packages only goes so far in assuring the quality of the code. Computers do not yet possess the levels of reasoning necessary to look at a piece of code and deduce that it is not necessarily producing the result specified by the design document. I guess when that day comes, we'll all be out of a job.

The code review is the process whereby the human mind reads, analyzes, and evaluates computer code, assessing the code in its own right instead of running it to see what the outcome is. It is, as the name suggests, a thorough examination of two elements:

• The code itself

• The flow of the code

A code review should also ascertain whether the coding style used by the developer violates whatever in-house standards might have been set (while making allowances for personal programming styles). On a fairly large project a review should probably be conducted twice. The first review should be early on in the development, for example when the first few substantial pieces of code have been written. This will allow any bad practices to be nipped in the bud before they become too widespread. A subsequent review much later in the development cycle should then be used to verify that the code meets the design criteria.

The value of this process should not be taken lightly—it's a very reliable means of eliminating defects in code. As with anything, you should start by inspecting your own code and considering what the reviewer is going to be looking for. The sorts of questions that should come up are along these lines:

• Has the design requirement been met?

• Does it conform to in-house development standards?

• Does the code check for invalid or unreasonable parameters (for example, a negative age in a customer record)?

• Is the code Year 2000 compliant?

• Are all handles to resources being closed properly?

• If a routine has an early Exit subroutine or function call, is everything tidied up before it leaves? For example, an RDO handle could still be open. (The current versions of Windows are much better than their predecessors were at tidying up resources, but it's still sloppy programming not to close a resource when you are done with it.)

• Are all function return codes being checked? If not, what is the point of the function being a function instead of a subroutine?²

• Is the code commented sufficiently?

• Are Debug.Assert statements used to their best advantage? We've been waiting a long time for this, so let's use it now that we have it.

• Are there any visible suggestions that infinite loops can occur? (Look for such dangerous constructs as Do While True.)

• Is the same variable used for different tasks within the same procedure?

• Are algorithms as efficient as possible?