Static White-Box Testing: Examining the Design and Code

Static White-Box Testing: Examining the Design and Code

Remember the definitions of static testing and white-box testing from Chapter 4? Static testing refers to testing something that isn't runningexamining and reviewing it.

White-box (or clear-box) testing implies having access to the code, being able to see it and review it.

Static white-box testing is the process of carefully and methodically reviewing the software design, architecture, or code for bugs without executing it. It's sometimes referred to as structural analysis .

The obvious reason to perform static white-box testing is to find bugs early and to find bugs that would be difficult to uncover or isolate with dynamic black-box testing. Having a team of testers concentrate their efforts on the design of the software at this early stage of development is highly cost effective.

A side benefit of performing static white-box testing is that it gives the team's black-box testers ideas for test cases to apply when they receive the software for testing. They may not necessarily understand the details of the code, but by listening to the review comments they can identify feature areas that sound troublesome or bug-prone.

NOTE

Development teams vary in who has the responsibility for static white-box testing. In some teams the programmers are the ones who organize and run the reviews, inviting the software testers as independent observers. In other teams the software testers are the ones who perform this task, asking the programmer who wrote the code and a couple of his peers to assist in the reviews. Ultimately, either approach can work. It's up to the development team to choose what works best for them.

The unfortunate thing about static white-box testing is that it's not always done. Many teams have the misconception that it's too time-consuming , too costly, or not productive. All of these are untruecompared to the alternative of testing, finding, and even not finding bugs at the back end of the project. The problem lies in the perception that a programmer's job is to write lines of code and that any task that takes away from his efficiency of churning out those lines is slowing down the process.

Fortunately, the tide is changing. Many companies are realizing the benefits of testing early and are hiring and training their programmers and testers to perform white-box testing. It's not rocket science (unless you're designing rockets ), but getting started requires knowing a few basic techniques. If you're interested in taking it further, the opportunities are huge.

Formal Reviews

A formal review is the process under which static white-box testing is performed. A formal review can range from a simple meeting between two programmers to a detailed, rigorous inspection of the software's design or its code.

There are four essential elements to a formal review:

• Identify Problems . The goal of the review is to find problems with the softwarenot just items that are wrong, but missing items as well. All criticism should be directed at the design or code, not the person who created it. Participants shouldn't take any criticism personally . Leave your egos, emotions, and sensitive feelings at the door.

• Follow Rules . A fixed set of rules should be followed. They may set the amount of code to be reviewed (usually a couple hundred lines), how much time will be spent (a couple hours), what can be commented on, and so on. This is important so that the participants know what their roles are and what they should expect. It helps the review run more smoothly.

• Prepare . Each participant is expected to prepare for and contribute to the review. Depending on the type of review, participants may have different roles. They need to know what their duties and responsibilities are and be ready to actively fulfill them at the review. Most of the problems found through the review process are found during preparation, not at the actual review.

• Write a Report . The review group must produce a written report summarizing the results of the review and make that report available to the rest of the product development team. It's imperative that others are told the results of the meetinghow many problems were found, where they were found, and so on.

What makes formal reviews work is following an established process. Haphazardly "getting together to go over some code" isn't sufficient and may actually be detrimental. If a process is run in an ad-hoc fashion, bugs will be missed and the participants will likely feel that the effort was a waste of time.

If the reviews are run properly, they can prove to be a great way to find bugs early. Think of them as one of the initial nets (see Figure 6.1) that catches the big bugs at the beginning of the process. Sure, smaller bugs will still get through, but they'll be caught in the next testing phases with the smaller nets with the tighter weave .

In addition to finding problems, holding formal reviews has a few indirect results:

• Communications . Information not contained in the formal report is communicated. For example, the black-box testers can get insight into where problems may lie. Inexperienced programmers may learn new techniques from more experienced programmers. Management may get a better feel for how the project is tracking its schedule.

• Quality . A programmer's code that is being gone over in detail, function by function, line by line, often results in the programmer being more careful. That's not to say that he would otherwise be sloppyjust that if he knows that his work is being carefully reviewed by his peers, he might make an extra effort to triple-check it to make sure that it's right.

• Team Camaraderie . If a review is run properly, it can be a good place for testers and programmers to build respect for each other's skills and to better understand each other's jobs and job needs.

• Solutions . Solutions may be found for tough problems, although whether they are discussed depends on the rules for the review. It may be more effective to discuss solutions outside the review.

These indirect benefits shouldn't be relied on, but they do happen. On many teams , for whatever reasons, the members end up working in isolation. Formal reviews are a great way to get them in the same room, all discussing the same project problems.

Peer Reviews

The easiest way to get team members together and doing their first formal reviews of the software is through peer reviews , the least formal method. Sometimes called buddy reviews , this method is really more of an "I'll show you mine if you show me yours" type discussion.

Peer reviews are often held with just the programmer who designed the architecture or wrote the code and one or two other programmers or testers acting as reviewers. That small group simply reviews the code together and looks for problems and oversights. To assure that the review is highly effective (and doesn't turn into a coffee break) all the participants need to make sure that the four key elements of a formal review are in place: Look for problems, follow rules, prepare for the review, and write a report. Because peer reviews are informal, these elements are often scaled back. Still, just getting together to discuss the code can find bugs.

Walkthroughs

Walkthroughs are the next step up in formality from peer reviews. In a walkthrough, the programmer who wrote the code formally presents (walks through) it to a small group of five or so other programmers and testers. The reviewers should receive copies of the software in advance of the review so they can examine it and write comments and questions that they want to ask at the review. Having at least one senior programmer as a reviewer is very important.

The presenter reads through the code line by line, or function by function, explaining what the code does and why. The reviewers listen and question anything that looks suspicious. Because of the larger number of participants involved in a walkthrough compared to a peer review, it's much more important for them to prepare for the review and to follow the rules. It's also very important that after the review the presenter write a report telling what was found and how he plans to address any bugs discovered .

Inspections

Inspections are the most formal type of reviews. They are highly structured and require training for each participant. Inspections are different from peer reviews and walkthroughs in that the person who presents the code, the presenter or reader , isn't the original programmer. This forces someone else to learn and understand the material being presented, potentially giving a different slant and interpretation at the inspection meeting.

The other participants are called inspectors . Each is tasked with reviewing the code from a different perspective, such as a user , a tester, or a product support person. This helps bring different views of the product under review and very often identifies different bugs. One inspector is even tasked with reviewing the code backwardthat is, from the end to the beginningto make sure that the material is covered evenly and completely.

Some inspectors are also assigned tasks such as moderator and recorder to assure that the rules are followed and that the review is run effectively.

After the inspection meeting is held, the inspectors might meet again to discuss the defects they found and to work with the moderator to prepare a written report that identifies the rework necessary to address the problems. The programmer then makes the changes and the moderator verifies that they were properly made. Depending on the scope and magnitude of the changes and on how critical the software is, a reinspection may be needed to locate any remaining bugs.

Inspections have proven to be very effective in finding bugs in any software deliverable , especially design documents and code, and are gaining popularity as companies and product development teams discover their benefits.