TFD Paths

A test path is a series of flows, specified by the flow numbers in the sequence in which they are to be traversed. Paths begin at the IN state and end at the OUT state. A set of paths provides behavior scenarios appropriate for prototyping, simulation, or testing.

A path defines an individual test case that can be "executed" to explore the game's behavior. Path execution follows the events, actions, and states on the TFD. A textual script can be constructed by cutting and pasting primitives in the order they occur along the path. Testers then follow the script to execute each test, referring to the data dictionary for details of each primitive. Automated scripts are created in the same manner, except lines of code are being pasted together rather than textual instructions for a human tester.

Many paths are possible for a single TFD. Tests can be executed according to a single strategy for the duration of the project or path sets can vary according to the maturity of the game code as it progresses through different milestones. The TFD remains constant as long as the correct game requirements and behaviors do not change. Some useful strategies for selecting test paths are described in the following sections.

Minimum Path Generation

This strategy is designed to produce the smallest number of paths that will end up covering all of the flows in the diagram. In this context, "covering" means that a flow is used at least once somewhere in the test.

The benefits of using a minimum path set are that you have a low test count and the knowledge that you exercised all parts of the diagram at least once. The drawbacks are that you tend to get long paths that may keep you from testing some parts of the diagram until later in the project when something goes wrong early in the test path.

Here's how to come up with a minimum path for the TFD in Figure 11.13. Start at the IN and take flow 1 to NoGunNoAmmo .Then go to HaveGun via flow 2. Since flow 3 loops back to HaveGun , take that next and then exit HaveGun via flow 4. The minimum path so far is 1, 2, 3, 4.

Now from HaveGunHaveAmmo , go back to HaveGun via flow 5. Since flow 6 also goes from HaveGunHaveAmmo to HaveGun , take flow 4 again and this time use flow 6 to return to HaveGun . At this stage, the minimum path is 1, 2, 3, 4, 5, 4, 6, but there are still more flows to cover.

Take flow 7 out from HaveGun to go back to NoGunNoAmmo . From here you can take flow 9 to HaveGunHaveAmmo and return back using flow 8. Now the path is 1, 2, 3, 4, 5, 4, 6, 7, 9, 8. All that remains now is to use the flows on the left side of the TFD.

You are at NoGunNoAmmo again so take flow 11 to HaveAmmo and then return to NoGunNoAmmo via flow 10. Only flow 12 and 13 are left now, so take 11 back to HaveAmmo where you can take 12 to HaveGunHaveAmmo and finally exit via flow 13 to the OUT box. The completed minimum path is 1, 2, 3, 4, 5, 4, 6, 7, 9, 8, 11, 10, 11, 12, 13. All thirteen flows on the TFD are covered in fifteen test steps.

There is usually more than one "correct" minimum path for any given TFD. For example, 1, 11, 10, 11, 12, 8, 9, 5, 7, 2, 3, 4, 6, 4, 13 is also a minimum path for the TFD in Figure 11.13. Diagrams that have more than one flow going to the OUT box will require more than one path. Even if you don't come up with the smallest length path(s) mathematically possible, the purpose is to cover all of the flows in the smallest number of paths, which is one for the Ammo TFD.

Baseline Path Generation

Baseline path generation begins by establishing as direct a path as possible from the IN Terminator to the OUT Terminator that travels through as many states without repeating or looping back. This is designated as the baseline path . Then, additional paths are derived from the baseline by varying where possible, returning to the baseline path and following it to reach the OUT Terminator. The process continues until all flows in the diagram are used at least once.

Baseline paths are more comprehensive than minimum paths, but still more economical than trying to cover every possible path through the diagram. They also introduce small changes from one path to another, so a game defect can be traced back to the operations that were different between the paths that passed and the one(s) that failed. One drawback of baseline paths is the extra effort to generate and execute the paths versus using the minimum path approach.

Still using the TFD in Figure 11.13, create a baseline path starting at the IN box and then traveling across the most number of states you can in order to get to the OUT box. Once you get to the NoGunNoAmmo state from flow 1, the farthest distance to the OUT box is either through HaveGun and HaveGunHaveAmmo or through HaveAmmo and HaveGunHaveAmmo . Take the HaveGun route by taking flow 2, followed by flow 4, and exiting through flow 13. This results in the baseline path of 1, 2, 4, 13.

The next thing to do is branch wherever possible from the first flow on the baseline. These are called "derived" paths from flow 1. Flow 2 is already used in the baseline, so take flow 9 to HaveGunHaveAmmo . From there flow 8 puts you back on the baseline path. Follow the rest of the baseline along flows 2, 4, and 13. The first derived path from flow 1 is 1, 9, 8, 2, 4, 13.

Continue to check for other possible branches after flow 1. Flow 11 comes out from NoGunNoAmmo and has not been used yet so follow it to HaveAmmo . Then use flow 10 to return to the baseline. Finish this path by following the remainder of the baseline to the OUT box. This second path derived from flow 1 is 1, 11, 10, 2, 4, 13.

At this point there are no more new flows to cover from NoGunNoAmmo , so move along the next flow on the baseline, which is flow 2. Stop here and look for unused flows to follow. You need to create a path using flow 3. Since it comes right back to the HaveGun state, continue along the remainder of the baseline to get the path 1, 2, 3, 4, 13. The only other flow coming out of HaveGun is flow 7, which puts you right back on the baseline at flow 2. The final path derived from flow 2 is 1, 2, 7, 2, 4, 13.

Now on to flow 4! Flow 4 takes you to HaveGunHaveAmmo , which has three flows coming out from it that aren't on the baseline: 5, 6, and 8. We already used flow 8 in an earlier path, so there is no obligation to use it here. Flows 5 and 6 get incorporated into your baseline the same way since they both go back to the HaveGun state. The derived path using flow 5 is 1, 2, 4, 5, 4, 13 and the derived path for flow 6 is 1, 2, 4, 6, 4, 13.

It may seem like you are done now since the next flow along the baseline goes to the OUT box and you have derived paths from each other flow along the baseline. However, upon further inspection, there is still a flow on the diagram that is not included in any of your paths: flow 12 coming from the HaveAmmo state. It's not connected to a state that's along the baseline so it's easy to lose track of, but don't fall into that trap. Pick up this flow by taking flows 1 and 11 to HaveAmmo and then use flow 12. You're now at HaveGunHaveAmmo and you must get back to the baseline to complete this path. Take flow 8, which is the shortest route and puts you back at NoGunNoAmmo . Finish the path by following the rest of the baseline. This final path is 1, 11, 12, 8, 2, 4, 13.

As you can see, the baseline technique produces many more paths and results in much more testing time than a minimum path. Refer to the sidebar in this section for a summary of this baseline path set.

Summary of Baseline Path Set for Ammo TFD

Baseline:

1, 2, 4, 13

Derived from flow 1:

1, 9, 8, 2, 4, 13

1, 11, 10, 2, 4, 13

Derived from flow 2:

1, 2, 3, 4, 13

1, 2, 7, 2, 4, 13

Derived from flow 4:

1, 2, 4, 5, 4, 13

1, 2, 4, 6, 4, 13

Derived from flow 11:

1, 11, 12, 8, 2, 4, 13

 

Expert Constructed Paths

Expert constructed paths are simply paths that a test or feature "expert" traces based on her knowledge of how the feature is likely to fail or where she needs to establish confidence in a particular set of behaviors. They can be used by themselves or in combination with the minimum or baseline strategies. Expert constructed paths do not have to cover all of the flows in the diagram, nor do they have to be any minimum or maximum length. The only constraint is that like all other paths they start at the IN and end at the OUT.

Expert paths can be effective at finding problems when there is organizational memory of what has failed in the past or what new game functions are the most sensitive. These paths may not have showed up at all in a path list generated by the minimum or baseline criteria. The drawback of relying on this approach are the risks associated with not covering every flow and the possibility of tester bias producing paths that do not perform " unanticipated " sequences of events.

Some expert constructed path strategies are

• Repeat a certain flow or sequence of flows in combination with other path variations

• Create paths that emphasize unusual or infrequent events

• Create paths that emphasize critical or complex states

• Create extremely long paths, repeating flows if necessary

• Model paths after the most common ways the feature will be used

For example, the "emphasize critical or complex states" strategy can be used for the Ammo TFD in Figure 11.13. In this case, the HaveGun state will be emphasized . This means that each path will pass through HaveGun at least once. It is also a goal to cover all of the flows with this path set. To keep the paths short, head for the Exit flow once the HaveGun state has been used.

One path that works is to go to HaveGun , try to shoot, and then leave. This path would be 1, 2, 3, 4, 13. Another would incorporate the DropGun event in flow 7. The shortest way out from there is via flow 9 followed by 13, resulting in the path 1, 2, 7, 9, 13. You also need to include the two flows going into HaveGun from HaveGunHaveAmmo . This produces the paths 1, 2, 4, 5, 4, 13 and 1, 2, 4, 6, 4, 13. Finish covering all of the flows, leaving HaveGunHaveAmmo by using flow 8 in the path 1, 2, 4, 8, 9, 13.

All that remains are some slightly longer paths that cover the left side of the TFD. Flows 1, 11, and 12 get you to HaveGunHaveAmmo . The quickest way from there to HaveGun is either with flow 5 or 6. Choose flow 5, which results in the path 1, 11, 12, 5, 4, 13.

You can eliminate or keep the earlier path that was made for the sole purpose of covering flow 5 (1, 2, 4, 5, 4, 13). It is no longer essential since is has now also been covered by the path you needed for flow 12.

The last flow to cover is flow 10. Go to HaveAmmo , take flow 10, go back through HaveGun , and go out via flow 2. This gives you your final path of 1, 11, 10, 2, 4, 13. The following sidebar lists all of the paths that were just constructed for this set.

Expert Paths Emphasizing the HaveGun State

Expert path set:

1, 2, 3, 4, 13

1, 2, 7, 9, 13

1, 2, 4, 6, 4, 13

1, 2, 4, 8, 9, 13

1, 11, 12, 5, 4, 13

1, 11, 10, 2, 4, 13

Originally constructed but later eliminated:

1, 2, 4, 5, 4, 13

 

Combining Path Strategies

Testing uses time and resources that get more critical as the game project wears on. Here is one way to utilize multiple strategies that might make the best use of these resources for different stages of the project:

1. Use expert constructed paths early on when the game may not be code complete and everything might not be working. Limit yourself to paths that only include the parts that the developers are most interested in or paths that target the only parts of the game that are available for testing.

2. Use baseline paths to establish some confidence in the feature(s) being tested . This can begin once the subject of the TFD is feature complete. You may even want to begin by seeing if the game can pass the baseline path before trying to use the other paths in the set. Anything that fails during this testing can be narrowed down to a few test steps that vary between the failed path(s) and the successful ones.

3. Once the baseline paths all pass, use the minimum paths on an ongoing basis to keep an eye on your feature to see that it hasn't broken.

4. As any kind of delivery point nears, such as going to an investor demo, a trade show, or getting ready to go gold, revert back to baseline and/or expert paths.

This puts a greater burden on the construction of the test paths, but over the course of a long project it could be the most efficient use of the testers' and developers' time.