Test Planning

John Stevenson lives in Vancouver with his wife Cindy and their two kids Shawn and Cassie, who are the second cousins of Mary Shaw, who is married to Richard Shaw, whose grandmother was Stewart Werthington's housekeeper, whose kids Damien and Charlie went to the Mansfield Christian School for Boys with Danny Robinson, whose sister Berta Robinson ran off with Chris Tanner, who rides a motorcycle and greases his hair and their kid Christa used to go out with my pal Tom Slipper, who is the main character of this story, but not the narrator 'cause I am (Tommy couldn't write to save his life).

— Emma Dolan

Introduction

Mort Sahl, the brilliant social commentator of the 1960s, often began his act by dividing the world into the "right wing," the "left wing," and the "social democrats." The previous two chapters have described the right and left wings. Now it's time for the social democrats.

Scripted testing is based on the sequential examination of requirements, followed by the design and documentation of test cases, followed by the execution of those test cases. The scripted tester's motto is, "Plan your work, work your plan." Exploratory testing is a very different paradigm. Rather than a sequential approach, exploratory testing emphasizes concurrent product learning, test design, and test execution. The tester designs and executes tests while exploring the product.

Technique

Planning has been defined simply as "figuring out what to do next." Most of us would admit that to be effective and efficient, planning is important. But when and how should that planning be done? Scripted testing emphasizes the value of early test planning and design as a method of detecting requirements and design defects before the code is written and the system put into production. Exploratory testing challenges the idea that tests must be designed so very early in the project, when our knowledge is typically at its minimum. In his article, "Exploratory Testing and the Planning Myth," published on StickyMinds.com, James Bach discusses the planning of plays that are run in a football game. He examines when the plays can or should be planned. Let's consider this sport to learn more about planning.

But first, an apology or explanation. In this chapter the term "football" refers to the game of the same name as played in the United States and Canada and exported, with only marginal success, to the rest of the world. "Football" does not refer to that marvelous game played world-wide that North Americans call "soccer."

For More Information

To learn more about the game of football as played in North America see ww2.nfl.com/basics/history_basics.html

When are football plays planned? Our first thought might be in the huddle just before the play begins, but the following list shows more possibilities:

Planned Football Play

• Before the game begins - the first n plays are chosen and executed without regard to their success or failure to evaluate both teams' abilities
• Before each play - in the huddle, based on an overall game plan, field position, teams' strengths and weaknesses, and player skills and experience
• At the line of scrimmage - depending on the defensive lineup
• At the start of a play - play action - run or pass depending on the defense
• During the play - run for your life when all else has failed

Adaptive Planning

Adaptive planning is not an industry standard term. Other possible terms are:

• Dynamic
• Flexible
• Just-In-Time
• Responsive
• Pliable
• Progressive
• Purposeful planning

We could define the terms "classical planning" and "adaptive planning" to indicate these different approaches. The relationship between classical planning and adaptive planning in football is:

Table 14-1: Classical planning vs. Adaptive planning.

Classical Planning	Before the game begins (the first ten plays are scripted)
Adaptive Planning	Before each play (in the huddle)
	At the line of scrimmage (depending on the defensive setup)
	At the start of a play (play action - run or pass)
	During the play (scramble when all else has failed)

Let's now leave football and consider software test planning. (While we'd rather stay and watch the game, we've got software to test.)

Table 14-2: Classical test planning vs. Exploratory test planning.

Classical Test Planning	As requirements, analysis, design, and coding are being done—long before system is built and the testing can begin
Adaptive Test Planning	Choose a strategy (depending on our current knowledge)
	Before each screen / function / flow is to be tested
	At the start of an individual test (choose different strategies)
	During the test (as we observe things we don't expect or understand)

A reasonable planning heuristic would be:

• We plan as much as we can (based on the knowledge available),
• When we can (based on the time and resources available),
• But not before.

Aside from these new labels, haven't good planners always done this? Is this concept really new?

A remarkable little book simply titled Planning, published by the United States Marine Corps in 1997, describes the concepts of adaptive planning in detail.

The Marine Corps defines planning as encompassing two basic functions—"envisioning a desired future and arranging a configuration of potential actions in time and space that will allow us to realize that future." But, to the Marines, planning is not something done early which then becomes cast in concrete. "We should think of planning as a learning process—as mental preparation which improves our understanding of a situation." Plans are not concrete either. "Since planning is an ongoing process, it is better to think of a plan as an interim product based on the information and understanding known at the moment and always subject to revision as new information and understanding emerge."

The authors of Planning list these planning pitfalls to avoid:

• Attempting to forecast events too far into the future. By planning we may fool ourselves into thinking we are controlling. There is a difference.
• Trying to plan in too much detail. Helmuth von Moltke, German Army Chief of Staff during World War I said, "No plan survives contact with the enemy." In exactly that same way, no test plan survives contact with the defects in the system under test.
• Institutionalizing planning methods that lead us to inflexible or lockstep thinking in which both planning and plans become rigid. Rather than "Plan your work and work your plan" as our mantra, we should constantly "Plan our work, work our plan, re-evaluate our work, re-evaluate our plan."
• Thinking of a plan as an unalterable solution to a problem. Rather, it should be viewed as an open architecture that allows us to pursue many alternatives. "We will rarely, if ever, conduct an evolution exactly the way it was originally developed."
• Ignoring the need for a feedback mechanism to identify shortcomings in the plan and make necessary adjustments. This is a component of planning which often does not receive adequate emphasis. "Many plans stop short of identifying the signals, conditions, and feedback mechanisms that will indicate successful or dysfunctional execution."

Adaptive planning, as described above, acknowledges and deals with these pitfalls.

The following excerpt from Planning summarizes these concepts well:

• "Planning is a continuous process involving the ongoing adjustment of means and ends. We should also view planning as an evolutionary process involving continuous adjustment and improvement. We can think of planning as solution-by-evolution rather than solution-by-engineering. We should generally not view planning as trying to solve a problem in one iteration because most ... problems are too complex to be solved that way. In many cases, it is more advisable to find a workable solution quickly and improve the solution as time permits. What matters most is not generating the best possible plan but achieving the best possible result. Likewise, we should see each plan as an evolving rather than a static document. Like planning, plans should be dynamic; a static plan is of no value to an adaptive organization in a fluid situation."

Summary

• James Bach asks, "What if it [the plan] comes into existence only moments before the testing?" Why must the plan be created so very early in the project, when our knowledge is typically at its minimum?
• In adaptive planning we plan as much as we can (based on the knowledge available), when we can (based on the time and resources available), but not before.

  Key Point

  The use of scripted testing does not preclude the use of exploratory testing. The use of exploratory testing does not preclude the use of scripted testing. Smart testers use whatever tool in their toolbox is required.

• Since planning is an ongoing process, it is better to think of a plan as an interim product based on the information and understanding known at the moment and always subject to revision as new information and understanding emerge.
• The use of scripted testing does not preclude the use of exploratory testing. The use of exploratory testing does not preclude the use of scripted testing. As Rex Black wrote, "Smart testers use whatever tool in their toolbox is required. No paradigms here. No worldviews here. No screwdrivers vs. hammers. Let's do whatever makes sense given the problem at hand."

Practice

1. In what areas could you use adaptive planning where you now use classical planning? With what benefit? What would the challenges be? Who would support you in this new process? Who would oppose your efforts? Why?
2. In what movies about the Marine Corps were the process of planning and the value of plans emphasized over action? Can you explain why?

References

Bach, James. "Exploratory Testing and the Planning Myth." 19 March 2001. http://www.stickyminds.com/r.asp?F=DART_2359

Copeland, Lee. "Exploratory Planning." 3 September 2001. http://www.stickyminds.com/r.asp?F=DART_2805

"IEEE Standard for Software Test Documentation," IEEE Std 829-1998. The Institute of Electrical and Electronics Engineers, Inc. ISBN 0-7381-1443-X

Planning. MCDP 5. United States Marine Corps. https://www.doctrine.usmc.mil/mcdp/view/mpdpub5.pdf