Behaviors: Planning Versus Reactive

One interesting theory about biological intelligence is that even humans do little planning [Heuer99]. Neither animals nor insects are notorious for deliberating what to do next. In most situations, instinct makes it possible to act intelligently just by reacting to external or internal conditions. This is how evolution has allowed us to survive.

Although it can be demonstrated that humans can actually plan, this ability is certainly used less often than expected. Recent research has shown that even playing chess or go is more a matter of intuition [Chen02, Atherton02]!

Goal-Oriented Planning

Theoretically speaking, what's the difference between planning and just reacting to certain situations? The obvious answer is that planning actually thinks ahead.

Planning is generally seen as a nondeterministic process, also known as deliberative algorithms. From a given situation, multiple actions can be taken depending on many factors, such as the current goal. A planning algorithm can scan through the possible options and find the sequence of actions that matches the goal.

In essence, planning is nondeterministic because of the goal. But what if the goal is part of the situation? This is like modeling all the "hidden" factors as additional inputs. Then, it's possible to use reactive techniques to achieve exactly the same results.

Using reactive techniques to compute plans in such a way is known as reactive planning. This differs from standard deliberative planning, which generally uses search algorithms to scan through all the possible options. Generally, when planning is mentioned, a search algorithm is often implied.

Figure 3.3. Planning through a set of situations to reach a predetermined goal.

Planning in Games

A wide variety of techniques exist to build plans for different situations. However, theoretically, there is no difference between planners and reactive techniques. It's just a question of expressing the problem differently. Therefore, justifying the use of planning by claiming that reactive techniques cannot handle goal-oriented behavior is provably false.

So, is planning actually needed in games? Of course. In practice, planning is well-suited to particular types of problems. This is notably the case with a large number of possible goals, because modeling them as additional inputs increases the number of input configurations dramatically. Here, approximating the entire problem in memory with reactive techniques is not always possible; instead, planning uses more computation rather than memory to search a solution.

This is the case with movement. Game developers have found that so-called pathfinding is best done with planning techniques. These algorithms essentially search the terrain for the best possible path.

In this example, the planning component actually uses knowledge of the terrain; this is called a world model. As discussed, it's possible to use reactive techniques to plan based on this world model. However, generally, reactive behaviors use little memory (no state), so a component that contains a world model would not theoretically be classified as reactive.

Reactive Substitutes

However, there are surprisingly few cases where planning is actually necessary. In many situations, reactive decisions appear very intelligent especially high-level tactical decisions. Such instinctive decisions are surprisingly close to the way human experience works in most cases. To use reactive behaviors rather than deliberative algorithms, it's necessary to use properties of the situation as input, and use the output directly as a decision.

In fact, even movement can be handled with reactive behaviors. The problem can, for example, be treated as one of obstacle avoidance. Then, using a combination of information about the obstacles and the target, many scenarios can be handled. Usually, this approach is not as robust as the planning approach, because pathfinders use global knowledge of the terrain, and not only local obstacle information. A reactive planning approach, however, would be very competitive with a planner based on search.

Generally, however, reactive behaviors are extremely robust solutions. When combined with learning, they can significantly reduce the need for planning. When deliberative algorithms are needed, they are often enhanced by reactive techniques both in performance and efficiency.

So, although the general use of planning algorithms is debatable, reactive techniques as discussed in this book have an undeniable place in games and AI.