Partial Deployment

In partial deployment, our line of questioning shifts from "Does the system work?" (because we know it does) to "How can we improve it?" Partial deployment is basically the same thing as pilot deployment ”only this time the system takes more calls over a longer period of time. How much time? It varies, depending on the client and the application. Some applications are so complex that they need to be rolled out very slowly over a period of months, while others may be so small and simple that they can be fully deployed in a matter of days. Because we're talking about a much larger volume of calls, more of the analysis will be performed by software tools and less by listening to individual calls.

Unlike pilot deployment, where we focus on tuning up all the severe problems, partial deployment focuses more on trends such as periodic equipment failure (a bad hard drive?) or subtle shifts in callers ' usage patterns. Because partial deployment is closer to real life usage, we're able to monitor system performance under a wider range of conditions, including times of high call volume. We can see, for example, how promptly and efficiently calls are handled when the computing power and phone lines are reaching their limits. If things are going well at this point, we may be tempted to simply open the floodgates and go straight to full deployment ”but that's a temptation generally to be avoided. We don't want our entire calling population to become totally dependent on a system that may not be quite ready for all of the people, all of the time.

For example, perhaps we could make the calls go faster by cutting out any unnecessary silence (dead air) in the prompts. A few seconds of dead air may not seem like much, but after ten million calls they can add up to some hefty ”and totally unnecessary ”long distance charges. Some subtle improvements in deployed systems have been known to contribute to a savings of one million dollars a year ”accomplished simply by editing some of the prompts!

We also keep our eyes open for more subtle trends, such as trying to see if callers are exhibiting repeated behaviors that we can capitalize on for improvement, speed, and elegance . So, for example, if most callers who call in to a credit card company ask for a particular piece of information every time they call (like their credit card balance), the design could monitor the callers' performance and change its behavior to satisfy these callers. Instead of forcing them to ask for that information every time they call, the system could automatically offer to play that information at the beginning of every call.

When we go from pilot to partial deployment, we go from analyzing hundreds of calls to analyzing thousands of calls. The added volume can often reveal failures that didn't appear in the pilot phase. Why? Because the more calls we get, the greater the chance that groups of people will say something the system doesn't recognize.

The United Airlines Baggage Desk is an example of an application where the only way to improve it was by analyzing lots of calls. In order for the system to provide status information for a lost bag, it asked the caller, "Where did you file your claim?" In partial deployment, we discovered that, although most callers gave an expected response (for example, "Chicago O'Hare" or "L-A-X"), a small percentage of callers replied in a completely logical, valid, but unfortunately unanticipated way, saying "At the airport."

We saw two solutions to the problem.

1. We could change the question to "At which airport did you file the claim?"

2. We could program the system to respond differently when a caller said, "At the airport" by following up with another question: "What's the name of the airport?"

Both solutions would probably work. So which one did we select?

We ended up choosing the first option, and here's why. While sometimes changing the wording of an existing prompt may cause confusion among current users, it's worth doing if the system is relatively new. Also, changing how this program worked (adding another turn to the dialogue) would necessitate more development work and quality assurance testing, which could delay the launch of the system. And in this case, changing the call flow wasn't the best solution, especially since modifying just one prompt was all that was really necessary.