Innovation Games as a Market Research Technique
Because I've classified Innovation Games as a form of qualitative market research, it is appropriate to take a step back and briefly discuss the broader topic of market research. I have a sinking feeling that some of you may have cringed a bit when you read that last sentence, because not everyone has had favorable experiences with market research. It is worth making the investment to read this section because a grounding in the basics of market research is important to getting the best results from using Innovation Games, and the market research process described in this section provides a solid foundation for using the games in other contexts.
You Know Something About What You Don't Know
Individual knowledge can be organized into three areas:
Figure 1.4. What You Don't Know
To illustrate: I know how to ride a bike, change a diaper, and plan, facilitate, and post process the results of an Innovation Game (and a few other things). I don't know many more things, from scuba diving to performing knee-replacement surgery.
But I can't tell you anything about what I don't know I don't know. I can't even tell you that I don't know it, because when I tell you that I don't know something, it actually means that I do know somethingI know that I don't know it. The point is subtle and is worth repeating: When you claim that you don't know something, you actually do know somethingyou know that you don't know it. Thus, any movement from complete ignorance moves you into a continuum of knowing. We communicate our place in this continuum through convenient shorthand phrases that are contextually motivated: "Yes, I know how to play chess" or "No, I don't know how to replace the transmission on my car."
By collapsing the continuum of knowledge, we can simplify the organization of individual knowledge into two dimensions:
Figure 1.5. You Know Something About What You Don't Know
Part of the power of Innovation Games lies in their capability to move you from complete ignorance into a state of knowing. From this state of knowing, you can choose to create a variety of innovative products and services.
As you read this, keep in mind that market research is an incredibly complex topic. This book makes no attempt at a detailed discussion of market research. I may not cover an item that you think is important. I may not cover an item you know a great deal about in sufficient depth. That's OK. My goal isn't to write the definitive reference on market research. Instead, my goal is provide enough of a foundation in market research so that you can leverage basic concepts to help you succeed whether you're a market research expert or an engineer who wants to better understand the people who are using her product.
For our purposes, it is sufficient to define market research as an ongoing process of finding answers to questions; the answers enhance your understanding of your customers, your markets, and your product and service offerings. Market research, whether sophisticated or simplistic, enables you to make better decisions with greater confidence. It is ongoing because you, your customers, and the larger product and service ecosystem in which you exist are not static.
 Note that this definition means that all uses of the gameseven when used in requirements managementare a form of market research.
Effective market research is
Systematic Planned, well-organized, with a goal and a method
Objective Minimal researcher or method bias
Focused On specific questions
Actionable The results obtained enable you to take action
The first letter from each of these words forms the acronym SOFA, and like a comfortable SOFA, effective market research provides a comfortable position for sound decision making. As I describe the use of Innovation Games, I'll discuss how you can play them according to these tenets.
A Market Research Process
The following is an effective market research process. Although it is simple, it is not simplistic, and you'll find that using it will produce much better results than ad-hoc planning.
Figure 1.6. A Market Research Process
Forming Questions and Preparing for Answers
The first step is determining the questions you want answered and what you will do with the answers. Both are included as part of the first step because it doesn't make much sense to ask questions if you're not committed to at least do something with the answers. Of course, the specific things you do, such as create new products, adjust current product road maps, or change existing products to better meet market needs, can't be predicted in advance. In this way, then, market research starts like a great vacation: you select both a destination and some activities that you'll do once you arrive, even if the activities you define in advance are as imprecise as "explore the terrain."
Determining the Kind of Data Needed
Your questions and goals strongly determine the kind of data you'll need to acquire in your market research process, much like your vacation destination strongly influences your choice of travel. Focused and precise questions, such as "Which color of blue is most preferred by my customers," motivate the gathering of data that is very different from questions that are more open ended, such as, "How does my customer see this market landscape changing over time?"
Acquiring the Data
This phase encompasses all the activities you need to acquire the necessary data, from planning your data acquisition method to putting it into action. Continuing with our vacation analogy, after you've selected the destination and choice of travel, you have to do the detailed planning and take the trip.
Processing/Analyzing the Data
After you've arrived at your vacation destination, you will probably have to unpack and get ready for your activities. Similarly, after you've acquired the data, you have to process it into a form that allows people to take action. This is one of more rich and complex areas of market research, and it includes many topics that are beyond the scope of this book. Fortunately, as you'll read in Part Two, processing and analyzing game-related data is a relatively simple process.
Taking action means putting your newly obtained understanding of your customers to work. In some cases, "taking action" may actually mean making no changes to your current plan, such as when you find that your product road map matches customer and market needs. In other cases, as described by Joan Waltman in her foreword, taking action may mean creating an entirely new product offering based on unexpected information, much like you might spontaneously change your vacation plans to attend a local music and arts festival that you learned about from the hotel concierge after you've checked in.
Primary and Secondary Data
Market researchers classify data according to the kinds of questions it answers. The key distinctions are primary data, which are data designed to answer your specific question(s) as well as possible, and secondary data, which are previously collected and published data that may or may not answer your specific question(s). (See Figure 1.7.) Examples of secondary data include census bureau data, which may help you broadly size your market, but may not help you determine how your customers use your product or what they might want in future products.
Figure 1.7. Primary and Secondary Data Table
What Kinds of Questions?
In addition to the general destinations described in this section, you might have a whole host of specific questions regarding your products and services. Here are various questions that you might have relative to your customers, your markets, and your products and services. We'll start with secondary data, because these are the kinds of questions that are often asked when considering the broad opportunity that you're exploring through your market research.
Questions best answered through secondary data:
Questions best answered through primary data:
What kinds of products could or should we add to our product mix?
Which color of yellow is best for our packaging?
Which member of the family has the greatest influence in selecting products?
Questions best answered through Innovation Games:
What features do customers want in the next release?
How do customers use our product?
How do my customers perceive the relationships between my product and other products?
What is my customers' definition of success?
Primary data, on the other hand, is data acquired to answer a specific question and is further classified as qualitative and quantitative. Qualitative data is considered open ended, is usually acquired through intensive human interactions, and must be interpreted. Examples of qualitative data include focus groups, interviews, and Innovation Games. Quantitative data is based on facts or choices and can often be acquired through less intensive and/or semi-automated methods. Examples of quantitative data include surveys and conjoint analysis.
Whom Do I Ask? Market Segmentation
Primary data is acquired directly from your customers, which leads us to the process of selecting the customers who will participate in your market research. Unless your market is extremely small, you'll have to segment your market and choose representative customers from one or more of the resulting segments. Market segmentation is a process of dividing a total market into groups consisting of entities who have similar attributes that help you accomplish some larger goal, such as sales, marketing, or, in this case, better understanding customer needs.
Why Statistical Significance Doesn't Matter in Complex Business Architectures
In his seminal work on innovation, Dealing with Darwin, Geoffrey Moore makes a powerful case that companies who serve complex-systems markets must rely on qualitative research to guide their decisions. As Moore explains, a "Complex-systems architecture specializes in tackling complex problems and coming up with individualized solutions with a high proportion of consulting services." Such companies are characterized by relatively small numbers of customers, and a small number of transactions with each customer, with each transaction costing hundreds of thousands to millions of dollars.
As Moore describes it, qualitative scenarios drive the research efforts of complex-systems architectures. From p. 39: "In the complex-systems model, market research has a qualitative bias because each customer constitutes a market reality unto itself. For example, the commercial airline businesses at Airbus and Boeing have perhaps two hundred or so primary customers worldwide to consider. Statistically averaging insights across such a modest customer population makes no sense. Instead, you want to delve deeply into the specific circumstances of each account, seeking out unique patterns, not mathematical correlations. This is where war stories and hypothetical scenarios, even just the occasional apt metaphor, can prove so insightful."
Innovation Games are explicitly designed to provide you with these kinds of insights through direct interaction with your customers.
Like other aspects of marketing, market research, and product development, market segmentation is itself a rich discipline. A market segment is just one way to slice a market, and choosing a meaningful slicing is important for your goals. A common way to segment consumers includes such things as demographic data, such as where they live, disposable income, or highest educational degrees, their purchase behavior, and their perceptions of the product and/or service being offered. Common ways to segment businesses include the kind of business, its size in terms of revenue or employees, the kind of business (corporation, sole proprietorship, LLC, and so forth), rate of growth, and markets served, to name just a few.
You may find a marketing and sales bias to your existing segmentations. This is understandable; many times the most important aspect to segmentation is helping the marketing function size the market and find leads to hand over to sales. This approach, however, may not yield the best results when considering whom to invite to an Innovation Game. Many times you'll obtain more true insights by considering alternative approaches to market segmentation. Here are some ideas to get you started:
Experience with your product
Knowledge of the domain
Perceived motivation to use your product
Strategic importance to your firm (such as by revenue)
Figure 1.8. Experience With Product
Figure 1.9. Knowledge of Domain
Figure 1.10. Perceived Motivation to Use Product
The key objective is finding the people who are most likely to provide you with the best answers to your questions. You may find that the customers who pay you the most money are not necessarily the ones who have the best handle on how your product needs to evolve to meet future needs.
After you've finished your market segmentation, you'll still need to select specific customers. This aspect of the process is covered later in Part One under the detailed notes of preparing for the game. Keep an open mind as you consider your current market segments, as playing the games may provide you with insights that motivate changing them.
Questions, Data, Answers, and Actions
Acquiring data is not the same as having your questions answered. One important benefit to using Innovation Games is that they can offer considerably more information: you can get the answers to your questions and you can make new discoveries about your market. For example, in one of the more novel applications of Speed Boat, I was asked to help a major law firm in Silicon Valley identify the root cause of associate dissatisfaction. In the process, we acquired data about specific issues that contributed to associate dissatisfactionthe data that helped answer the specific question. We also acquired data far beyond the original question, such as the associates' ideas on how the law firm could better utilize IT resources and share information about the firm, all of which created new opportunities and possibilities.
Whom Should I Invite? Whoor Whatis a Customer?
One of the long-running debates in the field of product development is how a given product team should define a customer. When defined too narrowly, teams lose the opportunity to expand their market with new innovations or improve existing products by better understanding complex relationships. When defined too broadly or haphazardly, teams can become overwhelmed trying to make sense of data that simply does not apply to their situation. At times, emotions can get the better of common sense, with everyoneespecially the customerlosing in the process. That said, it has become clear that this is an important issue and that some guidance is important.
One source of tension is that product teams struggle to capture the meaning and impact of direct and indirect customers, often at the same time. By distinguishing between these two kinds of customers, teams can gain a much richer sense of overall market needs. A direct customer is any person or system that directly uses or consumes your product or service. An indirect customer is any person or system who is or will be affected by your product or service.
The chief benefit of this definition is that it challenges you to think broadly when considering which customers you should include when considering a game. It also is practical. If you're trying to understand operational issues, deal with customers who work in operations. If you're trying to understand financial/ROI issues, get access to the people who pay the bills. If you're trying to understand the effects of slow system performance for airline reservation software, ask the airline customer who has to wait in line while the ticket agent changes his reservation. In this case, the direct customer of the software is the ticket agent, and the indirect customer is the passengerwho just happens to be the direct customer of the airline!
Don't be misled by differences in usage profiles. Some customers are power users, people who use your product or service every day. Others might be occasional or episodic users, who use your product only when motivated by certain needs or circumstances. What are those needs? Those circumstances? Different people will use different capabilities or features. Which? When? Why? All these people have a right to be heard, although you should certainly feel free to assign weights to what they've said. Although it is important to select a reasonably good set of customers to play the game you've selected, you do not have to get some "perfect" set of customers together before you can get started.
Having answers to your questions does not always mean that you will take the actions it suggests. Continuing with the example of the law firm, certain actions were incongruent with the larger culture of the firm and were not taken. In terms of the vacation analogy, the law firm had a specific destination in mind and the intent to act on the data it provided. However, after they arrived at that destination, they surveyed the possibilities and chose some activities, but not others.
It is common, and often recommended, to use a combination of market research approaches to gain the confidence to take a specific course of action. Suppose that while playing Product Box to answer questions about marketing messages for an existing product, a customer identifies a new product opportunity. Before building this new product, you might consider engaging in other kinds of market researchfor example, secondary market research to help determine the size of this market, and additional forms of primary market research to determine how much people might pay for this product.
The Unique Benefits (and Drawbacks) of Qualitative Market Research
Each form of market research has its own strengths and limitations. Table 1.3 presents some of the strengths and weaknesses of qualitative market research.
Table 1.3. Strengths and Weaknesses of Qualitative Market Research
Generates deeper understanding through contextual, multifaceted, verbal/nonverbal communication.
By definition, is less objective than other market research methods. We account for this in Innovation Games by using facilitators, multiple observers, and post processing results as a team.
Can strengthen customer relationships, especially in B2B and B2P markets.
Does not scale to large numbers of people.
Builds customer empathy within the team doing the research.
Is not statistically significant (see the sidebar "Why Statistical Significance Doesn't Matter in Complex Business Architectures").
Creates vivid, concrete language and commitment to solve customers problems.
Relatively costly on a "per customer" cost basis (but often relatively cheap on "actionable results" basis).
Forms the foundation of innovation by letting you explore "what you don't know you don't know."
The strengths of working directly with customers through qualitative market research strongly outweigh the weaknesses. Simply put, qualitative market research (finding ways to interact, experience, and collaborate with your customers) is the strongest foundation for creating truly innovative products and services.