Analyzing Competitive Research

The goal of traditional competitive analysis is to create a side-by-side, feature-for-feature comparison grid. The assumption in creating such a matrix is that point-by-point matching of features is important for competitiveness. This isn't always the case from the user's perspective. There is usually a core set of functionality that needs to be similar for two products to be considered competitive, but beyond that the actual feature set can vary broadly and two products can still compete. Many online companies compete directly with brick-and-mortar companies, for example, though the actual user experiences are drastically different.

Competitive user experience research should reveal the fundamental strengths and weaknesses of the competition, not create a scoreboard.

The comparison begins with the research reports you produced in the data collection phase. Taking the report, enumerate the advantages that the competition's product gives their users and the hindrances it places in their way. Again, these should be from the users' perspective. Even though it may be in your best interest to have people stay on your site longer, a site that lets people get in and out quickly may be preferable. Similarly, although new daily content may be positioned as an advantage of one product, it may make little difference to users, and, in fact, a frequently changing page may be seen as a hindrance if it requires them to relearn the navigation.

Looking at the results more closely, what do they say about the company and its development process? Where does it provide consistently strong experiences? What are the characteristics of those experiences? How does it regularly falter? Business analysts call the collection of a company's strengths their portfolio of core competencies. Likewise, it may be useful to maintain a portfolio of frequent foibles that lists consistent weak points in the competition's approach to creating experiences.

CNET, for example, were masters of brand identity and design cohesion. With all of the distinguishing marks removed, people who had seen a CNET site could immediately pick out a different CNET site from a lineup. Users considered this to be a benefit because they trusted the company and felt comfortable navigating its products. However, CNET's navigation was less consistent than its branding. When researched competitively (by me while working for Wired Digital), its users felt that they knew how to use a CNET site because it resembled other CNET products, but they actually had more trouble navigating some CNET sites than ones made by CNET's competitors. When researching a number of CNET sites, it was seen that both the strong branding and the bad navigation were consistently present in many of their products.

Traditional competitive analysis can, of course, also be revealing. It shouldn't be the only kind of analysis, but it can provide a useful shorthand. If, for example, users clearly value a certain set of features, then it may be appropriate to create a feature comparison matrix with those features. Such side-by-side comparisons, however, need to be between actually comparable items. Thus, a site map somewhere on a site should probably not be compared with one that has a map front and center on the homepage.

Benchmarking

In the interest of creating a shorthand by which to evaluate competition, some competitive analysis techniques introduce numerical or letter grade competitive ratings. You can, for example, give each feature in a feature matrix a score—say, 0–5, where 5 means the product does something well and 0 means it doesn't do it at all. Adding up all the scores gives you a total for the competitiveness of the site. This allows you to quickly see which ones are, theoretically, most competitive because relative to the others they are the most successful at what they do.

Such techniques are useful when new competitors are constantly entering the field and you need to evaluate them quickly, but it obscures important information about which aspects of the competition influenced the final score, which may actually be more important than the overall value. However, the exercise can be useful when figuring out which strengths are serious threats and which weaknesses are important opportunities.