Hatching a Catastrophe


My colleague Scott McGregor sent me the following story when I asked him if he knew of any cases where development projects got out of control because of a lack of design. His story is a tragic one, made ever more so by the fact that it is so typical of our industry.

Scott is a very talented man, as his well-written story attests. He is also a skilled designer with a fine pedigree academic and practical in both software engineering and design. He joined a small, venture-funded start-up company in Silicon Valley. The original founders of the company also had well-established credentials, including several successful years at Apple. Scott invited me over one afternoon to meet the founders and to pitch my company. The CEO and the vice president of engineering showed our team what they were working on, and we were impressed. The product idea was excellent. It exploited an aspect of productivity software in a very clever way. The product was based on an appropriately small amount of good technology that served a very real marketplace need.

They had everything they needed to succeed except for design. Here is Scott's story in his own well-crafted words:

Our CEO said we would beat others because we were so fast and spry, and he went on to recommend with pride that we were following a "ready, fire, aim" strategy to reach success before other companies were even done aiming. Of course when we fired, it was obvious that we had shot ourselves in the foot!

Although we met our schedule of delivering our 1.2 release on December 31, we did it by saying that Release 1.2 is whatever is done at 5:00 p.m. on December 31. There was no fixed spec that engineers were building from. Substantial new changes showed up for the first time without any prior warning on December 29.

Earlier, I had suggested that we needed to follow a design method. I said that we should begin by identifying who all the key users and other stakeholders were and write up profiles on them, and then develop statements of their goals and the tasks they would go through to accomplish those goals. Then, from those tasks, I suggested we could come up with proposed visual representations of key objects and interaction behaviors. Once we had done this, we could start building it.

Unfortunately, our management felt all that was a luxury we couldn't afford. We didn't have the time for that. Instead, we visited many customer prospects, and the CEO would relate our grand vision. People loved it, but they wanted to know specifics. But each prospect had a different axe to grind in the specifics. One wanted a product for their sales people, another for independent resellers, a third for customers. One had many documents they wanted to manage, another was interested in Web pages, etc. With each customer contact the definition for 1.2 grew, becoming the union of all imagined product features.

Even more unfortunately, prospects would talk about some new thing they wanted, but didn't talk about features they already had in their existing software or their browsers and which they took for granted. Since these weren't talked about they were never added to the product spec, and they were never built.

Our newly hired vice presidents of sales and marketing could not get the product to install on their systems for weeks. When they did get it to work, it corrupted or lost all their data several times a day. Performance continued to degrade. In a demonstration with no more than 100 data views, performance was acceptable but slow, and that is all the developers ever tested. But real-world use called for over a thousand views, and performance was downright snail-paced.

There were three major screens in the product, but to simply edit a document required jumping between each of the three screens several times. Many single tasks required the user to make a dozen mouse clicks with windows coming and going, and lots of switching between the mouse and the keyboard.

In the end the product was impossible to learn, was unusable from a cognitive and performance standpoint, had poor reliability and destroyed data regularly. Although it was full of lots of "differentiating" features, it was missing necessary basic features that were standard in all other competing products.

As might be expected, by the end of February, the board took action, and the CEO and VP of engineering were forced to step down.

This is of course just a single anecdote. It might appear to be an isolated incident, except for the fact that this has happened repeatedly at companies where I have worked over the last 20-plus years.

One thing I've noticed is that you get what you measure and reward. Prior to January, the only measures that our board had were dates and the features promised. They never set out measures for minimum quality (e.g., mean time between data corruption, crashes, etc.) so quality was sacrificed. There were never any measures for performance (e.g., number of seconds between the key press and something happening) so these got arbitrarily long. There were never any measures about how long it would take to learn something, or how often a user could work without an error, so learnability and usability were sacrificed. But the things that were measured the schedule and the list of features were achieved, and because there wasn't a full description of the features, many of them were achieved in name only.

Scott is highlighting a fundamental truth: You can "expect what you inspect." If you repeatedly point to the calendar, your project will be on time, and if you ignore user satisfaction, your users will be trod upon. Scott continues his narrative:

Investors repeatedly say, "We don't have enough money to spend on building a product we can't sell, so we have to know our customers before we start." Yet engineering managers seem to constantly believe that "We don't have enough time and money to spend designing we could design forever and run out of money before we had a product to build." So in the end, they keep building new products rather than improving the design until the money runs out.

Looked at with some detachment, the past few months seems sort of like an old screwball comedy movie, or a soap opera without the romance. And I can appreciate it all as such. Of course, if I only looked at it that way, I wouldn't have bothered to go into it in such detail. However, I am passionate about this. I feel there is a moral imperative to stop wasting people's lives in useless activities.

In About Face you wrote how important it is to stop wasting users' time. I wholeheartedly agree. But that is just the tip of the iceberg. That only happens when products actually get to market and are purchased. But there are many more projects that are canceled before they get to market or which fail to be purchased. Every engineer I've ever met cared deeply that the products she worked on might never be used. But when the resulting product was canceled or failed due to lack of design it meant that their effort was wasted. And the world doesn't have so much of these skills that it can afford to waste them. That's the moral imperative that I see, not just "stop wasting users' time," but stop wasting everyone's, including programmers, time and lives.

It was very painful to watch in a Cassandra-like role foretelling the impending doom, but unheeded, and watching opportunities to avoid the doom pass. I've come to the conclusion that operant conditioning is so powerful that it allows a person properly conditioned to be impervious to reasoning based upon facts and figures.

I want to emphasize that Scott's experience is not atypical. Here is a story from another industry colleague, John Rivlin. John runs a small but very successful software design and development firm in Palo Alto, California. He sent me this story:

We always do detailed product designs prior to beginning any software project. This particular case is no exception. We started the project by creating a 15-page spec outlining the user interaction of the software we were proposing to write. It included the overall assumptions of the project so that we could move beyond the initial one-sentence description. This is important because we work on a fixed cost estimate where we assume the risk.

The client's development manager running the project concurred with the notion of writing a specification, and we agreed on a fixed cost to do so. The specification was then completed and submitted to the development manager's boss, the chief technology officer. The response we got was "Why have you spent so much time writing a spec? You have used up a major piece of the budget. We don't write specs here. We just go off and get the job done." Upon further examination, it also became clear that his assumptions about functionality were significantly different than those of his development manager. This discrepancy was only made visible by the "wasted" specification, but not even this persuaded him that designing software is a good thing. This is the CTO of a publicly traded technology company with annual revenues exceeding $100,000,000. The inmates are truly running the asylum.

Although the fear that many development managers hold for design is irrational, it is often based on very real personal experience. Previously, in the quest for better products, managers have asked programmers to design interaction, and the results have been painfully unproductive. Anyone untrained in interaction-design methods tends toward self-referential design, in which one imagines himself as the user, and programmers naturally fall into this trap. Any group of people designing self-referentially will have a devilish time getting to closure on issues because they have no firm, common ground about users, and the process drags on interminably.



Inmates Are Running the Asylum, The. Why High-Tech Products Drive Us Crazy and How to Restore the Sanity
The Inmates Are Running the Asylum Why High Tech Products Drive Us Crazy &How to Restore the Sanity - 2004 publication
ISBN: B0036HJY9M
EAN: N/A
Year: 2003
Pages: 170

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net