Product Development Processes: Creating Release 1.0

The product development process is the biggest big-picture perspective on the product. My experience is that software development processes, from waterfall to spiral to XP, are each more effective when considered and conducted in this context. This section presents a conceptual foundation of a successful product development process associated with a first release. In the next section, I will show how this foundation can be used to deal with subsequent releases.

The process presented in Figure 2-1 organizes product development in stages. These stages, and some of the product management and engineering activities associated with each, are presented to the right. I'm focusing only on product management and engineering. Other activities such as finance, legal, quality assurance, and technical publications , are clearly important to the overall product development process, but a detailed description of their involvement is beyond the scope of this book.

Figure 2-1. Product management and engineering processes

As you review these steps, keep in mind that the actual sizes of the product management and engineering teams are quite different. A product management team is usually very small when compared with the total engineering/development team. I've worked on products where two to four product managers were easily able to keep 30 to 40 developers busy. Some of my friends report even greater variability. One friend told me about a product manager who was able to keep a team of 50 developers satisfied.

Another key difference concerns the distribution of work. In the early phases of the project, product management may be perceived as doing more work than engineering. This is partly because the product management team is producing more tangible results, but mostly because, in a well-run product development process, the early documents created by the product management team are the "high stakes" ones. Many decisions, including those that will affect the product for many years , are being made during the development of these early documents, most notably the business plan. Taking care to create good ones is worth the results.

Bright clever ideas form the foundation of the initial product. The best of these fill us with enough passion to take the considerable leap necessary to create a product. Some companies actually formalize this aspect of the overall process in a subprocess called ideation.

Concept Proposal

The purpose of the concept proposal is to establish sufficient motivation for producing the product. Usually created by a small team, the concept proposal includes enough business data to justify the market and enough technical data to justify feasibility. If neither of these conditions is met, the project is gracefully terminated , in a process I describe in greater detail later in this chapter.

Product Proposal/Business Plan

The product proposal/business plan is the key document created by the team to justify the product. It outlines a number of important elements so that the business can be certain that the project is justified. It is so important that I will discuss it in depth later.

Note that during this phase of product development it is not uncommon for engineering to be doing literally nothing while marketing is creating the business plan.

Development Plan

After approval of the business plan, the two teams begin the highly collaborative development planning. In this phase, the primary responsibility of marketing is to clarify and prioritize market needs, expressing them as desired features of the target product. In turn , the primary job of engineering is to analyze the dependencies within these features, identify necessary architectural capabilities, create some crude estimates of time required to complete various tasks , evaluate technology solutions, and begin to identify needed resources. Marketing should also take primary responsibility for including other teams, as needed, in the development of the product.

The results of this process can range from a formal marketing requirements document (MRD) to an informal set of user stories written on cards and taped to a wall. Indeed, in one system I managed we generated requirements by simply directing questions to a very experienced member of the team, secure in our knowledge that this person would answer each question with unfailing consistency. This was an unorthodox way to obtain and manage requirements, but it worked great. It illustrates that the development team should let the size of the project, the size of the team, and the complexity of the product dictate the formality of the development planning process. Regardless of its ultimate form, the true requirement of a requirement is that it be expressed with a sufficient level of clarity to enable the development team to egage in useful work.

At this point the development team will also create whatever analysis, design, or other predevelopment artifacts they feel are appropriate given their project and their chosen development process. The challenge is, of course, in determining the artifacts needed for success. Sometimes informal processes and artifacts work great, as when a development team sketches the design for a new user interface on a whiteboard using an existing system as a model. At other times formal processes and artifacts are required, as when a development team is working on a hardware device and must create a solution within a narrow set of weight, power consumption, and size constraints. Just about every development team should be conducting some experiments based on the continued advancement of various Web-based collaborative technologies. It's surprising how easy it is to improve communication within geographically distributed teams with just a Web site and a digital camera!

A useful description of design documents, and the value they provide to the team, is found in Agile Software Development [Cockburn 2002]. Alistair refers to design documents as the residue produced by the development team as they create the system. The goal is to create the minimum amount of residue needed for current and future success. I've found this perspective useful, as it helps me determine if a given design document is worth creating. That said, you must take care in your definition of "development team." My own definition explicitly includes technical publications and QA. These teams can both contribute to the development planning process and benefit from their inclusion in it.

Disciplined development organizations usually hold a final review meeting at this point before development begins. The purpose of this review is to assess whether the product can be created within time and cost estimates and to kill the project if it cannot. While it is relatively rare to find organizations killing projects after this stage (most are killed after the concept proposal; many, after the business plan), a well-run product management organization will stop work when it becomes clear that the product cannot be created in a way that will make it a winning solution.

Development

The next phase of the process deals with actually building the system. I find this stage interesting because product management now focuses its attention on what happens after the product is releasedbefore it is even built. Engineering, on the other hand, is actually creating the product. There are a wide variety of options, here, including traditional development methods and the newer agile methods, such as XP, SCRUM, and Crystal, but since many authors have written a great deal about development and development processes, I won't go into the details. What I will say is that you should adjust your development process according to a variety of factors, including the size of the team and its geographical distribution and the nature of the product. A complete discussion of how these, and other, variables can affect the team and its processes, can be found in my book, Journey of the Software Professional.

Modern development practices mandate that working systems should be delivered in reasonably well-defined pieces that cascade or overlap. Specifically, development does not happen "all at once." Instead, it is staggered or overlapped so that working versions of the system can be delivered to QA, technical publications, alpha testers, key customers, and other important stakeholders. One example of overlap is when a working system is given to QA to perform initial testing and to refine final test plans while the development team continues to work on the next set of deliverables.

Final Quality Assurance

The next key phase of the development process is final quality assurance, which comprises a range of activities, each providing a different kind of business value. The foundation is testingthat is, checking that the system created reasonably matches the requirements and providing the product manager with data to help her determine if the system is good enough to ship to the customer (see Appendix C for a thorough discussion of "good enough" software). These data include bug trends, bug reports , workarounds for known problems, and so forth, which the product manager uses, along with other data, to make the ultimate business decision: when to ship and to whom. This decision should be made in collaboration with others involved in the development process.

I've been fortunate to work with world-class quality assurance managers. Building on the solid foundation of testing, they understand business strategy and can actively participate in the ship decision. If you're similarly fortunate, learn to include QA's point of view. If not, try to educate your team on the business issues surrounding the productin the long run you'll get better results.

Quality assurance teams can contribute to development areas beyond testing, such as the following:

• Monitoring and, at times, enforcing development process agreements

• Collecting and publishing testing and nontesting metrics

• Assisting developers in root-cause problem analysis (some problems only appear during testing)

• Helping support and services organizations understand workarounds and/or replicate customer issues

• Managing the source code system and the daily build process, especially when the build process is highly dependent on automation

• Assisting developers in proactively designing the application to make testing easier and/or helping to choose designs that are likely to be more correct

• Participating in the requirements management process, including testing the requirements

Participation in these areas is not concerned with final quality assurance, which is only one part of product development. Rather, they represent the broadening of QA's role and its inclusion in other development processes. Ultimately, QA participation in any of these activities depends on the specific skills and experiences of the QA team.

Since modern development requires QA involvement, you might argue against the need for final quality assurance. This sounds good in theory, but every complex system that I've worked on has required final quality assurance before delivery to a customer. Instead of trying to eliminate final quality assurance through developer-centric automated testing, leverage this testing with final QA. The net result is higher quality.

Prelaunch

On or near the successful completion of the system, the development process moves into the prelaunch phase. Depending on the estimated time of final quality assurance, prelaunch can happen in parallel with this phase or after. One way to characterize this phase is that the work of the engineering team begins to decrease while that of product management begin to increase. The engineering team prepares to hand its work to services and support, prepares the source code for maintenance and future development, and often takes a short break to catch up on sleep. Product management, on the other hand, is often a whirlwind of activity, doing a variety of things to ensure that the product is successful in the marketplace , including everything from preparing sales collateral to meeting with key analysts to make certain they understand the new product.

Beware of QA Automation Blinders

I'm a strong proponent of the agile development methods, especially those such as XP, that place a heavy emphasis on test driven design. In this approach, developers create test cases before they write any code. As the code is created it is constantly run against these test cases. By the time you've released the system the typical team will have created several thousand test cases for a moderately complex system.

Unfortunately, some people believe that test driven design and extensive , automated tests make the need for final quality assurance obsolete. This is a dubious claim, and as the complexity of your software system increases so, too, does the need for final quality assurance for several reasons, including the inevitable blinders that QA automation puts on your developers.

To illustrate , one of the products I managed protected software from being illegally copied by binding the software to a given machine through a "hardware fingerprint." The exact composition of the fingerprint is proprietary, but it includes several physical machine parameters that can collectively identify a machine while allowing the end user to modify several of them (e.g., the processor id or the amount of physical memory). Changing just one or two of these won't invalidate the fingerprint , but change enough of them and the software will stop working to prevent piracy.

One of the values used in the fingerprint is the MAC address of the primary Ethernet card. In a Macintosh, this is supposed to be built-in, so my developers created a series of automated tests to ensure the MAC address was always present. They never thought of actually removing the NIC card because they had automation blinders on. Our QA team was able to crash the program in about 10 minutes by simply removing the NIC card and running the automated tests, primarily because they were not wearing automation blinders. More generally , the process of creating and testing the machine fingerprint is characterized by a rather large number of tests that cannot be automated. In these cases, final QA is essential because they aren't blinded by automation, and they were willing to take the time to create the necessary physical configurations needed to properly certify the product.

There are other equally important reasons to engage final QA. Multiplatform development requires a careful allocation of resources. What I've found works best is to have the development team create and test the software on the two or three most prevalent or most important platforms. Final QA then confirms the operation of the software on the rest. This approach saves time by saving developers from the inevitable setup and teardown costs associated with loading and testing the program on each platform, and it saves money by allowing the development organization to make a significant investment in a QC lab that can truly represent customer environments.

The importance of security in computer systems is yet another strong motivation for final QA performed by a separate organization. It is very hard to thoroughly test your own systems for such security loopholes as buffer overflows or inappropriately modifying system resources in a way that leaves them vulnerable to an attack. It is far better to have a separate QA organization perform security tests.

Launch

While engineering often celebrates the creation of the "golden master," product management usually waits until a launch event. This may be as simple as the official press release or as complex as a special event managed by a public relations firm. A final, but critical, step in the launch phase is some kind of customer-focused monitoring and feedback. When engineering returns from the party, they should be ready to address issues and escalations coming from the field.