|  A working product at all times should be the ultimate goal for every software team. The further a product is away from being in a working state and the longer it is in that state, the greater the time required to get the product back to a working state.
Software products need to be in a working state at all times. This could be called virtually shippable because the product could be shipped after a short period of verification testing and stabilization, the shorter the better. A working product does not mean that the product is feature complete or that all features are done; rather, that it can be shown to or given to a customer for use with the minimum possible number of unexplainable errors. The closer a team can get to continually keeping its product in a working state, the better the chances that its efforts can be maintained over time because of the level of discipline required and also because having a working product imparts greater flexibility and agility. Why Working Product and Not Working Software? I use the term working product and not the more common phrase working software deliberately. This is because from the customers' point of view, they rarely receive just software. Software is almost always accompanied by explicit deliverables such as documentation plus supporting infrastructure like a web site for information, updates, and support. There are also implicit deliverables such as an expectation that there has been testing before the customer sees the product. Hence, I use the term working product to encompass all the explicit and implicit deliverables and to reinforce the requirement for multifunctional collaboration within a team where software is one important part, but not the only part. |
A working product is an underpinning of agile software development. Agile development demands the ability to get continual customer feedback, and the only way to get feedback on features and to keep customers interested in working with the product is to provide the customers a working product on a regular basis. A working product is a demonstration to customers that this is a product they can rely on. It is extremely difficult to keep customers interested in using a product if they develop a low opinion of it because it is unreliable, crashes, and has unpredictable problems and regressions (where features they depend on no longer work). Regressions are particularly dangerous, and one important aspect of having a working product is that nothing should be added to the product until the regression is fixed. A working product also gives the product team an incredible amount of flexibility, not only in being able to change directions, but also in taking the latest version of software and shipping it at any time with the critical feature(s) the market demands. This flexibility translates into a sense of power for the development team derived from feeling in control and using that control to respond to changing situations. The working product principle is a deliberate contrast to the traditional code-then-fix method of software development. This method, which is probably the most commonly used method of software development, is the practice of developing a feature and then debugging and altering the code until the feature works. When software is developed in short iterations of two to six weeks as in agile software development, the temptation is to believe that it is acceptable to apply code-then-fix in each iteration. However, the problem with the code-then-fix method is that teams are forced to waste time and effort debugging, effort that should be applied to satisfying customer problems. Wasted effort is a lost opportunity. Hence, having a working product every day and fixing defects as you go minimizes wasted effort and allows teams to focus on their customers. It also dramatically reduces the buildup of technical debt in the product, because chances are very high that problems are caught as soon as possible. As explained in Chapter 5, this requires a change in mentality about software development: All attempts must be made to prevent defects from reaching people, whether they are testers or customers. In order to keep a product in a working state, teams need to focus on the quality of everything they do. Achieving this level of product quality requires a great deal of discipline and focus on the part of the development team: focus on the quality of the product and the discipline to ensure that every software modification passes an extensive suite of automated tests, or it is removed from the product until it does pass the tests (see Ruthless Testing in Chapter 5). Whenever a product is in a non-working state and changes are still being made to it, the development team is literally flying blind and technical debt is accumulating; a debt that will have to be repaid later. This situation is a recipe for disaster because the cost of fixing problems increases with time, as does the probability that multiple problems will show up in the final product and be found at customer sites. The Need to Address Quality Promptly If your car gets a flat tire, you know that if you keep on driving without fixing the flat, the rim will need replacing, and then the axle, and then various body parts as they are ground down through contact with the pavement. Eventually, your car is going to need major repairs, and even when you do get it repaired, it will never be the same. Most people wouldn't even consider the option of not changing a flat tire on their car. Yet with software teams the severity of a problem and its consequences are not so obvious, so the temptation is there to just leave it even if it leads to more problems down the road. In Lean Software Development [Poppendieck and Poppendieck 2003] the analogy is drawn with an assembly line used in a manufacturing plant. If a problem is found with what is being produced on the assembly line, a big flashing light goes off and production stops until the problem is fixed. Software teams may hate having to stop their work, but addressing quality concerns promptly is a critical part of keeping a product in a working state while minimizing the effort required to keep it in that state. |
A useful conversation to have with a product team is around the question What would it take for us to ship our product every day to our customers? In this case ship could mean providing daily updates or a complete new version of the software to install. The purpose of answering this question is to get the team thinking about minimizing the overhead activities that occur after the software is declared complete and can then be shipped to customers. The answers should be enlightening, because they will highlight not only all the changes the team will have to make but also the ripple effect this would have on the team's surrounding organization. Here are a few of the topics that might come up: Quality assurance and testing become critical issues. How are you going to keep the level of quality high in your software? If you are going to ship every day, a person can't do all your testing. You're going to have to automate as much testing as possible, and this likely means designing the product and product features for testability as opposed to thinking about testing after. And where people are required for testing you're going to have to ensure that they are available when they are needed, not working on another project. How much validation testing must take place on the final version of the product before the team and its customers have enough confidence in the product that it can be fully deployed? How many of the current validation tests could be automated and run every night or on a more frequent basis so that the testing is essentially done much earlier? Is your documentation and web site going to be ready to reflect the new changes every day? Most likely you're going to have to work out some automated way of updating the web site while ensuring that product documentation can be loaded off the Internet and not just the local hard drive (i.e., updated live). What happens when a customer finds a problem? How is he going to report it to you? How are you going to prioritize customer reports so that urgent fixes are made as rapidly as possible? How are you going to minimize the chances that a defect is fixed (reliably) without breaking something else? When a customer reports a problem to you, one of the first things she is going to want to know is what the status of the fix is. Are you going to let the customers review the status of their defect reports online and see when they can expect fixes? How are you going to develop features that take a long time (weeks to months) to complete? Most definitely the wrong answer is to wait until the feature is complete before merging it into the source code. You are going to have to find a way to integrate these features in piecemeal while ensuring the product still works and while telling users what to expect every day, and you're going to have to find ways to get input from customers on the new features as they progress. What if a critical feature breaks in a daily release? Customers would have to be able to easily report the problem to you and back up to the previous day's version while they wait for a fix. This is the time when you will have to demonstrate responsiveness to your customers, because the time they spend waiting for a fix or backing up to the previous version costs them time and money.
The goal in this exercise is to ship the product to customers as frequently as possible. Shipping frequently encourages teams to minimize their overhead while maximizing their productivity; so they can ship frequently AND implement the features that their customers need. This is only possible by focusing on a working product, because by doing so teams will have to be ultra-efficient at finding, fixing, and preventing failures, as explained in Chapter 5. Teams that are able to achieve the frequent ship/features balance will consistently outinnovate teams that have one or more delays as they try to fix the problems that prevent their product from being in a working state. Shipping daily is the ultimate goal, but there may be factors that make it possible to ship once a week or once a month. What you want to avoid is being able to ship only a couple of times a year. Note that it's one thing to ship twice a year and quite another to only be able to ship twice a year! Not all of your products need to get installed by all possible users, it is simply important to get the software installed and used on real problems as frequently as possible. Many open source projects and a smattering of commercial products are able to actually ship every day or at least very frequently. A few examples are: Eclipse (http://www.eclipse.org) is an open-source IDE project from IBM. A reliable update mechanism is built into the software so that updates can be made at any time. Qt (http://www.trolltech.com) is a commercial GUI cross-platform toolkit. Renderware (http://www.renderware.com) is a commercial game engine produced by Criterion that ships weekly. Torque (http://www.torque.com) is an inexpensive but high-quality game engine that is virtually open source. Microsoft regularly provides updates to the Windows operating system and Office tools. Although Microsoft makes an easy target for many people, I think it's worth pointing out that, overall, its updates work. Norton Anti-Virus is a common desktop tool that is updated whenever a new virus has been detected on the Internet.
Think of shipping every day from a purely technological standpoint. We have the technology right now in the Internet and binary update mechanisms to actually perform live updates of customer software as the developers merge their changes into the product, which would essentially entail multiple software updates every day. There are many reasons why we don't do this; I think the biggest is simply the baggage that the software industry is carrying around: that software quality is poor and updates should be done carefully or only after some (usually long) period of verification testing. There are clearly cases where verification and extreme caution before installation of a new software version is mandatory: for software systems where a software error could cause a person or other living thing to be injured or killed, the economy to collapse, or an expensive piece of equipment to be permanently disabled or destroyed. Clearly, these mission-critical software systems shouldn't be installed without a great deal of caution. The problem today is that the vast majority of software produced today is not mission-critical, yet the software industry has done such a good job of training its users to expect poor quality that they expect to not be able to continue to do their work when they perform a simple software upgrade. Consider a commercial business that has a production pipeline made up one or more software products. The amount of confidence this business would have in being able to upgrade a product while people are working on a project is nil today. But, what if a production pipeline could be changed during a project? And frequently changed? What is the advantage to the software team that could offer this ability to its customers? That is the power of having a working product. All the practices recommended in this chapter help project teams concentrate on keeping their product in a working, virtually shippable, state. Having a singular focus on a working product should be a rallying cry for every project team, one that will lead to significant positive team morale and a sense of ongoing accomplishment and ability to be proactive not reactive (i.e., in control). Achieving a working product is not extra work if it is done every day! Ship at Unpredictable Times One of the chief advantages of having a working product is being able to ship at unpredictable times. I saw a project a while ago where the requirement came in to ship a Japanese version of the shipping product. A large deal with a software distributor was at stake; the distributor needed to get the Japanese version within a month or the opportunity would be missed for another year. The problem is that team members were in the middle of a development cycle for the new version of the product. One of the key architectural changes they had made for the new version was to internationalize the product because they anticipated the need for a Japanese version. However, because the architecture had changed quite a bit from what was currently shipping, they could not port the changes back to the shipping version. In a typical development team, this situation would cause a great deal of panic (and a missed opportunity). Luckily, this wasn't a typical team. Their solution to the problem was to take the software they were working on, turn off all the new features so the software was functionally identical to the shipping version, then complete the localization. The end result was a Japanese version of the product in less than two weeks; the English to Japanese translation and testing took more time than the engineering work! This is one example of the power of having a working product. Having a working product put this team in control of its situation and allowed the flexibility and confidence to ship when it wanted. |
Be Wary of Artificial Milestones One of the teams I worked with had a functionality freeze milestone where all new features for the release had to be integrated into the product before the functionality freeze date. The intention of the milestone was good (largely to ensure that the product could be documented with the features in a stable state and to ensure there was adequate time to test and stabilize the final product before shipping). However, for most features, the functionality freeze came too early in the schedule. As a result, developers would often postpone bugfixing until after the milestone, or they would check in incomplete features and sometimes just code stubs so that the feature would appear to be there and could then be fixed up during the bugfixing phase! Also, team leaders would encourage developers to postpone bugfixing until after the milestone so the features could be completed. Hence, with artificial milestones (as with many practices), it is important to watch out for unintended side effects. In the case of the functionality freeze milestone, the emphasis on all development work before the milestone was on getting features into the code base, not on ensuring that the software still functioned. On this product, the software would often be in an unshippable state for as much as six or nine months out of every year! Although it was tempting for teams to think they'd made terrific progress at the milestone ("Just look at all the features we checked in!") all hell would break loose after the milestone as the team would gallantly try to get the software back to a shippable state, with the end result that many defects couldn't be fixed due to a lack of time. Even worse, many of the features would have to be removed or have the user interface disabled just to get the software working again. A great deal of effort was wasted, not only in fixing all the defects long after they were first introduced and getting the software working again, but in all the features that were implemented but never shipped. Due to the chaos, each product release had slightly lower quality than the release before it. The end result was a gradual degradation of the software's quality over time and a rapid accumulation of an ever-increasing backlog of defects. This in turn led to longer bugfix death marches in every release, where even though the team was working hard to fix all the bugs, it never caught up. Because there were so many bugs, the feature freeze milestone was forced earlier for more stabilization time at the end of the release cycle. And, of course, the pressure to develop new features never went away. The development team literally felt trapped because it had to choose between spending additional time bugfixing and less time developing features, or focus on developing new features and fixing fewer defects. They were also spending an incredible amount of time in their defect tracking system sorting, prioritizing, and analyzing defects. This is part of a classic software development death spiral described in Chapter 2, one that stifles the ability of the team to be innovative and agile. Note that I'm not using this example to imply that milestones are bad, just that milestones need to be treated with caution. There is nothing wrong with a functionality freeze milestone, for example. The team that I describe above still uses the milestone, but its use is more rational. If you feel that this milestone is necessary for your project, then you should challenge each other to ensure that functionality freeze is as late as possible in the release cycle. The problem comes when people get too conservative and start believing in the milestones rather than in each other, or they emphasize the milestone over quality and stability. |
|
