Some software companies are able to periodically reinvent themselves and their products. These companies don't need to completely rewrite their software products and in fact are able over time to add to their product line, usually with the same underlying technology and implementations. How do these companies do it? For some of the answers, let's look at some interesting research from a seemingly unrelated industry: chemical manufacturing.
Some interesting research into productivity at chemical manufacturing plants has parallels in software development [Repenning and Sterman 2001] This research focused on chemical plants that are in deep trouble. These are plants that had low productivity, low employee morale, etc. The companies who owned the plants were seriously considering, or in the process of, closing them down and moving operations to another location with higher returns on investment.
What the researchers found is that in the plants in question the first response to trouble is to ask people to work harder, usually through longer hours. However, while working harder results in a definite short-term increase in overall capability, the long-term effect is actually declining capability, as shown in Figure 1-2.
Figure 1-2. Working harder (more hours) results in declining capability over time. Working smarter, with an emphasis on continual improvement, leads to increasing capability. From [Repenning and Sterman 2001].
One of the reasons for declining capability over the long term when working harder is the resulting vicious cycle or death spiral. This cycle is due to unanticipated side effects of the decision to work harder: As more hours are worked, more mistakes are made, and there is a greater emphasis on quick fixes and responding to problems. These mistakes and quick fixes lead to the requirement for more work.
In a chemical plant a mechanic might incorrectly install a seal on a pump. The seal might rupture hours or days later. When it does, the entire pump would need to be replaced, which takes an entire section of the plant offline while leaking more chemicals into the environment. People will think they aren't working hard enough, so they'll put in more hours. Then, the extra costs of all the overtime and environmental cleanups kick in and costs are cut in other areas such as carrying fewer spare parts to compensate. When parts aren't available when needed, the plant is down longer. The longer the plant is down, the greater the reduction in revenue and the higher the costs. The greater the reduction in revenue, the greater the pressures to further reduce costs. Eventually, people are going to be laid off, and the fewer people available, the less the ability of the plant to produce output. And so it goes.
The largest reason for a decline in long-term capability is that working harder results in an inability to implement necessary improvements. In the plants studied, mechanics were too busy fixing problems in the pumps to do anything else. As any car owner who ignores basic regular maintenance knows, the longer mechanical parts are left untended, the greater the chance they will eventually fail, not to mention the greater the eventual cost. This leads to another vicious cycle: The harder people work and the more problems they are trying to fix (or more appropriately, the more fires they're trying to put out), the greater the chance that problems will continue to build and grow worse over time. No doubt you've been in situations like this. The problem quickly becomes one of having time stand still through continuous death march releases, or fixing things.
The employees of the chemical plants turned things around by developing a realistic simulation of their situation. The simulation was developed in such a way that it demonstrated to participants the results of various decisions. Importantly, the simulation was not designed to teach or test skills. They recognized that the mechanics, for example, didn't need to be taught to be better mechanics; after all, they were very adept at their craft through all the crucial problems they had to fix on the spot. The simulation, implemented as a game, realistically demonstrated the various important tradeoffs that can be made in a plant between working harder and working smarter. In a chemical plant, working smarter consists of activities like preventive maintenance, where a pump is taken offline, disassembled, examined, and lubricated on a regularly scheduled basis. Working smarter is also taking the time to examine the entire plant's systems and processes and continually attempting to identify problems before they occur while reducing the complexity of the overall system in order to increase efficiency.
The results of the simulation were an eye opener for the plant's employees. The results were also counterintuitive to many: They showed that working smarter (especially doing preventive maintenance) consistently produced better results over the long term. The simulation also demonstrated that with any attempt to work smarter there is an initial dip in capability caused by the need to let some problems go unfixed while initial improvements are made as shown in Figure 1-2. This helped the employees expect the dip in capability and have the persistence to follow through with the changes (a perfectly human response would be to think that the dip was permanent and revert back to the old ways). Eventually, as the number of improvements started to make a difference, capability would climb until a point where the plant entered a virtuous cycle, where each additional investment in improvements led to further efficiencies and gains in output with lower environmental impact, which in turn led to more time being available to make further improvements, and so on. People were actually able to accomplish more work by working smarter than they had before.
As a result of the simulation, the chemical plants described experienced a complete turn-around. Not only were these plants kept open, but they also received additional work and their business grew. And some of the changes introduced by the employees had a lasting effect, some with a return on investment of 700,000 percent! The most astonishing thing, which perhaps isn't so astonishing when you consider the rut these companies were stuck in, is that virtually all of the changes that were required to make the turnaround were well known to the employees but they'd never been implemented because they were always too busy!