1.11. Productization: The Key to Understanding the Challenge of Using Open Source
Perhaps the simplest way to understand what you are getting into by using open source is to think of it in terms of the productization idea introduced earlier in this chapter:
Most open source projects are only partially productized. But all of the information required for you to work around the lack of productization is available.
The key questions are:
The models described in the next chapters are aimed at answering those questions:
It is no accident that the most skilled engineering teams in the country are also the largest users of open source. For them, the cost of overcoming the productization gap is small. The rest of this book will help IT departments understand the size of the gap for them.
In fairness to the open source community, we should mention another interpretation. From the perspective of a person with the required open source skills, the lack of productization is not a problem. Productization might even get in the way of a developer's needs. From this perspective, the barrier to wider adoption is not the lack of productization, but the lack of skills in those who desire to use open sourcethe skills gap mentioned earlier.
Remember, productization in a commercial product is not black and white. Some companies do a better job of it than others. There will always be problems to overcome with any software, and commercial software also comes with a productization gap most of the time. In the rest of this chapter, we will look at commercial software and open source software side by side.
1.12. Comparing the Risks of Commercial and Open Source Software
The episodic, sporadic, incremental evolution of open source is in sharp contrast to the more methodical design process that most commercial projects go through. This chapter has pointed out that IT technologists and executives who are seeking to understand the opportunity open source provides must understand the nature of commercial software, which is born out of a completely different process and has different strengths and weaknesses.
But the different processes can easily obscure the nature and quality of the end product. Whether open source or commercial software is better for a particular company or a particular purpose is a complex decision that is a function of the quality of the software, how well it fits to a particular task, and the skills present in the development team. The models that we describe in subsequent chapters provide a framework for understanding these issues.
The fact is that most software, open source and commercial, has problems that must be overcome for it to be useful. Figure 1-2 shows how the best software from the open source world shares certain characteristics, as does the worst. But most software is not located at the extremes of this scale. Most software falls into a gray area, meaning it has significant problems that might or might not be showstoppers, depending on the context.
Commercial software vendors would have you think their software is perfect and without blemish. However, anyone who has bought commercial software and used it extensively finds all sorts of rough edges. This is true even for the best software from the best companies. The closer you get to a commercial software product, the uglier it looks. Most software is in the gray area, which means that in the evaluation process the nature of the software's defects must be revealed to determine whether a program is suitable for your company. In the following discussion, we will examine some important differences between open source and commercial software that affect the evaluation process, and differences in the risk of owning and operating each type of software.
1.12.1. The Sales Process
As we have pointed out repeatedly, open source software generally doesn't come with a salesperson to guide you through the process of learning how the software might help you. While salespeople can be a tremendous help in gathering information about a piece of software, even the most nave among us knows that all the information comes with a strong positive bias. Whitepapers, references, case studies, and so on, all paint a rosy picture. Software companies spend millions of dollars to influence the opinions of third-party analysts. Much of the most useful information, such as product support databases, is not available until after purchase. Sales staff is seldom rewarded for providing reasons not to buy the software. The sales process is generally of little help in getting to the key problems of software in the gray area.
Figure 1-2. Shared characteristics of open source and commercial software
With open source, much more information about the software is available on the product's frequently asked question (FAQ) lists and bulletin boards . Much of the discussion on these forums actually concerns the problems of keeping an open source project in the gray area. The information will not come to you, but in most cases it is there for you to find. And you must be careful about accepting such information as authoritative . Some open source project participants have anonymously and systematically posted negative information about competing projects.
With a commercial product, the company controls most of the information. The claims for product features might or might not be backed up by solid code, especially when it comes to new features. Customers who have run into problems usually don't post this information for public display, to avoid harming their relationship with the company or to avoid legal retribution. Independent user groups also usually want to keep good relations with the company. And user - group information is frequently available only after you are a customer anyhow.
In an open source project, the entire sausage factory is there for you to see, in all its complexity and ugliness, from day one. The chief barrier to getting the information is the time it takes to rummage through source code, bulletin boards, and so on. Search engines have made this process much easier. The most annoying and controversial bugs are usually not hard to find.
The ultimate test of whether an open source software product will work for your company is to install it and try it out. With commercial software, this is not always an option, and one that frequently costs money when it is an option. In open source, the costs are measured in terms of time and effort, but the opportunity to test the software is always available. While it might require effort, with an open source project there are no barriers to finding the problems in the gray areas.
There are well-defined ways to extend a commercial product. APIs are defined to allow users to write programs using the functionality of the applications. If the APIs allow everything you need, all is well. But if the product doesn't expose some deep, inside functionality in the right way, it can be years before a requested change becomes part of the product. The company's view of how many other customers want the same type of feature determines its decision to incorporate the requested change.
In open source, there is no such barrier. If you want to use deep, inside functionality of the software, you can write your own APIs to get access to it. Of course, this requires time, effort, and expertise, and for a large program it is hard to know if such deep surgery will have unintended side effects. But you have nobody to ask. You must simply do the work.
1.12.4. Risk of Quality
It is not uncommon for a commercial product to be interesting because of a new feature. That is why new features are createdto attract customers. But how well does that new feature work? With a commercial product, it is hard to tell until you buy it, mostly because of the issues mentioned already. And if you encounter any problems, you can't fix them.
In an open source project, a new feature might be implemented to spur discussion. The problems with the new feature might be announced on the project web site, without shame. The idea, of course, is that the developer might not have the time or skill to get the feature right. In an open source project, everything is open to improvement, if you have the time and skill. Figuring out what skill level you possess and what kind of tasks you can handle is the goal of the Open Source Skills and Risk Tolerance model.
In an open source project, it is generally possible to access the knowledge of senior programmers. In healthy open source projects, intelligent questions that are crucial to determining the product's future, or fixing new bugs, are addressed by the most senior developers on the project. With commercial software, access to senior engineers is difficult for an external company to get.
1.12.5. Risk of Productization
As a general rule, commercial software is more productized than open source software. Installation scripts, administrative interfaces, and documentation are usually better for a commercial product than for an open source product of the same age.
This chapter argues that to succeed with open source software, it is wise to plan for overcoming the lack of productization. It is the fundamental argument of this book that developing such skills can pay great benefits.
1.12.6. Risk of Failure
Both commercial software vendors and open source projects can crash and burn. Commercial software companies almost always try to protect customers by putting their source code in escrow, so it will be available should the vendor go out of business. The problem is that escrows can become out-of-date. It requires a lot of effort to keep an escrow current, along with everything needed to create the software. Does the escrow match the software at every customer site? How can it be made to match? Who would do this work?
In an open source project, a working copy of the source code is in the possession of the company using the software, from day one. This copy of the software is not out of sync, and it has everything needed to change, recompile, and assemble the working program. If an open source project tails off, those using the software are at a much lower risk.
1.12.7. Risk of Takeover
In the world of commercial software, a takeover can change everything. If a larger company buys a smaller competitor, the customers of the smaller company might eventually face a forced migration to the larger company's product. In a recent series of shocks to the IT industry, PeopleSoft bought J.D. Edwards, only to be acquired in a hostile takeover by Oracle. It's not hard to imagine how worried the original J.D. Edwards customers are. Sometimes, an acquiring company bends over backward so as not to alienate its new customers. But in this case, we've heard horror stories concerning loss of support and forced migration.
Open source projects are similar to takeovers in that both have forks, whereby a group of developers takes the source code and moves it in a different direction. A fork can take people's attention away from coding or hurt morale . But unlike what happens in some takeovers, nobody is forcing work to stop.
Support is perhaps the biggest advantage of commercial software. The 15% to 25% of the license fee paid every year funds a staff of engineers and support technicians who are on call to help when you have a problem. This is of great value to most companies and is a great comfort if the support is well organized and of high quality.
Some open source projects provide excellent support to their users; sometimes the quality of that support exceeds that of commercial products. But in a broad sense, while commercial software has clear channels of support, open source support comprises figuring things out on your own.
With open source projects, large collections of information can help with some elements of support, but for the most part, you are on your own. This is a scary prospect for all but the most skilled programmers. Some companies have stepped into this gap and are offering support services for popular open source products. One of the fundamental arguments of this book is that companies who become skilled enough to be able to support open source have much to gain.
Looking back on this list of risks, we can see many risks and responsibilities for both commercial and open source software. With commercial software, the customer pays the vendor to manage the risks. In the world of open source, you must manage the risks yourself. By the end of Chapter 5 of this book, you will be an expert at determining and comparing these risks.