I'm sure you all recognize how big a showstopper manual integration on a monthly basis, for example, can beeven if your team just consists of two developers on a project. The problem increases exponentially when you add more developers. Among other things, it costs lots of time, it is error prone, and you will find that it won't happen as planned. "Just one more day" will be a common sentence used in the project. My friend Claudio Perrone is just the right guy to describe the popular solution to this, known as continuous integration.
By Claudio Perrone
Sooner or later, all the components that have been created and modified by different developers need to be built and assembled together to form a single system.
In the past, I've witnessed (and, yes, occasionally caused) spectacular delays in projects originated by last-minute integration efforts where unexpected defects were discovered in the final phases of a development cycle.
A build may fail for a variety of reasons, often in combination, such as
Note that, despite all the best efforts, only some of these problems can be limited by the discipline and communication capabilities of the developers involved. The reality is that if the team does not integrate often and if the number of new classes in the system is sufficiently large, you may find yourself anxiously hunting combined bugs foreveran unpleasant scenario commonly known as "Integration Hell."
The Solution (Or at Least a Big Step in the Right Direction)
The key to significantly reducing integration problems is to generate your builds automatically and to use an incremental integration strategy where all the code is rebuilt and tested at least daily, if not continuously. The idea is that if your recently added code breaks a build, you either fix it immediately or roll back the changes to restore the system to the last known good state.
The fundamental parts that constitute a continuous integration system are as follows:
There are several integration products available that you may want to investigate. Currently, my favorite is CruiseControl.NET [CC.NET], which I use in combination with NAnt, a very popular XML-based build engine. Both tools are open source. It takes quite an effort to configure CruiseControl.NET but, once up and running, it takes care of calling NAnt scripts whenever there is a change in the codebase. It notifies you of the progress and status of all current builds using a wide variety of client modules, including a little application that uses a Windows tray icon to show summary information at a glance through color coding and notification balloons.
Lessons Learned in My Organization
When we first contemplated the possibility of implementing a continuous integration system at InnerWorkings, most of us thought that it was a really great idea. Although we were already under severe scheduling pressure, we knew that it was definitely worth investing a few days to implement the system. If I think about it now, however, we certainly underestimated the profound impact that such a system would have on the quality of our products and the confidence of our team.
Today we have about 500 continuously integrated solutions, and the number is still increasing. A third of these solutions share a custom-built common framework, which is also integrated. Integration steps for all these solutions include compilation, obfuscation, packaging, testing, and deployment using different configurations and platforms.
Before we started this project, we were told that it would be impossible to integrate all of these solutions continuously. However, I'm convinced that it would have been impossible to do otherwise and that this effort constitutes a critical factor in our success.
A good starting point for learning more about continuous integration is a paper written by Martin Fowler and Matthew Foemmel called "Continuous Integration" [Fowler/Foemmel CI].
Thanks, Claudio! Now let's move on to some operational aspects.