Section 12.4. Project Builds

12.4. Project Builds

Integration of work is an important aspect of any software development process. With multiple developers working on the same project, it is important to frequently integrate everyone's work, build the entire project, and test that all of the changes work together. Linking the version control system into this process is vital, because the VCS is generally one of the primary tools for performing the integration of work.

With Subversion, there are a couple of different approaches that you can take toward accommodating project build integration, which I will discuss in a little while. When setting up project builds, though, you also need to take into consideration how you will run your build process (automated or manual), and how individual bits of the build will be configured.

12.4.1. Configuration

Project builds generally require some sort of build configuration file, such as a makefile for a version of the classic UNIX Make program, a build.xml configuration for that Javabased Ant build system, or a Visual Studio .dsp file. Often, with these build configurations, you have a base configuration file that needs to be used by everyone who is working on the project; however, each developer also often needs to make local modifications that shouldn't be fed back to the version stored on the repository.

The naïve approach to local build configurations is often to commit the base configuration file to the repository, and then have individual developers modify the file in their working copy to meet their needs. This works fine until someone accidentally commits their local changes to the repository (better not run svn commit with no options). Furthermore, if a developer needs to commit some of his local changes to the base version in the repository, but not others, it can be a major pain to do.

A better approach to local build configuration files is to commit the base configuration file as a template, under a different name. Then, add an svn:ignore property entry to force Subversion to ignore the real configuration file. When developers check out a working copy of the project, they can then copy the template to create a new configuration file that can be edited locally. So, for example, if you have a project that is built using GNU Make, with a makefile named Makefile, you would create a makefile template named Makefile.tmpl, which would be added to the repository. When you check out a working copy of the repository, you would then copy Makefile.tmpl to Makefile and make any necessary local changes.

Even better still is to have two configuration files (if your build system supports this). In one, you place the bulk of your base build configuration and commit it as-is. It then includes a second build configuration that holds values likely to change from one local installation to another. For that file, you commit a template instead of the actual file, and use svn:ignore as I mentioned previously. The advantage of doing things this way is that it allows changes to be made to the base build configuration, without requiring you to hand-merge those changes in each working copy to the local build configuration every time a change is made to the base.

12.4.2. Daily Builds

Traditionally, a big part of most software development processes has been the daily build of the entire project, which integrates everything and allows the full project to be tested, either with automated tools or by a QA testing team. Recently, as the popularity of rapid development techniques grows (as well as the available computing power), daily builds are more frequently becoming multiple builds per day. If you are going to be performing frequent full-project builds like this (especially if your project is large), it is important for you to know how to best accommodate the daily build (I will refer to them as daily builds, even if you perform them more frequently than once per day) in your Subversion repository.

There are two major overall policies for handling your build process. You can either have a manual build, with a build engineer (which may be a regular developer doing doubleduty or a full-time job for large projects), or you can have an automated build system that runs the build, and possibly a test suite, without any regular human interaction.

From a Subversion perspective, manual daily builds are the easiest to accommodate. The build engineer simply needs to have his own working copy of the project's main development branch, which he can update before every build. For more information on the process of getting each developer's work integrated into the development branch in preparation for the daily build, see Section 12.4.3, "Integration."

Manual daily builds may be the easiest to prepare your Subversion repository for, but in the real world, there are a lot of compelling reasons to use automated daily builds instead, such as reliability, cost, and speed. If the build is going to be run automatically, though, you need to put a little more thought into how the automated build will interact with Subversion.

A simple approach to automated builds is to have a script that runs at a set interval (daily, hourly, and so on) to execute the build. When the script runs, it should create a tag for the build, switch the automated tests working copy to the new tag, and then run the build system against the newly created tag. Additionally, you might want to have a test suite run (although that may be built into the build system), and add notification about the build's results, which could be e-mailed, added to a Web site, or included in an RSS feed.

Here is a simple example of a script that you might have run your automated build. It uses the svn commands to interact with the repository, and uses make to run a build and a test suite. To set it up for a daily build, you could configure cron to run it every night and report the results to a build engineer, or post them to a Web site.

 #!/bin/sh # run_daily_build.sh # Get the repository working copy path from the script's arguments WORKING_COPY=${1} # Get the URL of the main branch, and the URL of the tags directory TRUNK_BRANCH=${2} TAGS_DIR=${3} # Check to make sure the working copy exists. # If it doesn't exist, exit with an error message. if [ ! -f "${WORKING_COPY}" ]; then echo "No working copy\n"; fi # Change directories to the working copy cd "${WORKING_COPY}" # Update the working copy # Redirect the results into a status file svn update . > build_results.txt # Get the current revision REV=` svn info . | grep "^Revision: " | cut -c 11-` # Execute the build and test suite make && make testsuite # Make a daily build tag of the build svn copy -m "Tagged daily build" -r ${REV} ${TRUNK_BRANCH} ${TAGS_DIR}/build-` date "+%m%d%Y"` 

CruiseControl

Rather than rolling your own scripts for doing automatic scheduled builds, you can make use of a system designed for doing automatic scheduled builds. The CruiseControl system (cruisecontrol.sourceforge.net) is a framework that checks your Subversion repository on a configurable schedule to see if any changes have been committed. If they have been committed, it downloads the changes, builds your system, and runs all of your unit tests. If you do any sort of rapid development, this can be a huge timesaver. It is a Javabased system, and uses Ant for its project builds, which makes it specifically suited for development of Java-based projects, but you should be able to make it build other projects (possibly with a bit less integration) if you are so inclined.

To set up CruiseControl to build your project, you first need to install CruiseControl on your build system, by following the instructions for CruiseControl installation that can be found on the project's Web site or in the CruiseControl distribution. Then, to set up CruiseControl to talk to your Subversion repository, there are a few things you need to configure in the config.xml file for CruiseControl.

The first elements to add are two <plugin> elements to load the two Subversion plugins.

[View full width]
 <plugin name="svnbooststrapper" classname="net.sourceforge.cruisecontrol.bootstrappers .SVNBootstrapper"/> <plugin name="svn" classname="net.sourceforge.cruisecontrol.sourcecontrol.SVN"/> 

Then, you need to add the <svn> element to the <modificationset> element.

 <modificationset>    <svn localWorkingCopy="svnrepos/trunk"         repositoryLocation="https://myserver.com/svnrepos"         username="bill"         password="mypass"/> </modificationset> 

Also, in your Ant build, you need to add an <exec> element to make Ant update the Subversion repository.

 <exec executable="svn">    <arg line="up"> </exec> 

12.4.3. Integration

If you are doing a daily build of your full project, it is important for you to have the project's developers integrate their work into a single development branch. The biggest question, though, is when this should happen. A developer making a single minor change to the project is unlikely to cause major integration problems, but a developer working on a more in-depth feature will likely be making numerous intermediate commits, which may temporarily break the project in relation to other developers. Avoiding broken builds is important for obvious reasons, which means that you need to put some thought into how you are arranging your Subversion repository to accommodate both the work flow of individual developers and the overall integration problem for full-project daily builds.

Continuous Integration

One approach to integration is to have a policy whereby every developer works on the main development branch, committing their changes as they go. This has the advantage that there are no worries about merges, and everyone stays very up-to-date with the current state of the project. It is also the easiest for people to understand, and lends itself to small to medium-sized projects that are undergoing small incremental development. The downside, however, is that because all commits go onto the same branch everyone else is working on, every commit needs to integrate perfectly with the rest of the project or it breaks the build. Although this encourages careful committing and conservative changes, it also discourages frequent commits and may lead to larger commits that don't encapsulate a single change to the project.

Task Integration

An alternate approach to integration is to use task branches. With this integration policy approach, each developer does all of her nontrivial project development (more than a couple of lines of code) on a branch created for that specific task, where she can freely commit small changes without fear of breaking the project build. Then, as soon as an individual task is complete, the task branch can be merged into the main development branch for inclusion in the full project's daily build.

The biggest downside to this approach is its complexity. Each developer needs to have a working understanding of Subversion branches and merges, along with the discipline to use the task branches properly. It also splits the project's history off into a large number of branches, which can make finding a specific change more difficult.

On the positive side, task branches encourage frequent small commits, while maintaining the integrity of the main project build. They also keep the history of the project's main branch clean, because small commits get aggregated into a single merge log message for each task. These advantages make task integration a good choice for large, complex projects with a large number of developers, as well as for projects that use fewer daily builds with automatic unit testing. If the integrated build is only built and tested a few times a day or less, task integration tends to make it easier to find and repair broken builds. On the other hand, if you are doing very frequent builds with continuous testing, task branches may get in the way for all but the most complex of tasks.