Section 9.3. Creating the Release | Practical Development Environments

9.3. Creating the Release

Ideally, system testing should be performed on releases, rather than on builds without the final packaging steps of a release. That way, when a particular release is deemed worthy of shipping to customers, the release process can use a known set of files to build the release. This also ensures that the installation software is tested, at least in the most common way that testers install the software.

When a build is converted to a release for internal testing, or an internal release is converted into a customer release, the following steps are commonly followed:

Obtain virgin copies of the correct files using the local SCM tool. The version of the product that uses these files has already been built and tested.
Set the release number and other build information, either in a file, a database, or on the command line.
Build the product for each platform.
Build the packages that are released to customers.
Update the release notes as part of the packages.
Retest the different packages prior to releasing them. Don't just check that the installer worksrun as many of the unit and system tests as possible against the installed product.
Use the SCM tool to tag all of the files that are part of the release, including all automated tests and test results.
Archive all customer releases and any other important builds.

9.3.1. Automated Releases

Automating the process of creating a release will reduce human error and speed up both regular customer releases and emergency fixes. There are some common tasks that are awkward to automate, so here are some ideas. Some of these ideas are also discussed in the context of how to choose an automation environment (see Section 3.4).

First, use your build tool to create as much of the release as possible. Debugging a mixture of different build and release tools only makes getting a process right much harder, and using one good tool will save confusion. For instance, a single set of makefiles is much easier to work with than shell scripts that call make and Perl scripts. Newer build tools including SCons and Ant have built-in support for using various SCM tools to obtain the correct source files, for tagging the source files after a build, and for creating different kinds of packages.

The version number for a release is often set by hand (in one central location), and then a build number is automatically incremented. Incrementing the build number requires storing the current build number, as shown in Example 9-1. If a single build machine is used, then a simple text file may suffice. A better scheme is to store the build number in a file that is controlled by the SCM tool. Just to be sure, add checks to make sure that contents of the file storing the build number always correspond to the build label used when tagging that file.

Example 9-1. Updating the release number using a shell script

#!/bin/bash # An example shell script to store the build number in a file. # The file where the current build number is stored between builds CBN_FILE=/home/build/current_build_number . . . PREVIOUS_BUILD_NUMBER=`cat ${CBN_FILE}` BUILD_NUMBER=`expr $PREVIOUS_BUILD_NUMBER + 1` . . # Build the product. . . # Make sure that you can work out whether the build number was updated # just by looking at the build log. echo ${BUILD_NUMBER} > ${CBN_FILE} echo "Build number ${BUILD_NUMBER} was recorded in ${CBN_FILE}"

If an IDE is being used to develop the product, take the time to find out whether the IDE can be called by a build tool, or whether you have to export a build file from the IDE before you can build from the command line. Some IDEs provide fewer ways to build products from the command line than when using the GUI, which is frustrating to discover later on in a new project.

In older versions of Visual Studio, adding a /Y3 argument to msdev on the command line is an undocumented way to display the time spent in each part of the build.

Some tools used during a build require input from the user. If these inputs are textual, are consistently in the same order, and consist of answers that are known ahead of time, then redirecting the input to a file containing the required text can work. For example, if a configure file requires some input, generate a suitable input file and pass it to configure with ./configure < configure.input.

Some of the visual tools used as part of a build may not provide any command-line interaction or may have pop-up windows with choices that appear occasionally. GUI testing tools can sometimes help you here, or the underlying API of the tool may be available for scripting, but maybe this tool really isn't designed to be part of a build and release process?

When an automated build finishes, it's useful to tell people about it. If it failed, then project leaders and developers should be told, particularly those who have committed changes to the source since the last successful build. If it succeeded, then people such as the testers waiting for an internal build should be informed. Email is one convenient way to do this, though changing who receives the email requires aliases or some other minor email list management. Email can also be used later on for working out when releases should have been available, but for one reason or another were late. Another approach is to use RSS feeds, which each interested person can subscribe to. Some people like to receive a brief text message about broken builds on their mobile phone, so they can track down the cause of the breakage quickly.

Sometimes a build may hang, and no notification at all will appear. To make sure you are made aware of this, a separate watchdog process can be started at the same time as the build. Assuming that you can predict how long the longest build should take, the watchdog process can monitor the build and look for some evidence that indicates that the build finished. One good piece of evidence is the text of the email notification sent when the build is finished. If no such text or file appears after the given time, then the watchdog should send out a "failed build" notification instead.

9.3.2. Automating Release Information

There are various kinds of information that can be generated automatically for a release, as discussed in Section 9.2.4, earlier in this chapter. A common request from testers is for a list of the bugs that are supposed to have been fixed in this particular release, along with the test reports from any automated tests that were run as part of the build. This list of bugs can then be used for testing and confirming that the bugs are indeed fixed. Producing this list automatically depends on whether the bug tracking system has an API that can be queried from within an automated build process. If it doesn't, then a file can be manually updated by developers, though of course this is prone to error.

If the bug tracking system can be queried from within a build, there are usually two approaches that are used. If the current release number already exists in the bug tracking system, then a simple query for the associated bugs is all that is necessary. Alternatively, if fixed bugs are in a state named something like "Fixed but Not Yet Available," then the current release number needs to be added to the bug tracking system metadata so that when the state of such a bug is changed to Fix Available, a field such as "Fixed in Release" is filled in with the current release number.

Another common request is for a list of all bugs fixed since the a particular customer release. This is often provided in release notes. When creating the names of releases within the bug tracking system, make sure that the tool supports searching for bugs both using an exact release number and using a pattern such as 1.3.* to find all bugs related to the 1.3 releases.

Other useful information for developers and testers is a change log, showing all the files that were changed since the previous release and what those changes were. Since this can be a large amount of information, an HTML change log with links to a graphical browser for the SCM tool is helpful. Change logs are discussed further in Section 4.4.

Another action that can be automated is the assembly of release notes from fragments of text associated with each bug or associated with a particular target release inside the bug tracking system. This helps to make sure that lots of small pieces of information are put together in an organized manner and none are accidentally left out.

9.3.3. Developers as Customers

The version of a product with the fewest bugs is usually the one that runs on the same platform that the product was developed on. This makes sense, because some bugs are only found by long-term use. Similarly, a product behaves best when it is used in the same way that the developers and testers used it when they created it.

One good idea is to make the released product resemble the versions that are used during development as much as possible. For instance, if the product is run as a service under Windows, build the product the same way for developers too. Make it easy for developers to build a release package and install it on their machines.

Another good idea is to avoid using #ifdef RELEASE and the related compiler -D arguments to change the build for a release. If system tests and other utilities have to be built for testing, then build them separately from the main product. If there are still unavoidable differences between the development and release builds, then try to capture all of them in a very small number of files, and then use the build tool to choose which files to build. Debugging later on will also be easier when customers' environments can be reproduced by developers.