Section 5.4. Common Build Problems


5.4. Common Build Problems

Some commonly encountered problems with build processes include:


Command-line length

The length of the commands executed by build tools can cause problems on different platforms. Shell commands are usually typed in by people, so they remain relatively short. It is not uncommon for commands that are thousands of characters long to be generated by build tools. Some reasons for this are absolute pathnames being used for all files, or long file lists being used to maximize the number of files processed each time that a slow-to-start compiler is invoked. Build tools can sometimes avoid this problem by using a temporary file containing the list of files to be used by another tool.


Filename formats

The names for the same file on different filesystems are quite different. For instance, Windows filesystems have drive letters, plus backslashes separating directories, as opposed to Unix forward slashes. Older Windows filenames could have only up to eight characters, with a three-character extension. Older Unix filesystems did not support spaces in filenames. If you want to build your project on multiple platforms, then a build tool with good support for avoiding filesystem-specific names is helpful. The easiest build tools to use are ones that allow you to specify a filename in one format and then automatically convert the filename to the appropriate format for the intended platform.


Unit tests for build tools

If you treat build tools like any other application, you might well expect to have a set of system tests to check that the tool is working as expected. Build tools are slightly unusual applications in that their input is so varied, but it is still useful to have a simple test project for testing your build tool. Detecting an error such as a file being rebuilt multiple times is hard to do purely by inspection of the build tool's output log. A collection of small test projects also provides an excellent place to develop new techniques for using your build tool.


Identifying builds

Unique identifiers for all builds that are to be used by other people helps clear communication within a project. The idea of a build label is introduced in Section 3.5. A label such as QA#1_3_0#129#2005_07_09 can identify a build as build 129, for internal testing (QA), Version 1.3.0, which was built on July 9, 2005. This kind of build label has many uses for SCM tools, bug tracking systems, and file and directory names.

5.4.1. Slow Builds

When discussing builds with project members, the most common request is always how to make them faster. There are a number of approaches that can help speed up your builds:


Profile the build

The first thing to do with a slow build is find out where most of the time is being spent. You can do something as simple as logging the current time at various interesting points in your build files. Some build tools such as SCons can provide more complete profiling information by treating the build tools as an application to be profiled just like any other application.


Build only once

Incorrect dependencies in build files may mean that files are built more than once or that other files that aren't needed are being built. To make sure this isn't happening to you, extract filenames from the build log and make sure each one is expected and occurs only once. Profiling can sometimes help here too.


Use a build server

If developers' desktop machines seem underpowered for the size of the project, you can try using shared build machines with more disk space and RAM. If there are a lot of disk accesses during a build, then a RAM disk cache (a ramdisk) may help. Of course, using a single build server makes your development environment more vulnerable to a single point of failure.


Create staged builds

If the build for your product is taking more than an hour or two, it may be time to consider splitting it up into multiple builds, where each one builds a subsection of the product. The results of the builds can then be stagedthat is, made available for other people in the project to use as starting points for their own builds.


Use ccache

When you compile a C file, the C preprocessor first creates an expanded source file, which is what is actually compiled into an object file. ccache (http://ccache.samba.org) is an application that keeps a copy of the generated object file, and if the expanded source file hasn't changed the next time that it needs to be compiled, then ccache just returns the saved copy instead of wasting time recompiling the file. If the file has changed, then ccache compiles it as usual. To do all this accurately, ccache has to also record which compiler flags were used when the saved object file was created. This idea is great for builds where you rebuild lots of files the same way every time (for instance, virgin builds), or if a build tool is rebuilding files by mistake.


Use parallel builds

Many build tools have arguments to run different parts of a build in parallel. Depending on how good the build tool is at separating the build into different parts, this may improve your build times, or it may not do much. The only way to find out is to try it, which is usually a straightforward matter of passing another command-line argument to your build tool. Another approach is to use a distributed compiler such as distcc instead of expecting the build tool to handle distributing the work for the build.


Use distributed compilation

Using a cluster of machines to build a product is sometimes helpful. One tool that does this is distcc (http://distcc.samba.org), an open source frontend to the GNU gcc compiler that distributes preprocessed files to remote machines for compilation. After being compiled remotely, the object files are then returned to the main build machine. distcc assumes that the build dependencies calculated by your build tool are correct and can be cleanly partitioned for a distributed compilation. Electric Make (mentioned in Section 5.5.2, later in this chapter) is another high-end distributed build tool.




Practical Development Environments
Practical Development Environments
ISBN: 0596007965
EAN: 2147483647
Year: 2004
Pages: 150

flylib.com © 2008-2017.
If you may any questions please contact us: flylib@qtcs.net