Implementing Your Own Process and Environment

This book has described many tools and techniques. By now you are probably wondering what is the best place to start and how you can implement these techniques in your environment. This section offers some suggestions, starting with your SCM environment.

Start with a Strong Base

The order of the chapters in this book is no coincidence. You no doubt have noticed that the first chapters described the SCM environment and, in particular, the configuration of ClearCase to best support the software build and release process. This was reinforced throughout the remaining chapters by illustrating how SCM labels, baselines, branches, and streams can be used to ensure traceability or isolation of the build or release. One of the first things you should do to create an effective SCM environment is make sure you have drawn up an SCM plan. Such a plan does not necessarily have to be a tome (unless you are required to prove some adherence to a software process standard such as CMMI or SPICE). Rather, it needs to effectively convey the following information:

• The definition of the roles on your project (such as Configuration Manager, Build Manager, Release/Deployment Manager) and who is responsible for them

• The set of tools to be used (such as ClearCase, ClearQuest, Ant) and their configuration, such as the ClearCase server infrastructure

• The naming conventions for the ClearCase VOBs, views, UCM projects, and baselines (or Base ClearCase labels)

• The project directory structure, such as the composition of source, build, and intermediate directories (similar to that described in Chapter 4)

• The project stream or branching strategy, such as how ongoing releases will be managed, whether build or release streams will be used, and so on

• The definition of the change request management process, such as what types of changes are to be captured (such as enhancements or defects) and how they are to be related to builds and releases

• The definition of the release and deployment process, such as how releases are packaged and deployed to their runtime environments

Although the definition of a lot of this information might seem like overkill for defining a build or release process, it is very difficult to construct a reliable and automated build process without it. For example, if the project's baselines and streams are not created according to a convention, how can you automate their creation or hook into them for further automation? For more information on SCM plans, see Haas [Haas02] or Wahli, Brown, Teinonen, and Trulsson [Wahli04].

Automate Only What You Need To

There is a common misconception in software development that automation is good and everything that possibly can be automated should be. Well, this is true to a degree. Certainly this book has looked at how many aspects of the software build and release process can be automated. However, automating everything takes time. You need to ask yourself how often the task that you will automate will be run. If it will be run 100 times, the effort of automating it is worth it. However, if it will be run just two or three times, is the investment in automation really worth it?

It takes time to build up your automated process, so you need to identify the areas where automation is typically first needed. This depends on which phase of development you are in, as illustrated in Figure 12.4. Usually automation is first needed in the compilation, integration, and unit testing processes, which are relied on heavily during initial active development. An early adoption of automated Ant build scripts driven by CruiseControl can certainly provide significant benefits. Once you have managed to automate this part of the process, you can move on to other activities, such as automating the generation of baselines, change request reports, or release packages.

Figure 12.4. Project automation steps

Finally, just because everything can be automated and initiated by a tool such as Cruise Control, this doesn't mean you have to or should. There might be areas where you want to do things manually. The example I have referred to throughout the book has been the generation of the final release for deployment to the user acceptance testing (UAT) or production environments (or the golden master to create the distribution medium). There is no reason why this release couldn't be generated automatically by CruiseControl, but I prefer a separation of concerns and to have a known, stable environment under which I initiate the construction of a Release Build.

Find Your Project Rhythm

One of the most important decisions you will make on your project is how often you will create an Integration or Release Build. To some degree your release schedule will be defined by project and customer expectations. However, internal to the project you will have greater control over how changes are integrated and on what schedule. There is no predefined schedule; you need to find the "project rhythm" that suits you best, typically by looking at patterns such as how long the build takes, how often developers can sensibly deliver, and so on. If you are adopting a continuous integration approach, this can mean building many times a daymaybe every 20 minutes or every hour. With more traditional forms of development this can mean building once a day (maybe as part of a nightly build).

A Build Occurs Only When There Is Something to Build!

Remember that with CruiseControl, if nothing is checked in on the project's integration stream, there will be nothing to build. So even if you define the schedule for a build every 20 minutes, you might still be building only a few times a day.

Your project rhythm also is determined by any dependencies that exist between components or projects and how long the build process takes from beginning to end. Here are some questions you can ask to help determine your project rhythm:

• Are there any dependencies between internal components? If so, do the components have to be built in any specific order?

• Are there any dependencies on the output of external projects or components? If so, how will these outputs be made available?

• How many developers are working on the project? How often are they expected to deliver changes into the integration area?

• What are the inputs for the developer's Private Builds? Will they build against the complete source code structure or against a partial structure and prebuilt binaries from the Integration or Release Build?

• How long does it take to execute a full Integration or Release Build (including workspace population, compilation, unit testing, and internal deployment)?

• How often are Integration Builds expected to be carried out? Where will the outputs be stored?

• How often are Release Builds expected to be carried out? Where will the outputs be stored?

This is not an exhaustive list of questions, but it should be enough to help you determine the setup of your initial automated build and release environment. The next section answers some of these questions in the context of a working example.

Build Efficiency

Remember that the build process is not just about how frequently you build but also how frequently your builds are successful. It is not just about doing builds; it's also about the process you define to fix a build when it fails. This is the measure of build efficiency.

One important point to note is that any one project might have multiple frequencies, typically one for each scale of integration or type of build. In Figure 12.5, as the scale of integration progresses, builds are carried out less frequently. Normally this is because each kind of build has a different type and level of visibility, and hence a more appropriate frequency to go with it, as illustrated in Table 12.1.

Figure 12.5. Build frequencies

Table 12.1. Build Frequencies for Different Consumers

Build Type	Consumer	Frequency Between Builds
Private Build	Developers	Minutes/hours
Component Integration Build	Individual development team	Hours/days
Multicomponent Integration Build	Multiple development teams	Days/weeks
System Integration Build	System Release Management	Weeks/months
Release Build	Customer and/or end user	Months/quarters

As Table 12.1 shows, each type and scale of build has a direct consumer, and the consumer needs to be involved in deciding what the build frequency is. It should be as frequent as possible, but it may also be subject to constraints within the consumer's tolerance level. For example, System Release Management might not be able to tolerate a build each day for each component, or customers might not be able to easily accept a new release as often as monthly or even quarterly without significantly disrupting their environment. The important thing is to plan and identify when the different types of builds will typically happen in your own environment.

Have Fun

When you have fun in your work, I believe you will be more productive. This is certainly true for me. Having an automated build process that constantly updates the development team on the progress of your builds is one way of achieving this. People like to see successful progression, and seeing working build after working build is a good indicator of success.

Just What Is Success?

What defines when your build is successful? Is it when the build compiles? When all the unit tests have run? Or when the build has been deployed? In truth, it is a combination of all of these. Certainly the build should compile and the unit tests execute successfully. However, if you frequently integrate and build, as a "life coach" might say, every failure can be seen as a success, because you have exposed a potential problemand early enough to do something about it!

Out of the box, you can use CruiseControl to send e-mails to indicate build success or failure. However, some development teams have been more imaginative and actually linked CruiseControl to visual indicators such as flashing lights or even lava lamps (see [Clark04] for more information). Unfortunately, I can't guarantee that you will have fun on your project. But if you take away a lot of the drudgery of manual and error-prone tasks through automation, you will certainly be starting to create the right environment for it.