This preface discusses my motivations and experience for writing this book and what it is intended to convey. I then describe the book's framework with a brief description of each chapter. Finally, I discuss how best to read the book, depending on your own role, interests, and experience.
Why This Book Came About
I have always been interested in building software, in taking source code written by me or other people and running it through a magical machine called a compiler. A compiler is black and white: the build either compiles or does notthe feedback is usually pretty instant. On initial consideration, building software sounds simple and not especially important. It's only when you get drawn in that you realize how fundamental the build is to the entire software development team. It's only then that you realize how just a little change here or there can have a decided impact on the team's productivity.
One of my first jobs was working as a build and integration engineer on a large C++ software development project. When I first joined, the build process seemed to work, but it was not pleasant. From my inexperienced vantage point, it seemed that developers were delivering the build in whatever format they liked. We had several hundred fragmented makefiles, and that number was growing by the week. Far too regularly there were problems with the buildcompilation errors, missing files, incorrect release instructions. Because the build was often carried out over the weekend, no one was around to fix these issues. The build also took an inordinately long time; with full optimization, it took more than 24 hours. This was largely due to our inability to reliably detect and maintain the build dependencies across our large code base. Consequently, every build was a full rebuild.
Gradually, over time we improved the build process. We standardized the deliverable format, started running the builds earlier to detect configuration issues, and distributed some of the compilation to bring down the overall build time. The makefiles were refactored and simplified. Then we started looking at standardizing the version control process; some teams were using RCS, some CVS, and some, well, nothing! We saw version control as the real enabler for the build. It was at this point that I was first introduced to IBM Rational ClearCase, or Atria ClearCase as it was known then.
To me, ClearCase seemed to be exactly what we had been looking for. We now had the capability to do build avoidance, to recompile only the source code that had changed using the automatic dependency checking of the ClearCase clearmake tool. We had build distribution to run the build across as many servers as we could throw at it, and we also had build auditing so that we could tell exactly what went into the build. This was all backed up by a sophisticated branching and merging capability so that the organization as a whole was now better at managing multiple releases and bug fixes from the same evolving code base. Everything was now working well with the build process. It was optimal and stable, and we thought we could move on and tackle other development issues. And then Java happened!
Java was just gaining acceptance in the software development community, so some parts of the project were starting to use the language for their implementation. At first, we thought it would simply be a matter of creating some new makefiles, and we could continue to leverage the build management capabilities of ClearCase. We started to look at how we could build Java projects by creating new makefiles for use with ClearCase. However, the problem was that a large number of our build files were based on templates sourced from development or third parties. They were the ones who understood the applications and the libraries being built. They were really the ones driving the requirements for the build process, and they were suddenly giving us template build.xml filesfiles to be used by Apache Ant. We investigated Ant and appreciated that it had a lot of positives. Its build scripts were portable and could be run on any environment (quite important for us at the time), and it had built-in capabilities for packaging, deploying, and manipulating the file system, some of which exceeded the capabilities of standard make. We could see the benefits and reasons for changing over to Ant, so we did. In doing so, we lost a lot of the capabilities we had patiently implemented.
The Purpose of This Book
As the person responsible for constructing or consulting on the build and release process on the project just described and many others since, I have had to adapt and find new ways of working with ClearCase and Ant in the Java world. I have been doing this job for many years now, with software development companies and now with IBM Rational, where as a consultant I have been fortunate enough to see and help other people meet this challenge. Over time, I have learned that the build and release process is not just about ClearCase and it is not just about Ant. It is about finding a balance and deciding what tools work best for you. I also believe a software build and release process is created, not adopted. Sometimes you might find that aspects of your process even need to be coded. I have yet to see a completely generic software build and release process. The exact process, along with the tools and their configuration, will ultimately depend on your own project's requirements. However, you can use a number of best practices and tools to create your own process. We are fortunate that the Java development community ecosystem has a wealth of open-source and commercial tools that can help us do so.
This book therefore is about using ClearCase, Ant, and other open-source and commercial tools (most notably, CruiseControl) to define and implement a Java build and release process. By the end of the book, I hope I will have demonstrated how the combination of these tools can be used to implement an agile, traceable, and repeatable build and release process. You don't need to know much about ClearCase to be able to benefit from this book. You certainly don't need to know much, if anything, about Apache Ant or CruiseControl. In fact, I will revisit some of the basic ClearCase concepts when I discuss early on how features such as branching and labeling can support the build and release environment. Don't worry if all this talk of makefiles and clearmake has gone over your head. The reason that ClearCase is used as the foundation for this book is because, from experience, I believe that certain capabilities within this tool can be used to really get the build process moving. Similarly, I will also discuss ClearQuest, IBM Rational's change request and defect tracking tool. This is because the build and release process is about more than just code and versions. It is also about managing change and successfully deploying the fruits of your labor.
This book can help you automate your processes in many ways; there are a few highlights that are worth pointing out. By the end of the book you should be able to do the following:
In the examples, I use the capabilities of both Base ClearCase and Unified Change Management (UCM)IBM Rational's best-practice Software Configuration Management usage model. UCM has some additional capabilities you can take advantage of to really get your builds moving. That said, the majority of the principles and examples can be applied to Base ClearCase as well. Any exceptions or differences are explicitly pointed out.
Finally, as you can probably ascertain from this book's title, the intended audience is Java developers. However, I use the word "developer" to describe anyone who is involved in the creative part of software development that is directly related to the build and release process. This can include software developers (or coders), software architects, build engineers, release engineers, deployment engineers, and so on. There is therefore an implicit assumption that you know something about Java coding and scripting, even though you don't need to be a coder. However, even if you are a development manager or team leader, you should still be able to get something out of the first two chapters at least.
What This Book Covers
The chapters describe the complete Software Build and Release Management process.
Chapter 1, "Introduction to Software Build and Release Management"
The first chapter introduces the key concepts and definitions I use throughout the book. I start by defining what I mean by the terms build and release, and then I describe the process through which they are constructednamely, Software Build and Release Management. Next, I examine the challenges that project teams typically face when building and releasing software. Finally, with the aim of addressing these challenges, I define an integrated software build and release life cycle solution. The constituent parts of this solution form the book's framework.
Chapter 2, "Tools of the Trade"
You can use a number of different tools to support and automate the Software Build and Release Management process. Chapter 2 discusses the tools I use throughout this book. Since IBM Rational ClearCase is the foundation of the build and release environment, I spend some time discussing its concepts and how they can support you in your endeavors.
Chapter 3, "Configuring Your SCM Environment"
Chapter 3 describes a set of best practices for configuring a ClearCase environment to support the Software Build and Release Management process. This includes many aspects of a complete Software Configuration Management environment, including creating a suitable build tools directory structure, defining a baselining and branching strategy, and setting up suitable workspace configurations for building and releasing from.
Chapter 4, "Defining Your Build and Release Scripts"
Apache Ant is a Java-based build tool. Chapter 4 describes the basic mechanisms for constructing Java builds using Ant build scripts. I walk through the process of installing Ant, creating Ant build scripts, and invoking them. I also describe how to edit and invoke Ant build files from inside the Eclipse development environment.
Chapter 5, "Apache Ant Best Practices"
Chapter 5 looks in more detail at some best practices for using Ant on real projects. First I describe the use of property files to enable the configuration of the build process depending on a user's role and requirements. I then describe how best to integrate Ant with IBM Rational ClearCase. Finally, I consider some general best practices for supporting the build process on large projects.
Chapter 6, "Running Your Build Scripts"
CruiseControl is a wrapper around Apache Ant for automatically executing build scripts. Chapter 6 looks in detail at how you can create, install, and configure an environment for running CruiseControl.
Chapter 7, "CruiseControl Best Practices"
Chapter 7 explores some best practices for implementing CruiseControl. First, I cover how to install, configure, and use the CruiseControl Build Results web. Then I move on to how best to integrate CruiseControl and ClearCase, paying particular attention to how you can automate labeling and baselining. Finally, I also include some best practices for scheduling CruiseControl builds and publishing the results.
Chapter 8, "Baseline and Change Request Reports"
You can interface with ClearCase and ClearQuest to automate the creation of reports in a number of ways. Chapter 8 describes some best practices for reporting on the set of baselines and change requests that are created or used during the build process. First I walk you through generating Base ClearCase and UCM baseline reports from the command line and Perl. Then I show you how to interface with IBM Rational ClearQuest to generate change request reports. Finally, I go over how to generate RSS feeds from ClearQuest change request information.
Chapter 9, "Build Reporting and Auditing"
Chapter 9 presents some best practices for reporting on and auditing the build process. First I describe how to generate a baseline report that lists the exact contents of a ClearCase label or baseline in XML and then subsequently transform this output into HTML format. Next I cover how to customize the CruiseControl Build Results web to display the same report. Finally, I take you through initiating a ClearCase audited build and producing a report of the environment and element versions that the build encompasses.
Chapter 10, "The Art of Releasing"
The Software Build and Release Management process does not stop when your build has been compiled, integrated, and unit-tested. Chapter 10 covers how software releases are constructed and deployed. First, I define what development artifacts are typically packaged as part of a release. Then I go over how Release Builds are constructed and their outputs made available for packaging and subsequent deployment.
Chapter 11, "Release Packaging and Deployment"
Chapter 11 describes the final parts of the Software Release Management process: Release Packaging and Release Deployment. First, I expound on how different types of Release Packages can be created. Then I move on to how the Rational Build and Deployment Tracking capability can be used to create a Deployment Unit file and automate the deployment workflow. Finally, I discuss different strategies for physically deploying the Release Packages to their runtime environments.
Chapter 12, "Putting It All Together"
In the final chapter, I revisit the concepts and challenges of Software Build and Release Management that were introduced in Chapter 1, relating them to the techniques that have been described throughout the book. I discuss some approaches to applying these techniques in your own environment. Finally, I also revisit the working scenario from Chapter 1, illustrating how the techniques can be realized in a typical environment.
How to Read This Book
You don't need to adopt everything that's in the book; I'm sure there are aspects of what is described that you do already. However, like any good book, this one has a beginning, middle, and end. The first chapter takes the whole of the build and release life cycle, takes it apart piece by piece, and then puts it back together, examining how to best automate each aspect. Chapter 2 then describes the set of tools you can use to automate your build and release process, and Chapter 3 covers the fundamental configuration of IBM Rational ClearCase to support it. I therefore suggest that you read Chapters 1 through 3, because they provide a good framework for the rest of the book.
If you understand the concepts and tools defined in the first three chapters, you can dip into the section of the book where you think your own build or release process might need some more consideration or automation. For example, if you want to know more how Ant and ClearCase can be used to accelerate the Java build process, read Chapters 4 and 5. Similarly, if you're familiar with Ant but you want to know how CruiseControl can be used to automate the execution of your Ant build scripts, read Chapters 6 and 7. Finally, if you want to know how best to release and deploy your software, read Chapters 10 and 11.
To help you implement an automated build and release process, I illustrate how to customize the set of tools outlined in Chapter 2, either by writing scripts or even by creating new code extensions. I endeavor to explain in detail how these customizations are carried out. This is because I believe one of the advantages of the chosen toolset is the ability it gives you to create a unique and customized build and release process that exactly matches your requirements. However, if you are not that interested in how some of the more detailed customizations are carried out, you can skip forward to see how the results of the customizations would be used.