Generating Base ClearCase Baseline Reports

Generating reports is a necessary part of the software development process. However, having to generate them frequently and manually can often be time-consuming and error-prone. This section looks at how to use cleartool commands and Perl to automate the generation of Base ClearCase baseline reports.

Using the ClearCase Command Line

The ClearCase command line (and, in particular, the cleartool command) is one of the main mechanisms for automating the generation of baseline reports. ClearCase has a UNIX heritage; you can see this in how its command line is used. If you are a Windows user, the command line might take a bit of getting used to, but once you understand its power and capabilities, I'm sure you will find it extremely useful.

Since a ClearCase VOB is essentially a database, you might expect a SQL interface to be available for querying it. Unfortunately, this is not the case; instead, ClearCase has its own query language, which is somewhat cryptic. This language supports a number of primitives such as brtype, created_by, and version for querying ClearCase objects such as branches and elements. It also supports logical and comparison operators similar to those found in the C programming language. A query typically searches one or more VOBs and returns information about the objects you specify. For example, the following command could be used to display the version of an element with a specific label:

>cleartool describe -short -version "{lbtype(RATLBANK_01_INT)}"Account.java Account.java@@\main\2

In this example the cleartool describe command is used with the -short parameter to limit the display of the ClearCase element's object information to just its pathname. The ClearCase query is specified via a version selector and executes a query to find the version of the element that is labeled with RATLBANK_01_INT (and on any branch). You can carry out similar queries for ClearCase branches, attributes, and users. The full list of primitives and operators is described in detail on the ClearCase query_language manual page.

Displaying Manual Pages from the Command Line

You can display any ClearCase manual page from the command line by typing cleartool man page. An example is cleartool man query_language.

The query language is particularly powerful when used in combination with the cleartool find command, because you can return information on a set of objects rather than an individual object. For example, to find all elements labeled with RATLBANK_01_INT, you could execute the following:

>cleartool find . -version "lbtype(RATLBANK_01_INT)" -print .\src\com\ratlbank@@\main\3 .\src\com\ratlbank\exception@@\main\2 .\src\com\ratlbank\model@@\main\2 .\src\com\ratlbank\model\Account.java@@\main\2 .\src\com\ratlbank\model\Bank.java@@\main\14 .\src\com\ratlbank\model\Customer.java@@\main\2 .\src\com\ratlbank\model\TransRecord.java@@\main\1 ...

In this example the cleartool find command navigates from the current directory downward. For each element it checks whether the label RATLBANK_01_INT has been applied. If it has, the element name and version number are printed. As you can probably imagine, this command could be used as the beginnings of a baseline report.

You will notice from the commands executed so far that the output is somewhat limited. If you want to customize this output, you can use the -fmt option. It is supported by a number of cleartool commands, such as describe, lshistory, and lscheckout. For example, you could modify the previous cleartool describe example to display information about the element, version, owner, last modified date, and check-in comment:

[View full width]
>cleartool describe -fmt "Element %En: \n version: %Vn \n checked in by user: %u \n on  date: %d \n last comment %c" Account.java Element: Account.java  version: \main\2  checked in by user: alex  on date: 08-Jan-06.15:23:55  last comment: Refactored transaction link

In this example you will see a number of conversion specifications, such as %En and %c, and also a number of escape sequences, such as \n for a new line. Together these produce a formatted string, rather like the C language printf statement. There are a vast array of conversion specifications, some of which have modifiers. For example, %n specifies an element's full name, including its version specification. Additionally including an E modifier, such as %En, limits the display to the element's name. For more information on format strings, refer to the ClearCase fmt_ccase manual page.

With the commands just described, you should now have enough information to be able to generate a simple baseline report that compares two labels, nominally the current and previous builds, and that displays which elements have changed between them. A Windows batch file to achieve this is given in Listing 8.1.

Listing 8.1. Windows baseline_report.bat File

[View full width]

@echo off @echo CLEARCASE BASELINE REPORT echo. cleartool describe -fmt "First Label: %%n \n created: %%Sd \n by: %%u \n status: % %[locked]p \n" lbtype:%1 echo. cleartool describe -fmt "Second Label: %%n \n created: %%Sd \n by: %%u \n status: % %[locked]p \n" lbtype:%2 echo. @echo ELEMENT VERSIONS CHANGED BETWEEN %1 AND %2: echo. cleartool find . -all -ver "!lbtype(%1) && lbtype(%2)" -print

It would be a relatively simple task to convert this batch file to a Linux or UNIX shell script, since the cleartool commands are invoked in exactly the same way. If this batch file were executed, you would see output similar to the following:

>baseline_report.bat RATLBANK_01_INT RATLBANK_02_INT CLEARCASE BASELINE REPORT First Label: RATLBANK_01_INT  created: 11-Jan-06  by: alex  status: locked Second Label: RATLBANK_02_INT  created: 12-Jan-06  by: chris  status: locked ELEMENT VERSIONS CHANGED BETWEEN RATLBANK_01_INT AND RATLBANK_02_INT: C:\Views\RatlBank_bld\RatlBank\build.xml@@\main\8 C:\Views\RatlBank_bld\RatlBank\src\com\ratlbank\model\Account.java@@\main\2 C:\Views\RatlBank_bld\RatlBank\src\com\ratlbank\model\Bank.java@@\main\14 C:\Views\RatlBank_bld\RatlBank\src\com\ratlbank\model\Customer.java@@\main\12 ...

As you can see, this gives you all the information you need about any Base ClearCase label. However, the output format is limited to what the cleartool command can produce. It would be nice to script up the sequence of commands and manipulate the information to produce a nicely formatted report. This type of scripting is probably beyond the capabilities of simple Windows batch files or UNIX shell scripts. What is needed is a Practical Extraction and Reporting Languagewhich is where Perl comes in.

Using Perl

As its "expanded" name suggests, Perl is a great language for manipulating and reporting on textual information. Through its powerful built-in support for regular expressions and pattern matching, you can easily search, match, and replace text patterns, thus allowing you to construct more readable baseline or change request reports. To execute Perl scripts, it is recommended that you use the executable called ratlperl. This is a common shared component that is shipped with both ClearCase and ClearQuest.

The main difference between Perl scripts and something like Windows batch files is that it is easy to retrieve the output of a cleartool command and manipulate it. For example, to retrieve the output of a cleartool command and place it in a variable, you could use the following:

my $output = 'cleartool describe -short -version "{lbtype(RATLBANK_01_INT)}" Account.java'; print "Version is: $output\n";

This places the output of the cleartool command in the Perl variable $output and prints it as follows:

Version is: Account.java@@\main\11

Once you have the output in a variable, you can start manipulating it. For example, you can use Perl's split function to separate the element's name and version:

my ($elem_name, $elem_ver) = split /@@/, $output; print "Element $elem_name "; print "is at version $elem_ver\n";

This produces the following output:

Element Account.java is at version \main\11

This example works by splitting the line returned by the cleartool command at the ClearCase special @@ symbols, the variables $elem_name and $elem_var being set to the values of the line on either side of the @@ symbols.

When Directories Have Versions

Since directories can also have versions, the preceding Perl example will fail if the full version extended pathname has to refer to the version of a directory with the pathname of the element itself. An example would be if the cleartool describe command returned. Although this situation is rare, it can happen, so you have to code for it accordingly.

To execute a Perl script, you create a text file containing the code you want to execute (the normal convention is for the script to have a .pl extension). You can subsequently execute the script as follows:

>ratlperl name_of_script.pl

One of Perl's more powerful features is its capability to redirect a command's output through a filehandle so that it can be parsed and manipulated a line at a time. For example, the output of the cleartool find command could be parsed a line at a time using the following:

open (IN, "cleartool find . -all -print|"); while (<IN>) {     # remove extended name     s{@@.*}{}g;     print " $_"; } close (IN);

In this example, a filehandle is opened to the cleartool command via the | operator. (This operator comes from UNIX and is usually called a pipe.) The while statement then reads from this pipe one line at a time. The content of each line is automatically placed in the special $_ variablehence the print statement. The other line, s{@@.*}{}g, is a Perl regular expression search and replace. It finds the first occurrence of the @@ symbol on the line and removes it and any subsequent characters. Consequently, when this script is executed, you see output similar to the following:

./build.xml ./default.properties ./src ./src/com ...

Although the end result isn't very exciting, it gives you an idea of Perl's capabilities.

To illustrate what Perl can do in more detail, the Windows batch file from Listing 8.1 can be rewritten quite easily in Perl. It is shown in Listing 8.2.

Listing 8.2. Perl difflabel_report.pl File

[View full width]

use Cwd; my $CLEARTOOL_CMD = "cleartool"; '$CLEARTOOL_CMD pwv' or die "Error: can't find cleartool command: $!\n"; my ($label1, $label2) = @ARGV; if ((@ARGV != 2) or ($label1 eq "") or ($label2 eq "")) {     die("Error: invalid ClearCase label specification.\n"); } print "CLEARCASE BASELINE REPORT\n\n"; # print information on the labels print '$CLEARTOOL_CMD describe -fmt "First Label: %n \\n created: %Sd \\n by: %u \\n  status: %[locked]p \\n\\n" lbtype:$label1'; print '$CLEARTOOL_CMD describe -fmt "Second Label: %n \\n created: %Sd\\n by: %u \\n  status: %[locked]p \\n" lbtype:$label2'; # works for both forward and backslashes in path my $cdir = cwd(); # what platform are we running on? if ($^O =~ /win/i) {     # windows, change slashes     $cdir =~ s:/:\\\\:g;     $cdir = $cdir . "\\\\"; # add trailing slash } else {     $cdir = $cdir . "/";  # add trailing slash } # print information on the elements changed print "\n\nELEMENT VERSIONS CHANGED BETWEEN $label1 AND $label2:\n\n"; open (IN, "$CLEARTOOL_CMD find . -all -ver \"!lbtype($label1) && lbtype($label2)\" -print|"); while (<IN>) {     # remove current directory     s{$cdir}{}g;     print "  $_"; } close (IN);

In this example, information is displayed about the two labels passed as parameters to the script, and then the difference in element versions between the two labels is displayed. (The parameters are retrieved using the Perl @ARGV array, where $ARGV[0] retrieves the first parameter, $ARGV[1] retrieves the second, and so on.) The bit of magic in the middle of the script finds out the directory from which the script is being executed so that it can be removed from the output. This is made possible using the Perl cwd() function, which returns the current working directory, and the rather cryptic $^O Perl variable. It contains a string describing which operating system the script is being executed on, such as MSWin32, unix, or linux. If this script were executed, you would see output similar to the following:

>ratlperl difflabel_report.pl RATLBANK_01_INT RATLBANK_02_INT CLEARCASE BASELINE REPORT First Label: RATLBANK_01_INT  created: 11-Jan-06  by: alex  status: locked Second Label: RATLBANK_02_INT  created: 12-Jan-06  by: chris  status: locked ELEMENT VERSIONS CHANGED BETWEEN RATLBANK_01_INT AND RATLBANK_02_INT:   build.xml@@\main\8   src\com\ratlbank\model\Account.java@@\main\2   src\com\ratlbank\model\Bank.java@@\main\14   src\com\ratlbank\model\Customer.java@@\main\12   ...

Notice how the display of elements is a bit more readable because it does not contain the absolute pathnames. The other advantage of this script is that it runs on any platform on which ClearCase can be installed.

Perl Scripting Best Practices

If you will be programming a lot of Perl, it is a good idea to observe some general best practices:

• Never assume that the cleartool command is actually on the PATH. It is better to check if it is available or use a variable such as $CLEARTOOL_CMD to refer to the fully qualified path.

• Use a wrapper function, such as sub cleartool_exec { ... }, for all command-line invocations. This greatly eases debugging and profiling. Another alternative is to use the Perl CPAN module CtCmd, which is discussed later.

• Always check the return codes (exit codes) of cleartool invocations if it matters in your script whether the command was successful.

• Always use consistent error and warning messages, typically making use of the Perl functions die and warn.

• Use "normal" delimiters for regular search-and-replace operators. Typically you would use the / character as a delimiter most of the time. However, if you need to search for /, standard usage is to use the :character or one of the balanced delimiters (curly braces) as the delimiterfor example, s{@@.*}{}g;.

• Make your Perl scripts self-executable. To do this on Linux or UNIX, you can add a line at the top of the file that defines the use of the Perl interpreter as the shell scriptfor example, #!/opt/rational/bin/ratlperl. On Windows you can "associate" all .pl file types with the ratlperl executable or create a batch file wrapper.

• Always use the Perl chomp() function instead of chop() to remove the newlines from the end of the output of commands. If you don't, you might find you are suddenly getting rid of the last (non-newline) character in a string.

There are a number of other best practices, but these should be enough to get you started. Thanks to Brad Appleton for suggesting an initial set.