Section 2.3. Handling Data Using Types

2.3. Handling Data Using Types

Ant supports a number of types, and the rest of this chapter is devoted to understanding them and how to work with them. These types work much like data types in programming languages, and as you're going to see, types and properties are intertwined. The data structures you create using types can be assigned to properties, and the data you use to set up those data structures is often stored in properties. Now that you've got properties under your belt, it's time to move on to types.

Much of what a build tool like Ant does is work with files in directory structures, so you might expect that many of the Ant types have to do with handling files and directories. You can see the available Ant core (that is, built-in) types in Table 2-7.

Table 2-7. Core Ant types

Type	Description
Assertions	Enables, or disables, Java 1.4 assertions
Description	Holds a description of the project that can be viewed if you use the Ant -projecthelp command
DirSet	Contains a group of directories
FileList	Contains a named list of files
FileSet	Contains a groups of files
File mappers	Maps original filenames to a new set of names
FilterChains	Contains a group of ordered FilterReaders
FilterSet	Contains a group of filters
PatternSet	Contains a group of filename-matching patterns
Path-like structures	Includes a wide variety of support for specifying file paths
Permissions	Contains the security permissions given to the code as executed in the JVM where Ant is currently running
PropertySet	Groups a set of properties together
Selectors	Groups files in a fileset selected using criteria other than filename, as provided by the include and exclude tags
XMLCatalog	Contains a catalog of public XML resources, such as DTDs or entities
ZipFileSet	Contains a special form of a fileset, using ZIP files

Many Ant tasks in the upcoming chapters depend on the types you see in Table 2-7, so it's worth going through them in detail, especially the ones that handle files. Understanding Ant types is central to using Ant; if you don't understand paths and FileSets, for example, you'll be severely hampered by what you can do with Ant.

2.3.1. Path-Like Structures

In Ant, paths are often handled by the path type, called a path-like structure. As you can imagine, you have to work with paths frequently. For example, a task like javac supports the path attributes srcdir, classpath, sourcepath, bootclasspath, and exTDirs, all of which are handled as path-like structures. That means you can set them as attributes:

  <javac destdir="${build}"          classpath="classes.jar"          srcdir="${src}"/>          debug="on">   </javac>

You can set paths as nested elements, not just as attributes, and you do that with the src, classpath, sourcepath, bootclasspath and exTDirs elements. For the javac task, the srcdir attribute becomes the src element. Nested elements are a good idea when you have multiple paths you want to work with that would otherwise be assigned to the same attribute. Here's an example:

  <javac destdir="${build}"          classpath="classes.jar"          debug="on">     <src path="${src}"/>     <src path="${src2}"/>   </javac>

In addition, when you want to specify path-like values, a nested element can be used with internal pathelement elements, which specify paths, like this:

  <javac destdir="${build}"          classpath="classes.jar"          debug="on">     <src path="${src}"/>     <src>       <pathelement path="${src2}"/>       <pathelement path="${src3}"/>     </src>   </javac>

In pathelement, the location attribute specifies a single file or directory relative to the project's base directory (or an absolute filename), while the path attribute holds colon-separated or semicolon-separated lists of locations (such as "classes.jar;classes2.jar").

You can separate directories with a / or a \ on any platform.

If you want to build your own paths and reference them later, you can use the path element with enclosed pathelement elements. For example, to create a new path and give it the ID build.classpath, you can use this path element and then refer to the new path with the refid attribute that path-like structure elements support:

<path >       <pathelement path="${classes}"/>       <pathelement path="${classes2}"/> </path> <target name="compile">     <javac destdir="${build}"          classpath="classes.jar"          debug="on">     <src path="${src}"/>     <classpath ref/>   </javac> </target>

If a task supports path-like structures, you can specify paths using the attributes of the task, nested elements with names that correspond to those attributes, or with a reference to a path you've explicitly created with path.

2.3.2. Working with Groups of Files

FileSet s are types that represent groups of files, and they're common in Ant because handling a group of files is a common thing to do. The fileset element can contain nested include, includesfile, exclude and excludesfile elements; here's an example using fileset, as well as include and exclude:

<fileset dir="${source}">     <include name="**/*.java"/>     <exclude name="**/*test*"/> </fileset>

Here's an example that lets you include certain files using the filename selector (more on selectors later in this chapter):

<fileset dir="${source}">     <filename name="**/*.java"/>     <filename name="test.cpp"/> </fileset>

Many Ant tasks support file sets, such as the copy task (see Chapter 4); here's an example, which copies all but .java files from the src directory to the dest directory:

<copy todir="../dest">     <fileset dir="src">         <exclude name="**/*.java"/>     </fileset> </copy>

You can see the attributes of the fileset type in Table 2-8.

FileSets can appear inside tasks that support file sets or at the same level as target (as when you want to assign a file set to a property).

Table 2-8. The fileset type's attributes

Attribute	Description	Required	Default
casesensitive	Specifies whether the include and exclude patterns be treated in a case-sensitive way.	No.	true
defaultexcludes	Specifies whether default excludes will be used or not (yes or no). Default excludes are used when omitted.	No.	TRue
dir	Specifies the root of the directory tree used to create this FileSet..	dir or file must be specified.
excludes	Specifies a list (comma- or space-separated) of patterns of files that you want to exclude.	No.
excludesfile	Specifies a file; each line of this file is used as an exclude pattern.	No.
file	A shortcut for creating a single file fileset.
followsymlinks	Specifies whether you want symbolic links to be followed. Defaults to true.	No.
includes	Specifies a list of patterns for files that you want included. Comma- or space-separated.	No.
includesfile	Specifies the name of a file; each line in the file will be used as an include pattern.	No.

As mentioned, the fileset element can contain include, includesfile, exclude and excludesfile elements, and you can see the allowed attributes of these elements in Table 2-9. The fileset element can contain patternset and selector elements, which you'll see later in this chapter.

Table 2-9. The include, includesfile, exclude and excludesfile type's attributes

Attribute	Description	Required	Default
If	If the corresponding property is true, use this pattern.	No
Name	Pattern to include/exclude (for include and exclude types) or the name of the file holding the patterns to include/exclude (for includesfile and excludesfile types).	Yes
unless	If the corresponding property is not set, use this pattern.	No

Some tasks form implicit file sets, such as the javac task. This means that a task supports all attributes of fileset (see Table 2-8). You don't have to use a nested fileset element at all if you don't want to because the built-in attributes of the task give you all the support you need. Each time we come across such a task, I'll be sure to mention this.

The FileSet dir attribute becomes srcdir in the javac task because a dir attribute would be ambiguous in the javac task:Is it the source or destination directory?

2.3.2.1 Default excludes

Some files are excluded by default from file sets since they correspond to system or temporary files of various kinds. Here are the patterns excluded by default (recall that ** means the current directory, or any subdirectory of the current directory):

• **/*~

• **/#*#

• **/.#*

• **/%*%

• **/._*

• **/CVS

• **/CVS/**

• **/.cvsignore

• **/SCCS

• **/SCCS/**

• **/vssver.scc

• **/.svn

• **/.svn/**

• **/.DS_Store

If you do not want these default excludes applied, you can disable them with defaultexcludes="no" in types such as FileSets. You can modify the list of default excludes by using the defaultexcludes task.

2.3.3. Working with Groups of Directories

Directory sets (DirSets) are types corresponding to groups of directories. This type is great for grouping directories together and handling them at one time with various tasks. DirSets can appear inside various tasks or at the same level as target; like file sets, directory sets can contain nested include, includesfile, exclude, excludesfile, and patternset elements. Here's an example:

<dirset dir="${build.dir}">     <include name="apps/**/classes"/>     <exclude name="apps/**/*Test*"/> </dirset>

You can find the attributes of this type in Table 2-10.

Table 2-10. The dirset type's attributes

Attribute	Description	Required	Default
casesensitive	Specifies whether you want to use case sensitivity.	No	true
dir	Contains the root of the directory tree you want to use.	Yes
excludes	A list of the patterns matching directories you want exluded. Comma- or space-separated list.	No
excludesfile	Specifies a name of a file; each line of the file is interpreted as an exclude pattern.	No
followsymlinks	Set to true if you want symbolic links to be followed.	No	true
includes	A list of the patterns matching directories you want included. Comma- or space-separated list.	No
includesfile	Specifies a name of a file; each line of the file is interpreted as an include pattern.	No

2.3.4. Creating Lists of Files

FileLists are types corresponding to explicitly named lists of files. While FileSets act as filters, returning only those files that exist in the filesystem and match specified patterns, FileLists are useful for specifying files individually. Here's an example:

<filelist            dir="${doc}"     files="type.html,using.html"/>

You can see the filelist attributes in Table 2-11.

FileLists are not supported by as many tasks as FileSets are. You might try using a FileSet with nested filename elements to add individual files to the FileSet.

Table 2-11. The filelist type's attributes

Attribute	Description	Required
dir	The base directory you want to use for this file list	Yes
files	A comma-separated list of filenames to include in the file list	Yes

2.3.5. Working with Patterns

A powerful way of working with multiple files is to use patterns, such as the pattern "*.java", which will match all files with the extension .java, "*.class", which matches files with the extension .class, and so on. If you want to work with multiple patterns simultaneously in Ant, patterns can be grouped into sets and later referenced by their id attribute. These sets are defined with the patternset type, which can appear nested into a FileSet or an implicit FileSet.

Here's an example, where I'm creating a file set, using a pattern set, that will match all of a project's documentation files, excluding beta documentation:

<fileset dir="${src}" casesensitive="yes">     <patternset>         <include name="docs/**/*.html"/>         <include name="prof/**/*.html" if="professional"/>         <exclude name="**/*beta*"/>     </patternset> </fileset>

You can see the attributes of this type in Table 2-12.

Table 2-12. The patternset type's attributes

Attribute	Description
excludes	A list of the patterns matching files you want exluded. Comma- or space-separated list.
excludesfile	Specifies a name of a file; each line of the file is interpreted as an exclude pattern.
includes	A list of the patterns matching files you want included. Comma- or space-separated list.
includesfile	Specifies a name of a file; each line of the file is interpreted as an include pattern.

PatternSets can include nested include, includesfile, exclude, and excludesfile elements, and you can nest PatternSets within one another.

2.3.6. Selectors

Besides all the types we've seen, selectors are powerful types and go far beyond selecting files based on filenames. They let you select files that make up a file set based on many criteria, such as what text a file contains, the date a file was modified, the size of a file, and more. Selectors have become one of the coolest things in Ant, and you can see Ant's core selectors, which come built into Ant, in Table 2-13.

Table 2-13. The core selectors

Selector	Means
contains	Selects files that contain particular text
containsregexp	Selects files whose contents contain text that matches a given regular expression
date	Selects files that were modified before a particular date and time, or that date and time
depend	Selects files that were modified more recently than files you compare them to
depth	Selects files based on how far down they appear in a directory tree
different	Selects files that are different from a set of target files you specify
filename	Selects files using a pattern to match filenames
modified	Selects files if an algorithm gives a different result from that stored
present	Selects files based on their presence, or absence, at some other location
size	Selects files larger or smaller than a particular size
type	Selects files based on their type: regular files or directories

You can nest selectors inside file sets to select the files you want. For example, here's how to create a file set of HTML documentation files that contain the text "selector" using the contains selector:

<fileset dir="${docs}" includes="**/*.html">     <contains text="selector" casesensitive="no"/> </fileset>

The date selector lets you choose file sets based on date, as here, where I'm selecting all documentation files before 1/1/2005:

<fileset dir="${docs}" includes="**/*.html">     <date datetime="01/01/2005 12:00 AM" when="before"/> </fileset>

You can use the filename selector much like the include and exclude tags inside a fileset. Here's an example:

<fileset dir="${source}" includes="**/*">     <filename name="**/*.cpp"/> </fileset>

The containsregexp selector limits the files in a fileset to only those whose contents contain a match to the regular expression specified by the expression attribute:

<fileset dir="${source}" includes="*.java">     <containsregexp expression="$println(.);"/> </fileset>

The type tag selects files of a certain type: directory or regular. Here's an example:

<fileset dir="${src}">     <type type="dir"/> </fileset>

FileSets are no longer about filenames. Now you've got access to more data, including what's inside files.

Selectors can be defined outside of any target by using the selector tag and using them as references.

2.3.7. File Filters

Filters let you filter the data in files, modifying that data if you want to. FilterSets are groups of filters, and you can use filters to replace tokens in files with new content. For example, say that your files contain a token, like @DATE@; you can use filters and a filter set to copy those files with the copy task, while replacing that token with the current date. You can find the attributes of the filter set type in Table 2-14; to use FilterSets, you can use the filter task, which is coming up next (with examples).

When a filter set is used in an operation, the files are processed in text mode and the filters applied line by line. This means that the copy operations will typically corrupt binary files.

Table 2-14. The filter set type's attributes

Attribute	Description	Required	Default
begintoken	The string, usually a character, that specifies the beginning of a token (e.g., @ in @AUTHOR#)	No	@
endtoken	The string, usually a character, that specifies the end of a token (e.g., # in @AUTHOR#)	No	@
id	The ID you want to use for this filter set	No
refid	The ID of a filter set you want to use while creating this filter set	No

You specify the filters inside a filter set with the filter task, coming up next.

2.3.7.1 Using the filter task

The filter task supports the actual filters in a filter set. Here's a filter set example where the build file instructs Ant to copy build.java from the ${archives} directory to the ${source} directory, and replace the token @DATE@ in the files with today's date:

<copy file="${archives}/build.java" toFile="${source}/build.java">     <filter set>         <filter token="DATE" value="${TODAY}"/>     </filter set> </copy>

Here's the same example where the token to replace is %DATE;:

<copy file="${archives}/build.java" toFile="${source}/build.java">     <filter set begintoken="%" endtoken=";">         <filter token="DATE" value="${TODAY}"/>     </filter set> </copy>

You can define a FilterSet and reference it later:

<filter set  begintoken="%" endtoken=";">     <filter token="DATE" value="${TODAY}"/> </filter set> <copy file="${archives}/build.java" toFile="${source}/build.java">     <filter set ref/> </copy>

If you just want to replace text in a file, you can use the Ant replace task. Here's an example:

<replace file="${src}/index.html"          token="author" value="Ted"/>

You can use the concat task, designed to concatenate files together, to concatenate text to the end of a file, like this:

<concat     destfile="readme.txt">Copyright (C) 2005</concat>

You can see the attributes of this task in Table 2-15.

Table 2-15. The filter task's attributes

Attribute	Description	Required
filtersfile	The file from which you want filters to be read (you format this file like a property file).	Token and value attributes must be provided, or only the filtersfile attribute.
token	The token string to match (omit the beginning and end tokens).	Token and value attributes must be provided, or only the filtersfile attribute.
value	The string to replace the token with.	Token and value attributes must be provided, or only the filtersfile attribute.

2.3.8. Filtering and Modifying Text with FilterChains and FilterReaders

A FilterReader filters text and can modify that text. A FilterChain is a group of FilterReaders, applied in order, and functions much like piping one command to another in Unix. Using FilterReaders and FilterChains, you can process text data in advanced ways. A number of Ant tasks support filterchain elements:

• concat

• copy

• loadfile

• loadproperties

• move

You can create filter chains with your own Java classes or with one of the elements you see in Table 2-16.

Table 2-16. FilterChain nested elements

Element	Does this
classconstants	Filters and outputs constants as defined in a Java class.
concatfilter	Appends a file to the filtered file, or prepends a file to the filtered file. Two optional attributes: prepend (name of the file to prepend) and append (name of the file to append).
deletecharacters	Deletes characters that you specify in the filtered content. The required attribute, chars, holds the characters to delete.
escapeunicode	Changes non US-ASCII characters into the matching Unicode escape sequence (\ plus 4 digits).
expandproperties	Replaces Ant properties (of the form ${...}) with the property's actual value.
filterreader	Defines a generic filter. Has one parameter, the required classname attribute. Supports classpath and param nested elements.
headfilter	Reads the header, that is, the first few lines, from the filtered data. Supports two optional attributes: lines (number of lines to read) and skip (number of lines from the beginning to skip).
linecontains	Filters out those lines that don't include lines that contain specified strings. One required attribute: contains (the substring to search for).
linecontainsregexp	Filters out lines that don't contain text-matching specified regular expressions. One required attribute: regexp (Pattern of the substring to be searched for).
prefixlines	Adds a prefix to every line. Required attribute: prefix (the prefix to use).
replacetokens	Filters text and replaces text between begintoken and endtoken with the text you specify.
stripjavacomments	Strips Java comments.
striplinebreaks	Strips line breaks from the filtered data. Set the optional linebreaks attribute to the line break text.
striplinecomments	Filters out lines that start with comments, as you specify. Set the comment attribute to the string that starts the line.
tabstospaces	Filters out tabs and replaces them with spaces (default is 8).
tailfilter	Reads the tail of the text, that is, the last few lines from the filtered text. Supports two optional attributes: lines (number of lines to read) and skip (number of lines from the end to skip).
tokenfilter	Tokenizes the filtered text into strings. One optional attribute: delimOutput (overrides the tokendelimiter returned by the tokenizer).

Each of these filters correspond to a Java class; for example, the head filter corresponds to org.apache.tools.ant.filters.HeadFilter. Here are a few examples: Suppose you want to read the first three lines of the file given by ${sourcefile} into the property sourcefile.header. You could use filterreader and the Ant filter reader class org.apache.tools.ant.filters.HeadFilter:

<loadfile srcfile="${sourcefile}" property="sourcefile.header">     <filterchain>         <filterreader classname="org.apache.tools.ant.filters.HeadFilter">             <param name="lines" value="3"/>         </filterreader>     </filterchain> </loadfile>

Ant gives you a shortcut with filter readers like this one, where you can use a task named headfilter to do the same thing:

<loadfile srcfile="${sourcefile}" property="sourcefile.header">   <filterchain>     <headfilter lines="3"/>   </filterchain> </loadfile>

The linecontains filter reader includes only those lines that contain all the strings you specify with contains elements. Here's an example, where I'm filtering only lines containing "apples" and "oranges":

<linecontains>   <contains value="apples">   <contains value="oranges"> </linecontains>

The linecontainsregexp filter reader includes only those lines that match the regular expression you've specified. For example, here's how to match only lines that contain lowercase letters:

<linecontainsregexp>     <regexp pattern="^[a-z]$"> </linecontainsregexp>

The stripjavacomments filter reader strips away comments from the data, using Java syntax guidelines. Here's an example:

<loadfile srcfile="${src.file}" property="java.text">     <filterchain>         <stripjavacomments/>     </filterchain> </loadfile>

The striplinecomments filter reader removes all those lines that begin with strings that represent comments as specified by the user. For example, here's how to remove all lines that begin with #, REM, rem, and //:

<striplinecomments>     <comment value="#"/>     <comment value="REM "/>     <comment value="rem "/>     <comment value="//"/> </striplinecomments>

The tabstospaces filter reader replaces tabs with spaces; here's an example:

<loadfile srcfile="${src.file}" property="src.file.detabbed">     <filterchain>         <tabstospaces/>     </filterchain> </loadfile>

The classconstants filter readers recovers constants defined in Java .class files. For example, say that you have a constant named MAXFILES:

public interface Files {     public static final String MAXFILES ="4"; }

To load the MAXFILES constant and make it accessible by name, you can use the loadproperties task and the classconstants filter reader:

<loadproperties srcfile="Files.class">   <filterchain>     <classconstants/>         .         .         .   </filterchain> </loadproperties>

As you can gather from the name, you can use multiple filter readers in a filter chain, and we'll add the prefixlines filter reader to prefix constants we recover with the text "Files.":

<loadproperties srcfile="Files.class">   <filterchain>     <classconstants/>     <prefixlines prefix="Files."/>   </filterchain> </loadproperties>

Now you've recovered the constant Files.MAXFILES from a compiled .class file, and can display it with echo:

<echo>${Files.MAXFILES}</echo>

2.3.9. Transforming One Set of Files to Another with Mappers

Mappers are another Ant type, and they're used to map one set of files to another. For example, one of the mappers available in Ant is the regexp mapper that lets you grab a set of files and rename them. You can use mappers in copy, move, apply, uptodate, and a number of additional tasks.

Here's an example, where I'm copying a file set from a directory named development to a directory named backup, renaming .java files to .backup files. This works because whatever matches to the expression inside the parentheses in the regular expression in the from attribute can be referenced in the to attribute as \1; a match to a second pair of parentheses may be referenced as \2, and so on:

<copy todir="backup">     <fileset dir="development" includes="**/*.java"/>     <mapper type="regexp" from="([a-z]*).java" to="\1.backup" /> </copy>

Another mapper is the flatten mapper, which lets you copy files while stripping off any directory information from the filenames. That's how mappers typically work; you specify a file set and then add a mapper to refine or manipulate the set of files you want to work with.

To use a mapper, you use the Ant mapper element, which has the attributes you see in Table 2-17.

The mapper classpath attribute holds a path, which means you can use a nested classpath element if you prefer.

Table 2-17. Mapper attributes

Attribute	Description	Required	Default
classname	Specifies the mapper using a class name	Either type or classname
classpath	Specifies the classpath you want used when searching for classname	No
classpathref	Specifies the classpath to use	No
from	Sets the value of the from attribute	Depends on implementation
to	Sets the value of the to attribute	Depends on implementation
type	Specifies a built-in mapper	Either type or classname

Experience shows that Ant will not automatically convert / or \ characters in the to and from attributes to the correct directory separator for your current platform. If you need to specify a directory separator here, use ${file.separator}.

What mappers can you use with the mapper element? You can see the available Ant mappers in Table 2-18.

Table 2-18. Ant mappers

Mapper	Does this
flatten	Flattens the target filename to the source filename but with all directory information stripped off from the front. To and from attributes are ignored.
glob	Lets you support wildcards (*) in the to and from attributes.
identity	Copies over the filename by making the target filename the same as the source filename. To and from attributes are ignored.
merge	Takes the target filename from the to attribute (from will be ignored).
package	Helps with Java package names by replacing directory separators with dots.
regexp	Lets you use regular expressions on the to and from values.
unpackage	Replaces any dots in the name of a package with legal directory separators (available since Ant 1.6).

When you use the identity mapper, the target filename is the same as the source filenamein other words, this one's transparent to Ant. Here's the way you'd use this mapper in a file set:

<mapper type="identity"/>

The flatten mapper is more interesting; this one strips all leading directory information off, allowing you to copy all files in a directory tree to a single directory. Here's an example:

<copy todir="backup">     <fileset dir="development" includes="**/*.java"/>     <mapper type="flatten" /> </copy>

The merge mapper copies multiple files to the same target file. Here's an example:

<mapper type="merge" to="backup.tar"/>

In the glob mapper, you can use filename patterns in the to and from attributes, and those patterns may contain at most one asterisk (*):

<copy todir="backup">     <fileset dir="development" includes="**/*.java"/>     <mapper type="glob" from="*.java" to="*.old"/> </copy>

The regexp mapper lets you use regular expressions in the to and from attributes. You can create regular expression matches using parentheses in the to attribute value and refer to the matched values as \1 through \9 in the from attribute value. We've seen an example of this mapper at work:

<copy todir="backup">     <fileset dir="development" includes="**/*.java"/>     <mapper type="regexp" from="([a-z]*).java" to="\1.backup" /> </copy>

For more on regular expressions, see Mastering Regular Expressions by Jeffrey Friedl (O'Reilly).

The package mapper is an interesting one as it replaces directory separators found in the matched source pattern with dots in the target pattern placeholder, reproducing Java package syntax. This inspiration here is that this mapper was designed to be useful to flatten directory structures where files have the fully qualified classname as part of their name (this mapper was designed for use with the uptodate and junit tasks). Here's an example:

<mapper type="package"     from="*Beta.java" to="Beta*Beta.xml"/>

For example, if you used this mapper element on org/steve/tools/ant/util/AntTaskBeta.java, it will create Beta.org.steve.tools.ant.util.AntTaskBeta.xml, flattening the directory structure but still letting you see where files come from.

The unpackage mapper, new since Ant 1.6, is the reverse of the package mapper; that is, it replaces dots in a package name with directory separators. Here's an example:

<mapper type="package"     from="Beta*Beta.xml" to="*Beta.java"/>

This is a useful one if you want to restore the original directory structure after flattening it with the package mapper. This example will take files such as Beta.org.steve.tools.ant.util.AntTaskBeta.xml back to org/steve/tools/ant/util/AntTaskBeta.java.