Building

Compiling
Resources
More On Building
Cleaning

To remove any confusion, Buildr’s build task is actually called build. It’s also the default task that executes when you run buildr without any task name.

The build task runs two other tasks: compile and its associated tasks (that would be, resources) and test and its associated tasks (test:compile, test:setup and friends). We’ll talk about compile more in this section, and test later on. We’ll also show you how to run build without testing, not something we recommend, but a necessary feature.

Why build and not compile? Some projects do more than just compiling. Other projects don’t compile at all, but perform other build tasks, for example, creating a database schema or command line scripts. So we want you to get in the practice of running the build task, and help you by making it the default task.

Compiling

Each project has its own compile task you can invoke directly, by running buildr compile or as part of another build task. (Yes, that build).

The compile task looks for source files in well known directories, determines which compiler to use, and sets the target directory accordingly. For example, if it finds any Java source files in the src/main/java directory, it selects the Javac compiler and generates bytecode in the target/classes directories. If it finds Scala source files in the src/main/scala directory it selects the Scalac compiler, and so forth. A single project cannot use multiple compilers at the same time, hence you may prefer creating subprojects by programming language. Some compilers like Groovy’s are joint-compilers, this means they can handle several languages. When the Groovy compiler is selected for a project, .groovy and .java files are compiled by groovyc.

Most often, that’s just good enough and the only change you need to make is adding compile dependencies. You can use compile.dependencies to get the array of dependency file tasks. For Java, each of these tasks points to a JAR or a directory containing Java classes, and the entire set of dependencies is passed to Javac as the classpath.

Buildr uses file tasks to handle dependencies, but here we’re talking about the Rake dependency mechanism. It’s a double entendre. It invokes these tasks before running the compiler. Some of these tasks will download JARs from remote repositories, others will create them by compiling and packaging from a different project. Using file task ensures all the dependencies exist before the compiler can use them.

An easier way to specify dependencies is by calling the compile.with method. It takes a list of arguments and adds them to the dependency list. The compile.with method is easier to use, it accepts several type of dependencies. You can use file names, file tasks, projects, artifacts specifications and even pass arrays of dependencies.

Most dependencies fall into the last three categories. When you pass a project to compile.with, it picks up all the packages created by that project. In doing so, it establishes an order of dependency between the two projects (see Defining the Project). For example, if you make a change in project teh-api and build teh-impl, Buildr will detect that change, recompile and package teh-api before compiling teh-impl. You can also select a specific package using the package or packages methods (see Packaging).

When you pass an artifact specification to compile.with, it creates an Artifact task that will download that artifact from one of the remote repositories, install it in the local repository, and use it in your project. Rake’s dependency mechanism is used here to make sure the artifact is downloaded once, when needed. Check the Artifacts section for more information about artifact specification and repositories.

For now let’s just show a simple example:

compile.with 'org.apache.axis2:axis2:jar:1.2',
  'org.apache.derby:derby:jar:10.1.2.1', projects('teh-api', 'teh-impl')

Passing arrays to compile.with is just a convenient for handling multiple dependencies, we’ll show more examples of that when we talk about Artifacts.

Likewise, the compile task has an array of file tasks that point at the source directories you want to compile from. You can access that array by calling compile.sources. You can use compile.from to add new source directories by passing a file name or a file task.

For example, let’s run the APT tool on our annotated source code before compiling it:

compile.from apt

When you call apt on a project, it returns a file task that points to the target/generated/apt directory. This file task executes by running APT, using the same list of source directories, dependencies and compiler options. It then generates new source files in the target directory. Calling compile.from with that file task includes those additional source files in the list of compiled sources.

Here’s another example:

jjtree = jjtree(_('src/main/jjtree'), :in_package=>'com.acme')
compile.from javacc(jjtree, :in_package=>'com.acme'), jjtree

This time, the variable jjtree is a file task that reads a JJTree source file from the src/main/jjtree directory, and generates additional source files in the target/generated/jjtree directory. The second line creates another file task that takes those source files, runs JavaCC on them, and generates yet more source files in target/generated/javacc. Finally, we include both sets of source files in addition to those already in src/main/java, and compile the lot.

The interesting thing about these two examples is how you’re wiring file tasks together to create more complicated tasks, piping the output of one task into the inputs of another. Wiring tasks this way is the most common way to handle complex builds, and uses Rake’s dependency mechanism to only run tasks when it detects a change to one of the source files.

You can also control the target directory. Use compile.target to get the target directory file task. If you need to change the target directory, call the compile.into method with the new path.

We use method pairs to give you finer control over the compiler, but also a way to easily configure it. Methods like dependencies and sources give you a live array you can manipulate, or iterate over. On the other hand, methods like with and from accept a wider set of arguments and clean them up for you. They also all return the same task you’re calling, so you can chain methods together.

For example:

compile.from('srcs').with('org.apache.axis2:axis2:jar:1.2').
  into('classes').using(:target=>'1.4')

Buildr uses the method pair and method chaining idiom in many places to make your life easier without sacrificing flexibility.

Occasionally, you’ll need to post-process the generated bytecode. Since you only want to do that after compiling, and let the compiler decide when to do that – only when changes require re-compiling – you’ll want to extend the compile task. You can do that by calling compile with a block.

For example, to run the OpenJPA bytecode enhancer after compiling the source files:

compile { open_jpa_enhance }

You can change various compile options by calling, you guessed, compile.options. For example, to set the compiler to VM compatibility with Java 1.5 and turn on all Lint messages:

compile.options.target = '1.5'
compile.options.lint = 'all'

Or, if you want to chain methods together:

compile.using :target=>'1.5', :lint=>'all'

Sub-projects inherit compile options from their parent project, so you only need to change these settings once in the top project. You can do so, even if the top project itself doesn’t compile anything.

The options available to you depend on which compiler you are using for this particular project, obviously the options are not the same for Java and Flash. Two options are designed to work consistently across compilers.

Buildr turns the warning option on by default, but turns it off when you run buildr --silent. It also sets the debug option on, but turns it off when making a release. You can also control the debug option from the command line, for example:

# When calling buildr
$ buildr compile debug=off

# Once until we change the variable
$ export DEBUG=off
$ buildr compile

Compiling Java

The Java compiler looks for source files in the project’s src/main/java directory, and defaults to compiling them into the target/classes directory. It looks for test cases in the project’s src/test/java and defaults to compile them into the target/test/classes directory.

If you point the compile task at any other source directory, it will use the Java compiler if any of these directories contains files with the extension .java.

When using the Java compiler, if you don’t specify the packaging type, it defaults to JAR. If you don’t specify the test framework, it defaults to JUnit.

The Java compiler supports the following options:

Option	Usage
`:debug`	Generates bytecode with debugging information. You can also override this by setting the environment variable `debug` to `off`.
`:deprecation`	If true, shows deprecation messages. False by default.
`:lint`	Defaults to false. Set this option to true to use all lint options, or specify a specific lint option (e.g. `:lint=>'cast'`).
`:other`	Array of options passed to the compiler (e.g. `:other=>'-implicit:none'`).
`:source`	Source code compatibility (e.g. ‘1.5’).
`:target`	Bytecode compatibility (e.g. ‘1.4’).
`:warnings`	Issue warnings when compiling. True when running in verbose mode.

Compiling Scala

Before using the Scala compiler, you must first set the environment variable SCALA_HOME.

The Scala compiler looks for source files in the project’s src/main/scala directory, and defaults to compiling them into the target/classes directory. It looks for test cases in the project’s src/test/scala and defaults to compile them into the target/test/classes directory.

If you point the compile task at any other source directory, it will use the Scala compiler if any of these directories contains files with the extension .scala.

When using the Scala compiler, if you don’t specify the packaging type, it defaults to JAR.

The Scala compiler supports the following options:

Option	Usage
`:debug`	Generates bytecode with debugging information. You can also override this by setting the environment variable `debug` to `off`.
`:deprecation`	If true, shows deprecation messages. False by default.
`:optimise`	Generates faster bytecode by applying optimisations to the program.
`:other`	Array of options passed to the compiler (e.g. `:other=>'-Xprint-types'`).
`:source`	Source code compatibility (e.g. ‘1.5’).
`:target`	Bytecode compatibility (e.g. ‘1.4’).
`:warnings`	Issue warnings when compiling. True when running in verbose mode.

You may use fsc, the Fast Scala Compiler, which submits compilation jobs to a compilation daemon, by setting the environment variable USE_FSC to yes. Note that fsc may cache class libraries—don’t forget to run fsc -reset if you upgrade a library.

Compiling Groovy

Before using the Groovy compiler, you must first require it on your buildfile:

   require 'buildr/java/groovyc'

Once loaded, the groovyc compiler will be automatically selected if any .groovy source files are found under src/main/groovy directory, compiling them by default into the target/classes directory.

If the project has java sources in src/main/java they will get compiled using the groovyc joint compiler.

Sources found in src/test/groovy are compiled into the target/test/classes.

If you don’t specify the packaging type, it defaults to JAR.

The Groovy compiler supports the following options:

Option	Usage
`encoding`	Encoding of source files.
`verbose`	Asks the compiler for verbose output, true when running in verbose mode.
`fork`	Whether to execute groovyc using a spawned instance of the JVM. Defaults to no.
`memoryInitialSize`	The initial size of the memory for the underlying VM, if using fork mode, ignored otherwise. Defaults to the standard VM memory setting. (Examples: `83886080`, `81920k`, or `80m`)
`memoryMaximumSize`	The maximum size of the memory for the underlying VM, if using fork mode, ignored otherwise. Defaults to the standard VM memory setting. (Examples: `83886080`, `81920k`, or `80m`)
`listfiles`	Indicates whether the source files to be compiled will be listed. Defaults to no.
`stacktrace`	If true each compile error message will contain a stacktrace.
`warnings`	Issue warnings when compiling. True when running in verbose mode.
`debug`	Generates bytecode with debugging information. Set from the debug environment variable/global option.
`deprecation`	If true, shows deprecation messages. False by default.
`optimise`	Generates faster bytecode by applying optimisations to the program.
`source`	Source code compatibility.
`target`	Bytecode compatibility.
`javac`	Hash of options passed to the ant javac task.

Resources

The compile task comes bundled with a resources task. It copies files from the src/main/resources directory into target/resources. Best used for copying files that you want to included in the generated code, like configuration files, i18n messages, images, etc.

The resources task uses a filter that can change files as it copies them from source to destination. The most common use is by mapping values using a hash. For example, to substitute ”${version}” for the project’s version number and ”${copyright}” for “Acme Inc© 2007” :

resources.filter.using 'version'=>version,
  'copyright'=>'Acme Inc (C) 2007'

You can also use profiles to supply a name/value map that all resources task should default to, by adding a filter element to each of the profiles. The following examples shows a profiles.yaml file that applies the same filter in development and test environments:

filter: &alpha1
  version: experimental
  copyright: Acme Inc (C) 2007

development:
  filter: *alpha1
test:
  filter: *alpha1

You can specify a different format by passing it as the first argument. Supported formats include:

Format	Usage
`:ant`	Map from `@key@` to value.
`:maven`	Map from `${key}` to value (default).
`:ruby`	Map from `#{key}` to value.
`Regexp`	Map using the matched value of the regular expression (e.g. `/=(.*?)=/`).

For example, using the :ruby format instead of the default :maven format:

resources.filter.using :ruby, 'version'=>version,
  'copyright'=>'Acme Inc (C) 2007'

For more complicated mapping you can also pass a method or a proc. The filter will call it once for each file with the file name and content.

If you need to copy resource files from other directories, add these source directories by calling the from method, for example:

resources.from _('src/etc')

You can select to copy only specific files using common file matching patterns. For example, to include only HTML files:

resources.include '*.html'

To include all files, except for files in the scratch directory:

resources.exclude 'scratch/*'

The filter always excludes the CVS and .svn directories, and all files ending with .bak or ~, so no need to worry about these.

A file pattern can match any file name or part of a file name using an asterisk (*). Double asterisk (**) matches directories recursively, for example, 'src/main/java/**/*.java'. You can match any character using a question mark (?), or a set of characters using square brackets ([]), similar to regular expressions, for example, '[Rr]eadme'. You can also match from a set of names using curly braces ({}), for example, '*.{html,css}'.

You can use filters elsewhere. The filter method creates a filter, the into method sets the target directory, and using specifies the mapping. Last, you call run on the filter to activate it.

For example:

filter('src/specs').into('target/specs').
  using('version'=>version, 'created'=>Time.now).run

The resources task is, in fact, just a wrapper around such a filter that automatically adds the src/main/resources directory as one of the source directories.

More On Building

The build task runs the compile (and resources) tasks as prerequisites, followed by any actions you add to it, and completes by running the test task. The build task itself is a prerequisite to other tasks, for example, package and upload.

You can extend the build task in two ways. You can add more prerequisites that will execute before the task itself, or you can add actions that will execute as part of the task. Which one you choose is up to you, we’ll show you how they differ in a second. If you call build with a list of tasks, it adds these tasks as prerequisites. Call build with a block, and it adds that block as an action. Again, a common idiom you’ll see elsewhere in Buildr and Rake.

Let’s look at a simple example. Say we want to generate a Derby database from an SQL file and include it in the ZIP package:

db = Derby.create(_('target/derby/db')=>_('src/main/sql/derby.sql'))
package(:zip).include db

There’s nothing fundamentally wrong with this code, if that’s what you intend to do. But in practice, you don’t always run the package task during development, so you won’t notice if something is wrong with this task when you build. For example, if it fails to generate the SQL file. In addition, the package task runs after build, so you can’t use the database in your test cases.

So let’s refactor it. We’re going to use the variable db to reference the file task that creates the database, and make it a prerequisite of the build task. And use that same variable again to include the database in the ZIP package:

db = Derby.create(_('target/derby/db')=>_('src/main/sql/derby.sql'))
build db
package(:zip).include db

Much better. We’re using the same task twice, but since we’re using Rake here, it will only execute once. In fact, it will only execute if we don’t already have a Derby database, or if it detects a change to the SQL file and needs to recreate the database.

Derby.create is not part of Buildr, you can get derby.rake here.

Here’s another example. We want to copy some files over as part of the build, and apply a filter to them. This time, we’re going to extend the build task:

build do
  filter('src/specs').into('target/specs').
    using('version'=>version, 'created'=>Time.now).run
end

The build task is recursive, so running buildr build picks the current project and runs its build task, which in turn runs the build task on each of its sub-projects. One build task to rule them all.

Cleaning

The build task has an evil twin, the clean task. It’s the task you use to remove all the files created during the build, especially when you mess things up and want to start all over.

It basically erases the target directories, the one called target, and if you get creative and change the target directory for tasks like compile, it will also erase those. If you decide to generate files outside the target directory and want to cleanup after yourself, just extend the clean task.

For example:

clean { rm_rf _('staged') }

The rm_rf method deletes the directory and all files in it. It’s named after UNIX’s infamous rm -rf. Use it wisely. This is also a good time to introduce you to FileUtils, a standard Ruby library that contains convenient methods for creating and deleting directories, copying and moving files, even comparing two files. They’re all free of charge when you use Buildr.

Now let’s talk about the artifacts we mentioned before.