Gradle Incremental Builds: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(133 intermediate revisions by the same user not shown)
Line 3: Line 3:
* https://docs.gradle.org/current/userguide/custom_tasks.html#incremental_tasks
* https://docs.gradle.org/current/userguide/custom_tasks.html#incremental_tasks
* https://docs.gradle.org/current/userguide/build_cache.html
* https://docs.gradle.org/current/userguide/build_cache.html
* https://blog.gradle.org/introducing-incremental-build-support
* https://medium.com/cirruslabs/mastering-gradle-caching-and-incremental-builds-37eb1af7fcde


=Internal=
=Internal=
* [[Gradle_Task#Incremental_Builds|Gradle Tasks]]
* [[Gradle_Task#Incremental_Builds|Gradle Tasks]]
* [[Gradle File Resolution]]


=Incremental Build=
=Incremental Build=
An incremental build is a build that avoids running tasks whose inputs and outputs did not change since the previous build, making the execution of such tasks unnecessary. An incremental build does not execute a task if it does not need to. Incremental builds save time, because they avoid doing unnecessary and some times lengthy work.  
An incremental build is a build that avoids running tasks whose inputs and outputs did not change since the previous build, making the execution of such tasks unnecessary. An incremental build does not execute a task if it does not need to. The build determines whether is needed or not to run the task by performing [[#up-to-date-check|up-to-date checks]]. Incremental builds save time, because they avoid doing unnecessary and some times lengthy work.  


The central element that enables such behavior is the capacity of tasks to analyze their [[#Task_Inputs_and_Outputs|inputs and outputs]] relative to the last build, detect that nothing that would require it to run changed and decide that they don't need to run. To be considered as part of an incremental build, a task must have at least one [[#Outputs|output]].
The central element that enables such behavior is the capacity of Gradle to analyze a task's [[#Task_Inputs_and_Outputs|inputs and outputs]] relative to the last build, detect that nothing that would require the task to run changed and decide that it does not need to run. A task must have at least one [[#Outputs|output]] to be considered as part of an incremental build. If it does not, it will be executed every time the build runs.


An even more sophisticated behavior is identifying which of the input elements had changed and perform work only for those elements. An incremental build is not required to support this behavior, it only has to avoid to run tasks unnecessarily. However, Gradle supports the behavior described above with [[Gradle Incremental Builds#Incremental_Task|incremental tasks]].
An even more sophisticated behavior consists in identifying which of the input elements had changed and perform work only for those elements. An incremental build is not required to support this behavior, it only has to avoid to run tasks unnecessarily. However, Gradle supports the behavior described above with [[Gradle_Incremental_Task|incremental tasks]].


All builds in Gradle are incremental builds by default, [[#Build_Cache|Build caching]] does not have to be enabled to allow them.
All builds in Gradle are incremental builds by default, [[#Build_Cache|Build caching]] does not have to be enabled to allow them.
 
A particular build can be configured to run all its task even if they are all up-to-date and meet the condition for an incremental build. For more details see [[#Disable_Incremental_Builds|Disable Incremental Builds]] below.


=Task Inputs and Outputs=
=Task Inputs and Outputs=
In most cases, a task takes some [[#Inputs|inputs]] and generates some [[#Outputs|outputs]]. For Java compilation, the inputs are the source files, target JDK version, whether we want debugging information or not, and the outputs are the generated class files. The amount of memory allocated to the compiler is not an input, but an [[#Internal_Task_Properties|internal task property]], as its change does not influence the tasks' outputs.
In most cases, a task takes some [[#Inputs|inputs]] and generates some [[#Outputs|outputs]]. For Java compilation, the inputs are the source files, target JDK version, whether we want debugging information or not, and the outputs are the generated class files. The amount of memory allocated to the compiler is not an input, but an [[#Internal_Task_Properties|internal task property]], as its change does not influence the tasks' outputs.


When it is about to execute a task, Gradle tests externally to the task whether any of the following changed from the last build:
Task inputs and outputs are evaluated during the configuration phase to wire up the task dependencies (but not to decide whether a task is executed or not, as UP-TO-DATE; that evaluation is done at execution time). This is why inputs and outputs need to be defined in a configuration block. To avoid unexpected behavior, make sure that the value you assign to inputs and outputs is accessible at configuration time. If you need to implement programmatic output evaluation, the method upToDateWhen(Closure) on TaskOutputs can be used. In contrast to regular inputs/outputs evaluation, this method is evaluated at execution time. If the closure returns true, the task is considered up to date.
 
==<span id='Up-to-date_Check'></span><span id='up-to-date-check'></span>Up-to-Date Check==
When it is about to execute a task, during the [[Gradle_Concepts#Execution_Phase|execution phase]], Gradle performs an "up-to-date" check: it tests externally to the task whether any of the following changed from the last build:
* any of the task inputs changed.
* any of the task inputs changed.
* any of the task outputs changed or it was removed.
* any of the task outputs changed or it was removed.
* the task code changed.
* the task code changed.
If none of the above elements changed, Gradle considers the task [[Gradle_Task#UP-TO-DATE|UP-TO_DATE]] and skips executing its actions. [[Gradle_Command_Line#-i.2C--info|gradle -i]] will display this. To be considered as part of an incremental build, a task must have at least one [[#Outputs|output]].
If none of the above elements changed, Gradle considers the task to be [[Gradle_Task#UP-TO-DATE|UP-TO_DATE]] and skips executing its actions. If gradle is executed with the [[Gradle_Command_Line#-i.2C--info|-i command line option]], the task outcome will be displayed. To be considered as part of an incremental build, a task must have at least one [[#Outputs|output]].


The cleanest way to designate task [[#Input|inputs]] and [[#Outputs|outputs]] is with annotation when implementing the task as a [[Extending Gradle with a Custom Enhanced Task|custom enhanced task]]: the corresponding properties are marked with annotations. Alternatively, a [[Gradle_Incremental_Builds#Declaring_Inputs_and_Outputs_with_Runtime_API|Runtime API]] can be used to achieve the same results. The Runtime API can be used with [[Gradle_Task#Simple_Task|simple tasks]].
The cleanest way to designate task [[#Input|inputs]] and [[#Outputs|outputs]] is by using annotation when implementing the task as a [[Extending Gradle with a Custom Enhanced Task|custom enhanced task]]: the corresponding getter method for the property should be marked with annotations, as shown below. ⚠️ If the task is written in Java, the annotation must be attached to the getter method. Annotations on setters or on the field will be ignored. Groovy allows annotating properties as well.


The remaining of this section shows how to declare inputs and outputs by annotating the getter method of the corresponding property with appropriate annotations. ⚠️ In Java, the annotation must be attached to the getter method. Annotations on setters or on the field will be ignored. Groovy allows annotating properties as well.
Alternatively, a [[Gradle_Incremental_Builds#Declaring_Inputs_and_Outputs_with_Runtime_API|Runtime API]] can be used to achieve the same results. The Runtime API can be used with [[Gradle_Task#Simple_Task|simple tasks]].


==<span id='Input'></span>Inputs==
==<span id='Input'></span>Inputs==
Line 33: Line 41:


If a task property affects the output of the task, it must be registered as [[#Input|input]], otherwise the task will be considered [[Gradle_Task#UP-TO-DATE|UP-TO-DATE]] when the corresponding property changes, and the task is not actually up-to-date. Conversely, properties that do not affect the output must not be registered as inputs, otherwise the task will potentially execute when does not need to.
If a task property affects the output of the task, it must be registered as [[#Input|input]], otherwise the task will be considered [[Gradle_Task#UP-TO-DATE|UP-TO-DATE]] when the corresponding property changes, and the task is not actually up-to-date. Conversely, properties that do not affect the output must not be registered as inputs, otherwise the task will potentially execute when does not need to.
===Simple Values===
 
Task inputs can be obtained with [https://docs.gradle.org/current/javadoc/org/gradle/api/Task.html#getInputs-- Task.getInputs()] which returns a [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskInputs.html TaskInputs] instances.
===<span id='Simple_Values'></span>Simple Values as @Input===
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Input.html @Input]}}
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Input.html @Input]}}
Strings or numbers, and in general, any type that implements Serializable, can be handled as a simple value. Java example:
Strings or numbers, and in general, any type that implements Serializable, can be handled as a simple value. Enums implement Serializable automatically, so they can be used here. Java example:
<syntaxhighlight lang='java'>
<syntaxhighlight lang='java'>
private String customVersion;
private String customVersion;
Line 47: Line 57:
   this.customVersion = s;
   this.customVersion = s;
}
}
</syntaxhighlight>
Note that the result of the @Input evaluation is displayed when running with [[Gradle_Command_Line#-i.2C--info|-i command line option]]:
<syntaxhighlight lang='text'>
> Task :example
Task ':example' is not up-to-date because:
  Value of input property 'customVersion' has changed for task ':example'
...
</syntaxhighlight>
</syntaxhighlight>
Playground Example:{{External|[https://github.com/ovidiuf/playground/tree/master/gradle/extending-gradle/incremental-builds/02-%40Input Simple @Input property]}}


===File Types===
===File Types===
Gradle detects if at least one file or directory changed, and executes the task as not UP-TO-DATE. It is possible to optimize the behavior even further, and only handle the file(s) or directory(es) that changed, but that is the responsibility of task, not Gradle. If the task is capable to handle these situations efficiently, it is called an [[Gradle Incremental Task|incremental task]]. Gradle does help task implementers via its [[Gradle Incremental Task|incremental task input]] feature.


====@InputFile====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFile.html @InputFile]}}
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFile.html @InputFile]}}
@InputFile can be used to annotate a single input file (not directory). Gradle will consider the task out-of-date when the file path or contents have changed.
Playground Example:{{External|[https://github.com/ovidiuf/playground/tree/master/gradle/extending-gradle/incremental-builds/03-%40InputFile @InputFile property]}}
====@InputDirectory====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputDirectory.html @InputDirectory]}}
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputDirectory.html @InputDirectory]}}
@InputDirectory can be used to annotate a single input directory (not file). Gradle will consider the task out-of-date when the directory location or contents have changed. To make the task dependent on the directory's location but not its contents, expose the path of the directory as an [[#Simple_Values_as_.40Input|@Input]] property instead.
====@InputFiles====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFiles.html @InputFiles]}}
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFiles.html @InputFiles]}}
@InputFiles can be used to annotate an iterable of input files and directories. The properties that can be annotated with these annotations are standard Java File instances, but also derivatives of Gradle's [[Gradle_File_Resolution#FileCollection|org.gradle.api.file.FileCollection]] type and anything else that can be passed to either the [[Gradle_File_Resolution#Project_file|Project.file(Object)]] method for single/file directory properties or the [[Gradle_File_Resolution#Project_file|Project.files(Object...)]] method. This will cause the task to be considered out-of-date when the file paths or contents have changed.
====@SkipWhenEmpty====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/SkipWhenEmpty.html @SkipWhenEmpty]}}
Used with [[#@InputFiles|@InputFiles]] or [[#@InputDirectory|@InputDirectory]] to tell Gradle to skip the task if the corresponding files or directory are empty, along with all other input files declared with this annotation. Tasks that have been skipped due to all of their input files that were declared with this annotation being empty will result in a distinct "no source" outcome. For example, NO-SOURCE will be emitted in the console output. Implies [[#@Incremental|@Incremental]].


[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFile.html @InputFile] can be used to annotate a single input file (not directory). [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputDirectory.html @InputDirectory] can be used to annotate a single input directory (not file). [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/InputFiles.html @InputFiles] can be used to annotate an iterable  of input files and directories. The properties that can be annotated with these annotations are standard Java File instances, but also derivatives of Gradle's [https://docs.gradle.org/current/javadoc/org/gradle/api/file/FileCollection.html org.gradle.api.file.FileCollection] type and anything else that can be passed to either the [https://docs.gradle.org/current/javadoc/org/gradle/api/Project.html#file-java.lang.Object- Project.file(Object)] method for single/file directory properties or the [https://docs.gradle.org/current/javadoc/org/gradle/api/Project.html#files-java.lang.Object...- Project.files(Object...)] method.
====@Incremental====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/work/Incremental.html @Incremental]}}
Used with [[#@InputFiles|@InputFiles]] or [[#@InputDirectory|@InputDirectory]] to instruct Gradle to track changes to the annotated file property, so the changes can be queried via @InputChanges.getFileChanges(). Required for [[Gradle_Incremental_Task|incremental tasks]].


===Nested Values===
====@PathSensitive====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Nested.html org.gradle.api.tasks.@Nested]}}
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/PathSensitive.html @PathSensitive]}}
@Nested can be used to annotate custom types that do not conform the other two categories but have their own properties that are inputs or outputs - the task inputs or outputs are nested inside these custom types.
Used with any input file property to tell Gradle to only consider the given part of the file paths as important. For
example, if a property is annotated with @PathSensitive(PathSensitivity .NAME_ONLY), then moving the files around without changing their contents will not make the task out-of-date.
 
===Classpath Types===
====@Classpath====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Classpath.html @Classpath]}}
An iterable of input files and directories that represent a Java classpath. This allows the task to ignore irrelevant changes to the property, such as different names for the same files. It is similar to annotating the property [[#@PathSensitive|@PathSensitive(RELATIVE)]] but it will ignore the names of JAR files directly added to the classpath, and it will consider changes in the order of the files as a change in the classpath. Gradle will inspect the contents of jar files on the classpath and ignore changes that do not affect the semantics of the classpath (such as file dates and entry order).
 
====@CompileClasspath====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/CompileClasspath.html @CompileClasspath]}}
An iterable of input files and directories that represent a Java compile classpath. This allows the task to ignore irrelevant changes that do not affect the API of the classes in classpath. The following kinds of changes to the classpath will be ignored:
* Changes to the path of jar or top level directories.
* Changes to timestamps and the order of entries in Jars.
* Changes to resources and Jar manifests, including adding or removing resources.
* Changes to private class elements, such as private fields, methods and inner classes.
* Changes to code, such as method bodies, static initializers and field initializers (except for constants).
* Changes to debug information, for example when a change to a comment affects the line numbers in class debug information.
* Changes to directories, including directory entries in Jars.
 
===<span id='Nested_Values'></span>Nested Values and @Nested===
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Nested.html @Nested]}}
@Nested can be used to annotate custom types that do not conform the other two categories but have their own properties that are inputs or outputs - the task inputs or outputs are nested inside these custom types. The custom type may not implement Serializable but does have at least one field or property marked with one of the [[#Inputs|input annotations]]. @Nested can be specified recursively.


===Internal Task Properties===
===Internal Task Properties===
Anything that is not a task input, meaning that it does not influence one or more [[#Outputs|outputs]], it is an internal task property.
Anything that is not a task input, meaning that it does not influence one or more [[#Outputs|outputs]], it is an internal task property.
====@Internal====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Internal.html @Internal]}}
Indicates that the property is used internally but is neither an input nor an output.


==Outputs==
==Outputs==
To be considered as part of an incremental build, a task must have at least one [[#Outputs|output]] and zero or more [[#Inputs|inputs]].
An output is a directory or one or more files. To be considered as part of an incremental build, a task must have at least one [[#Outputs|output]] and zero or more [[#Inputs|inputs]]. If a task declares outputs but those are not configured in the configuration phase, the build will fail with a message similar to "No value has been specified for property '...'". Task outputs can be obtained with [https://docs.gradle.org/current/javadoc/org/gradle/api/Task.html#getOutputs-- Task.getOutputs()] which returns a [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskOutputs.html TaskOutputs] instance.
 
===Output Annotations===
====@OutputFile====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/OutputFile.html @OutputFile]}}
A single output file (not a directory).
 
====@OutputDirectory====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/OutputDirectory.html @OutputDirectory]}}
A single output directory (not a file).
 
====@OutputFiles====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/OutputFiles.html @OutputFiles]}}
An iterable or map of output files. Using a [[Gradle_File_Resolution#File_Tree|file tree]] turns [[Gradle_Build_Cache#Task_Output_Caching|caching]] off for the task.
 
====@OutputDirectories====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/OutputDirectories.html @OutputDirectories]}}
An iterable or map of output directories. Using a [[Gradle_File_Resolution#File_Tree|file tree]] turns [[Gradle_Build_Cache#Task_Output_Caching|caching]] off for the task.


If a task declares outputs but those are not configured in the configuration phase, the build will fail with a message similar to "No value has been specified for property '...'".
===Outputs used as Inputs for another Tasks===
<font color=darkgray>
Expand on this. The palantir docker task accepts tasks.jar.outputs [[Com.palantir.docker#files|as "files"]].
</font>


==Declaring Inputs and Outputs with Runtime API==
==Declaring Inputs and Outputs with Runtime API==
Annotations in custom task types are the cleanest way to declare inputs and outputs. However, simple tasks can be configured to participate in incremental builds by using the Runtime API.
Annotations in custom task types are the cleanest way to declare inputs and outputs. However, simple tasks can be configured to participate in incremental builds by using the Runtime API.
{{Internal|Gradle Incremental Builds - Declaring Inputs and Outputs with Runtime API|Declaring Inputs and Outputs with Runtime API}}
==Input and Output Validation==
<font color=darkgray>TODO</font>


<font color=darkgray>TODO https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:task_input_output_runtime_api</font>
=<span id='.40Destroys_and_Destroyables'></span>Destroyables=
====@Destroys====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Destroys.html @Destroys]}}
Specifies one or more files that are removed by this task. Note that a task can define either inputs/outputs or destroyables, but not both.


==Examples==
=Task Local State=
====@LocalState====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/LocalState.html @LocalState]}}
@LocalState specifies one or more files that represent the local state of the task. These files are removed when the task is loaded from cache.


* [https://docs.gradle.org/current/userguide/more_about_tasks.html#sec:task_input_output_annotations Gradle documentation example]
=Other Annotations=
* [https://github.com/ovidiuf/playground/tree/master/gradle/extending-gradle/incremental-builds/01-one-output-zero-inputs Playground incremental build for a task with one output and zero inputs].
====@Console====
* [https://github.com/ovidiuf/playground/tree/master/gradle/extending-gradle/incremental-builds/02-simple-input Playground simple value input].
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Console.html @Console]}}
The annotation indicates that the annotated property is neither an input nor an output. It simply affects the console output of the task in some way, such as increasing or decreasing the verbosity of the task.
====@ReplacedBy====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/model/ReplacedBy.html @ReplacedBy]}}
Indicates that the property has been replaced by another and should be ignored as an input or output.
====@Optional====
{{External|[https://docs.gradle.org/current/javadoc/org/gradle/api/model/Optional.html @Optional]}}
Used with any of the property type annotations listed in the [https://docs.gradle.org/current/javadoc/org/gradle/api/model/Optional.html Optional] API documentation. This annotation disables validation checks on the corresponding property. Also see [[#Input_and_Output_Validation|Input and Output Validation]].


=Non-Deterministic Tasks=
=Non-Deterministic Tasks=
If a task generates different output for exactly the same inputs, these tasks should not be configured for an incremental build, by declaring [[#Input|inputs]] and [[#Outputs|outputs]], as up-to-date checks won't work and the task will not be executed when it should be.
If a task generates different output for exactly the same inputs, these tasks should not be configured for an incremental build, by declaring [[#Input|inputs]] and [[#Outputs|outputs]], as up-to-date checks won't work and the task will not be executed when it should be.


=Incremental Task=
=<span id='Incremental_Task'></span>Incremental Tasks=
{{Internal|Gradle Incremental Task|Incremental Task}}
{{Internal|Gradle Incremental Task|Incremental Task}}
=Disable Incremental Builds=
A build can be configured via command line to run all its task even if they are all up-to-date and meet the condition for an incremental build with the <code>--rerun-tasks</code> global option:
<syntaxhighlight lang='bash'>
gradle --rerun-tasks ...
</syntaxhighlight>
=Build Cache=
=Build Cache=
Gradle allows caching task output in [[Gradle Build Cache#Overview|build caches]], if the tasks producing those outputs are [[Gradle Build Cache#Cacheable_Task|cacheable tasks]], hence enabling sharing of outputs between builds. Build caching does not have to be enabled to allow [[#Incremental_Build|incremental builds]].
Gradle allows caching task output in [[Gradle Build Cache#Overview|build caches]], if the tasks producing those outputs are [[Gradle Build Cache#Cacheable_Task|cacheable tasks]], hence enabling sharing of outputs between builds. Build caching does not have to be enabled to allow [[#Incremental_Build|incremental builds]].
{{Internal|Gradle Build Cache#Overview|Builds Cache}}
{{Internal|Gradle Build Cache#Overview|Builds Cache}}
-----------------
==To process==
<font color=darkgray>
Gradle determines whether a task is up to date if the task's inputs and outputs have not changed since the last execution, by comparing their values with those preserved in the [[Gradle_Build_Cache#Overview|build cache]].
Does the build cache need to be enabled for inputs and outputs to be relevant?
If the input and output snapshots are identical to the previous ones, the task is not executed. An input can be a property, a directory or one or more files. An output is a directory or one or more files. Inputs and outputs are fields of the DefaultTask class. Task inputs and outputs are obtained with [https://docs.gradle.org/current/javadoc/org/gradle/api/Task.html#getInputs-- Task.getInputs()] and [https://docs.gradle.org/current/javadoc/org/gradle/api/Task.html#getOutputs-- Task.getOutputs()], which return [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskInputs.html TaskInputs] and [https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskOutputs.html TaskOutputs] instances, respectively.
Task inputs and outputs are evaluated during the configuration phase to wire up the task dependencies. That is why they need to be defined in a configuration block. To avoid unexpected behavior, make sure that the value you assign to inputs and outputs is accessible at configuration time. If you need to implement programmatic output evaluation, the method upToDateWhen(Closure) on TaskOutputs comes in handy. In contrast to regular inputs/outputs evaluation, this method is evaluated at execution time. If the closure returns true, the task is considered up to date.
Debug a Java compilation task to see how input/output works.</font>
===Annotations===
====@Input====
{{External|https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/Input.html}}
====@OutputFile====
{{External|https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/OutputFile.html}}
=Up-to-date Check=
[[Gradle_Task#UP-TO-DATE|UP-TO-DATE]]

Latest revision as of 05:17, 13 June 2021

External

Internal

Incremental Build

An incremental build is a build that avoids running tasks whose inputs and outputs did not change since the previous build, making the execution of such tasks unnecessary. An incremental build does not execute a task if it does not need to. The build determines whether is needed or not to run the task by performing up-to-date checks. Incremental builds save time, because they avoid doing unnecessary and some times lengthy work.

The central element that enables such behavior is the capacity of Gradle to analyze a task's inputs and outputs relative to the last build, detect that nothing that would require the task to run changed and decide that it does not need to run. A task must have at least one output to be considered as part of an incremental build. If it does not, it will be executed every time the build runs.

An even more sophisticated behavior consists in identifying which of the input elements had changed and perform work only for those elements. An incremental build is not required to support this behavior, it only has to avoid to run tasks unnecessarily. However, Gradle supports the behavior described above with incremental tasks.

All builds in Gradle are incremental builds by default, Build caching does not have to be enabled to allow them.

A particular build can be configured to run all its task even if they are all up-to-date and meet the condition for an incremental build. For more details see Disable Incremental Builds below.

Task Inputs and Outputs

In most cases, a task takes some inputs and generates some outputs. For Java compilation, the inputs are the source files, target JDK version, whether we want debugging information or not, and the outputs are the generated class files. The amount of memory allocated to the compiler is not an input, but an internal task property, as its change does not influence the tasks' outputs.

Task inputs and outputs are evaluated during the configuration phase to wire up the task dependencies (but not to decide whether a task is executed or not, as UP-TO-DATE; that evaluation is done at execution time). This is why inputs and outputs need to be defined in a configuration block. To avoid unexpected behavior, make sure that the value you assign to inputs and outputs is accessible at configuration time. If you need to implement programmatic output evaluation, the method upToDateWhen(Closure) on TaskOutputs can be used. In contrast to regular inputs/outputs evaluation, this method is evaluated at execution time. If the closure returns true, the task is considered up to date.

Up-to-Date Check

When it is about to execute a task, during the execution phase, Gradle performs an "up-to-date" check: it tests externally to the task whether any of the following changed from the last build:

  • any of the task inputs changed.
  • any of the task outputs changed or it was removed.
  • the task code changed.

If none of the above elements changed, Gradle considers the task to be UP-TO_DATE and skips executing its actions. If gradle is executed with the -i command line option, the task outcome will be displayed. To be considered as part of an incremental build, a task must have at least one output.

The cleanest way to designate task inputs and outputs is by using annotation when implementing the task as a custom enhanced task: the corresponding getter method for the property should be marked with annotations, as shown below. ⚠️ If the task is written in Java, the annotation must be attached to the getter method. Annotations on setters or on the field will be ignored. Groovy allows annotating properties as well.

Alternatively, a Runtime API can be used to achieve the same results. The Runtime API can be used with simple tasks.

Inputs

The essential characteristic of an input is whether it affects one or more outputs in any way. In the above example, source files, target JDK version, enabling debugging are all task inputs. The amount of memory allocated to the compiler is not an input, but an internal task property.

If a task property affects the output of the task, it must be registered as input, otherwise the task will be considered UP-TO-DATE when the corresponding property changes, and the task is not actually up-to-date. Conversely, properties that do not affect the output must not be registered as inputs, otherwise the task will potentially execute when does not need to.

Task inputs can be obtained with Task.getInputs() which returns a TaskInputs instances.

Simple Values as @Input

@Input

Strings or numbers, and in general, any type that implements Serializable, can be handled as a simple value. Enums implement Serializable automatically, so they can be used here. Java example:

private String customVersion;

@Input
public String getCustomVersion() {
  return customVersion;
}

public void setCustomVersion(String s) {
  this.customVersion = s;
}

Note that the result of the @Input evaluation is displayed when running with -i command line option:

> Task :example
Task ':example' is not up-to-date because:
  Value of input property 'customVersion' has changed for task ':example'
...

Playground Example:

Simple @Input property

File Types

Gradle detects if at least one file or directory changed, and executes the task as not UP-TO-DATE. It is possible to optimize the behavior even further, and only handle the file(s) or directory(es) that changed, but that is the responsibility of task, not Gradle. If the task is capable to handle these situations efficiently, it is called an incremental task. Gradle does help task implementers via its incremental task input feature.

@InputFile

@InputFile

@InputFile can be used to annotate a single input file (not directory). Gradle will consider the task out-of-date when the file path or contents have changed.

Playground Example:

@InputFile property

@InputDirectory

@InputDirectory

@InputDirectory can be used to annotate a single input directory (not file). Gradle will consider the task out-of-date when the directory location or contents have changed. To make the task dependent on the directory's location but not its contents, expose the path of the directory as an @Input property instead.

@InputFiles

@InputFiles

@InputFiles can be used to annotate an iterable of input files and directories. The properties that can be annotated with these annotations are standard Java File instances, but also derivatives of Gradle's org.gradle.api.file.FileCollection type and anything else that can be passed to either the Project.file(Object) method for single/file directory properties or the Project.files(Object...) method. This will cause the task to be considered out-of-date when the file paths or contents have changed.

@SkipWhenEmpty

@SkipWhenEmpty

Used with @InputFiles or @InputDirectory to tell Gradle to skip the task if the corresponding files or directory are empty, along with all other input files declared with this annotation. Tasks that have been skipped due to all of their input files that were declared with this annotation being empty will result in a distinct "no source" outcome. For example, NO-SOURCE will be emitted in the console output. Implies @Incremental.

@Incremental

@Incremental

Used with @InputFiles or @InputDirectory to instruct Gradle to track changes to the annotated file property, so the changes can be queried via @InputChanges.getFileChanges(). Required for incremental tasks.

@PathSensitive

@PathSensitive

Used with any input file property to tell Gradle to only consider the given part of the file paths as important. For example, if a property is annotated with @PathSensitive(PathSensitivity .NAME_ONLY), then moving the files around without changing their contents will not make the task out-of-date.

Classpath Types

@Classpath

@Classpath

An iterable of input files and directories that represent a Java classpath. This allows the task to ignore irrelevant changes to the property, such as different names for the same files. It is similar to annotating the property @PathSensitive(RELATIVE) but it will ignore the names of JAR files directly added to the classpath, and it will consider changes in the order of the files as a change in the classpath. Gradle will inspect the contents of jar files on the classpath and ignore changes that do not affect the semantics of the classpath (such as file dates and entry order).

@CompileClasspath

@CompileClasspath

An iterable of input files and directories that represent a Java compile classpath. This allows the task to ignore irrelevant changes that do not affect the API of the classes in classpath. The following kinds of changes to the classpath will be ignored:

  • Changes to the path of jar or top level directories.
  • Changes to timestamps and the order of entries in Jars.
  • Changes to resources and Jar manifests, including adding or removing resources.
  • Changes to private class elements, such as private fields, methods and inner classes.
  • Changes to code, such as method bodies, static initializers and field initializers (except for constants).
  • Changes to debug information, for example when a change to a comment affects the line numbers in class debug information.
  • Changes to directories, including directory entries in Jars.

Nested Values and @Nested

@Nested

@Nested can be used to annotate custom types that do not conform the other two categories but have their own properties that are inputs or outputs - the task inputs or outputs are nested inside these custom types. The custom type may not implement Serializable but does have at least one field or property marked with one of the input annotations. @Nested can be specified recursively.

Internal Task Properties

Anything that is not a task input, meaning that it does not influence one or more outputs, it is an internal task property.

@Internal

@Internal

Indicates that the property is used internally but is neither an input nor an output.

Outputs

An output is a directory or one or more files. To be considered as part of an incremental build, a task must have at least one output and zero or more inputs. If a task declares outputs but those are not configured in the configuration phase, the build will fail with a message similar to "No value has been specified for property '...'". Task outputs can be obtained with Task.getOutputs() which returns a TaskOutputs instance.

Output Annotations

@OutputFile

@OutputFile

A single output file (not a directory).

@OutputDirectory

@OutputDirectory

A single output directory (not a file).

@OutputFiles

@OutputFiles

An iterable or map of output files. Using a file tree turns caching off for the task.

@OutputDirectories

@OutputDirectories

An iterable or map of output directories. Using a file tree turns caching off for the task.

Outputs used as Inputs for another Tasks

Expand on this. The palantir docker task accepts tasks.jar.outputs as "files".

Declaring Inputs and Outputs with Runtime API

Annotations in custom task types are the cleanest way to declare inputs and outputs. However, simple tasks can be configured to participate in incremental builds by using the Runtime API.

Declaring Inputs and Outputs with Runtime API

Input and Output Validation

TODO

Destroyables

@Destroys

@Destroys

Specifies one or more files that are removed by this task. Note that a task can define either inputs/outputs or destroyables, but not both.

Task Local State

@LocalState

@LocalState

@LocalState specifies one or more files that represent the local state of the task. These files are removed when the task is loaded from cache.

Other Annotations

@Console

@Console

The annotation indicates that the annotated property is neither an input nor an output. It simply affects the console output of the task in some way, such as increasing or decreasing the verbosity of the task.

@ReplacedBy

@ReplacedBy

Indicates that the property has been replaced by another and should be ignored as an input or output.

@Optional

@Optional

Used with any of the property type annotations listed in the Optional API documentation. This annotation disables validation checks on the corresponding property. Also see Input and Output Validation.

Non-Deterministic Tasks

If a task generates different output for exactly the same inputs, these tasks should not be configured for an incremental build, by declaring inputs and outputs, as up-to-date checks won't work and the task will not be executed when it should be.

Incremental Tasks

Incremental Task

Disable Incremental Builds

A build can be configured via command line to run all its task even if they are all up-to-date and meet the condition for an incremental build with the --rerun-tasks global option:

gradle --rerun-tasks ...

Build Cache

Gradle allows caching task output in build caches, if the tasks producing those outputs are cacheable tasks, hence enabling sharing of outputs between builds. Build caching does not have to be enabled to allow incremental builds.

Builds Cache