Gradle Dependencies and Dependency Configurations: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
(One intermediate revision by the same user not shown)
Line 27: Line 27:
A configuration represents a named group of artifacts and their dependencies.  
A configuration represents a named group of artifacts and their dependencies.  
{{Note|Configuration, in this context, does not refer to a configuration file of any sort, as in a settings configuration [[settings.gradle]] or a build configuration [[build.gradle]]. It refers to named set of artifacts, and it can probably thought of as being a concept similar to a Maven [[Maven_Concepts_-_Dependencies#Dependency_Scope|scope]].}}
{{Note|Configuration, in this context, does not refer to a configuration file of any sort, as in a settings configuration [[settings.gradle]] or a build configuration [[build.gradle]]. It refers to named set of artifacts, and it can probably thought of as being a concept similar to a Maven [[Maven_Concepts_-_Dependencies#Dependency_Scope|scope]].}}
=<span id='The Local Gradle Dependency Cache'></span>The Dependency Cache=
{{External|https://docs.gradle.org/current/userguide/dependency_cache.html}}
Once a dependency is resolved as result of the [[#Dependency_Resolution|dependency resolution]], Gradle stores the associated artifacts in a local cache, known as the '''dependency cache'''. One of the goals of storing artifacts locally in the cache is to minimize the number of remote requests made during the dependency resolution process.
The dependency cache is located under <tt>$GRADLE_USER_HOME/caches</tt> (typically <tt>~/.gradle/caches</tt>). The directory contains two storage types: <span id='File-Based_Storage'></span>'''file-based storage''' for downloaded artifacts and raw metadata files, and a '''binary store''' of resolved module meta-data. This cache is also known as the '''metadata cache'''. The metadata cache contains, among other things, the result of resolving dynamic versions to concrete versions, the resolved module metadata, including module artifacts and module dependencies, the resolved artifact metadata, including a pointer to the downloaded file in the [[#File-Based_Storage|file-based storage]], the absence of a particular module or artifact, eliminating repeated attempts to access a resource that does not exist.
For a specific dependency version, the files are maintained under <tt>.gradle/caches/modules-2/files-*</tt> and the corresponding binary metadata is maintained in <tt>.gradle/caches/modules-2/metadata-*</tt>. For example, the binary artifacts for SLF4J API 1.7.25, where "org.slf4j" is the group ID and "slf4j-api" is the artifact ID, are stored as follows:
.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25
                                                          |
                                                          + df51c4a85dd6acf8b6cdc9323596766b3d577c28/slf4j-api-1.7.25.pom
                                                          |
                                                          + da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar
                                                          |
                                                          + 962153db4a9ea71b79d047dfd1b2a0d80d8f4739/slf4j-api-1.7.25-sources.jar
The corresponding binary metadata is stored as:
.gradle/caches/modules-2/metadata-2.58/descriptors/org.slf4j/slf4j-api/1.7.25
                                                                          |
                                                                          + 1dd858de07b774d6be9d3e38c5646087/descriptor.bin
                                                                          |
                                                                          + 33ddd5448b9e1258ce46034f20c899a8/descriptor.bin
                                                                          |
                                                                          + f8e6315c37eb56998f7a5ba08e30db71/descriptor.bin
Dependency resolution will fail if the required artifacts are not available in any repository specified in the build, even if the local cache has a copy of the artifact that was retrieved from a different repository.
Before downloading an artifact from a repository, Gradle tries to get its checksum first by attempting to download the SHA file associated with the artifact. If the checksum can be retrieved, and Gradle determines that a file with the same id and checksum exists in the cache. If the checksum cannot be retrieved, the artifact will be downloaded (and ignored if it matches an existing artifact).
The cache is periodically scanned (at most every 24 hours) for artifacts that have not been used for more than 30 days and obsolete artifacts are deleted, to ensure the cache does not grow indefinitely.
==Forced Cache Update==
Gradle can be told to ignore all cached entries for resolved modules and artifacts with the [[Gradle_Command_Line#--refresh-dependencies|--refresh-dependencies]] command line option. When used, a fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded. However, where possible Gradle will check if the previously downloaded artifacts are valid before downloading again. This is done by comparing published SHA1 values in the repository with the SHA1 values for existing downloaded artifacts. A new snapshot repository will produce the following log:
gradle clean build --refresh-dependencies
...
Download s3://my-repo/snapshots/com/example/my-artifact/1.0.0-SNAPSHOT/maven-metadata.xml
...
Download s3://my-repo/snapshots/com/example/my-artifact/1.0.0-SNAPSHOT/my-artifact-1.0.0-20181008.231746-18.jar
==Low-Level Cache Manipulation==
It is sometimes necessary to remove individual dependency from cache. <font color=darkgray>TODO.</font>

Revision as of 05:34, 13 January 2021

External

Internal

TO DEPLETE

Gradle Dependencies and Dependency Configurations TODEPLETE

Overview

Dependencies for a project can be managed with Project.dependencies(Closure), Project.getDependencies() and the "dependencies" build.gradle DSL element:

dependencies {
  // configuration closure
}

Configurations for a project can be managed with Project.configurations(Closure), Project.getConfigurations() and the "configurations" build.gradle DSL element:

configurations {
  // configuration closure
}

Dependency

Configuration

DSL Reference https://docs.gradle.org/current/dsl/org.gradle.api.artifacts.Configuration.html

A configuration represents a named group of artifacts and their dependencies.


Configuration, in this context, does not refer to a configuration file of any sort, as in a settings configuration settings.gradle or a build configuration build.gradle. It refers to named set of artifacts, and it can probably thought of as being a concept similar to a Maven scope.

The Dependency Cache

https://docs.gradle.org/current/userguide/dependency_cache.html

Once a dependency is resolved as result of the dependency resolution, Gradle stores the associated artifacts in a local cache, known as the dependency cache. One of the goals of storing artifacts locally in the cache is to minimize the number of remote requests made during the dependency resolution process.

The dependency cache is located under $GRADLE_USER_HOME/caches (typically ~/.gradle/caches). The directory contains two storage types: file-based storage for downloaded artifacts and raw metadata files, and a binary store of resolved module meta-data. This cache is also known as the metadata cache. The metadata cache contains, among other things, the result of resolving dynamic versions to concrete versions, the resolved module metadata, including module artifacts and module dependencies, the resolved artifact metadata, including a pointer to the downloaded file in the file-based storage, the absence of a particular module or artifact, eliminating repeated attempts to access a resource that does not exist.

For a specific dependency version, the files are maintained under .gradle/caches/modules-2/files-* and the corresponding binary metadata is maintained in .gradle/caches/modules-2/metadata-*. For example, the binary artifacts for SLF4J API 1.7.25, where "org.slf4j" is the group ID and "slf4j-api" is the artifact ID, are stored as follows:

.gradle/caches/modules-2/files-2.1/org.slf4j/slf4j-api/1.7.25
                                                         |
                                                         + df51c4a85dd6acf8b6cdc9323596766b3d577c28/slf4j-api-1.7.25.pom
                                                         |
                                                         + da76ca59f6a57ee3102f8f9bd9cee742973efa8a/slf4j-api-1.7.25.jar
                                                         |
                                                         + 962153db4a9ea71b79d047dfd1b2a0d80d8f4739/slf4j-api-1.7.25-sources.jar

The corresponding binary metadata is stored as:

.gradle/caches/modules-2/metadata-2.58/descriptors/org.slf4j/slf4j-api/1.7.25
                                                                         |
                                                                         + 1dd858de07b774d6be9d3e38c5646087/descriptor.bin
                                                                         |
                                                                         + 33ddd5448b9e1258ce46034f20c899a8/descriptor.bin
                                                                         |
                                                                         + f8e6315c37eb56998f7a5ba08e30db71/descriptor.bin


Dependency resolution will fail if the required artifacts are not available in any repository specified in the build, even if the local cache has a copy of the artifact that was retrieved from a different repository.

Before downloading an artifact from a repository, Gradle tries to get its checksum first by attempting to download the SHA file associated with the artifact. If the checksum can be retrieved, and Gradle determines that a file with the same id and checksum exists in the cache. If the checksum cannot be retrieved, the artifact will be downloaded (and ignored if it matches an existing artifact).

The cache is periodically scanned (at most every 24 hours) for artifacts that have not been used for more than 30 days and obsolete artifacts are deleted, to ensure the cache does not grow indefinitely.

Forced Cache Update

Gradle can be told to ignore all cached entries for resolved modules and artifacts with the --refresh-dependencies command line option. When used, a fresh resolve will be performed against all configured repositories, with dynamic versions recalculated, modules refreshed, and artifacts downloaded. However, where possible Gradle will check if the previously downloaded artifacts are valid before downloading again. This is done by comparing published SHA1 values in the repository with the SHA1 values for existing downloaded artifacts. A new snapshot repository will produce the following log:

gradle clean build --refresh-dependencies
...
Download s3://my-repo/snapshots/com/example/my-artifact/1.0.0-SNAPSHOT/maven-metadata.xml
...
Download s3://my-repo/snapshots/com/example/my-artifact/1.0.0-SNAPSHOT/my-artifact-1.0.0-20181008.231746-18.jar

Low-Level Cache Manipulation

It is sometimes necessary to remove individual dependency from cache. TODO.