Com.palantir.docker: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(71 intermediate revisions by the same user not shown)
Line 1: Line 1:
=External=
=External=


* https://plugins.gradle.org/plugin/com.palantir.docker
* https://github.com/palantir/gradle-docker
* https://github.com/palantir/gradle-docker


=Internal=
=Internal=


* [[Gradle_Plugins#Plugin_List|Gradle Plugins]]
* [[Gradle_Plugin_Concepts#External_Plugins|Gradle Plugin Concepts]]


=Overview=
=Overview=
The plugin is a wrapper around "[[Docker_build#Overview|docker build]]" command. It executes the command in the <code>build/docker</code> directory, relative to the project root. The command to be executed is:
<syntaxhighlight lang='text'>
docker build -t <name> .
</syntaxhighlight>
where <code><name></code> is the container [[#name|name]], specified in the plugin [[#Configuration|configuration]]. The command's environment can be inspected running gradle with <code>[[Gradle_Command_Line#-d.2C--debug|-d]]</code> option. The actual docker build execution can be inspected by running gradle with <code>[[Gradle_Command_Line#-i.2C--info|-i]]</code> option:
<syntaxhighlight lang='text'>
Starting process 'command 'docker''. Working directory: /Users/test/myProject/build/docker Command: docker build -t example.com/myImage:0.1.0 .
Successfully started process 'command 'docker''
Sending build context to Docker daemon  255.9MB
> ...
Step 1/3 : FROM example.com/open-jdk:1.0.0
---> f4e98563897e
Step 2/3 : COPY myApp.jar my-entrypoint ./
---> Using cache
---> 254e22a6581e
Step 3/3 : ENTRYPOINT ["/my-entrypoint"]
---> Using cache
---> 9c1b68c170ba
Successfully built 9c1b68c170ba
Successfully tagged example.com/myImage:0.1.0
</syntaxhighlight>


=Latest Version=
=Latest Version=
{{External|https://plugins.gradle.org/m2/com/palantir/docker/com.palantir.docker.gradle.plugin/}}
{{External|https://plugins.gradle.org/m2/com/palantir/docker/com.palantir.docker.gradle.plugin/}}
=Playground=
{{External|https://github.com/ovidiuf/playground/tree/master/gradle/plugins/com.palantir.docker}}


=<span id='Example'>build.gradle=
=<span id='Example'>build.gradle=


Common section:
Using plugin DSL:
<syntaxhighlight lang='groovy'>
plugins {
  id "com.palantir.docker" version "0.32.0"
}
</syntaxhighlight>
 
Alternatively, use legacy format common section:


<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
buildscript {
buildscript {
     ext {
     ext {
         ...
         ...
         dockerGradleVersion = '0.21.0'
         dockerGradleVersion = '0.21.0'
     }
     }
     repositories {
     repositories {
         mavenCentral()
         mavenCentral()
         maven {
         maven {
           
             url "https://plugins.gradle.org/m2/"
             url "https://plugins.gradle.org/m2/"
         }
         }
Line 42: Line 66:


ext {
ext {
   
     dockerNamespace = "com.example/playground"
     dockerNamespace = "com.example/playground"
}
}
...
...
apply plugin: 'com.palantir.docker'
apply plugin: 'com.palantir.docker'
...
...
</syntaxhighlight>
</syntaxhighlight>
Note that this format configures the build to pull the plugin code from https://plugins.gradle.org/m2/gradle/plugin/com/palantir/gradle/docker/gradle-docker, and only old version, up to 0.22.2, are available there.


==Spring Boot Application Container==
==Spring Boot Application Container==
Line 60: Line 82:
docker {
docker {
      
      
  dependsOn build as Task
   // bootJar.baseName resolves to the artifact name, without version and extension information
   // bootJar.baseName resolves to the artifact name, without version and extension information
   name "${dockerNamespace}/${bootJar.baseName}"
   name "${dockerNamespace}/${bootJar.baseName}"
  // by default the image will be tagged with "latest", to tag it with the project version use:
  // name "${dockerNamespace}/${bootJar.baseName}:${version}"


   dockerfile file('Dockerfile')
   dockerfile file('Dockerfile')
Line 73: Line 96:
   buildArgs(['JAR_FILE': "${bootJar.archiveName}"])
   buildArgs(['JAR_FILE': "${bootJar.archiveName}"])
}
}
tasks.getByPath('docker').dependsOn('build')
</syntaxhighlight>
</syntaxhighlight>


Line 91: Line 116:
<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
docker {
docker {
    dependsOn distZip as Task
     //
     //
     // define the distribution file
     // define the distribution file
Line 108: Line 130:
     buildArgs(['DISTRIBUTION_ZIP': distributionZip.name])
     buildArgs(['DISTRIBUTION_ZIP': distributionZip.name])
}
}
tasks.getByPath('docker').dependsOn('distZip')
</syntaxhighlight>
</syntaxhighlight>


Line 122: Line 146:
=Configuration=
=Configuration=


The plugin configuration is applied in a docker {} script block. The following parameters are available:
The plugin configuration is applied in a docker {} script block. All these configuration elements correspond to a DockerExtension instance, and they can be configured in build.gradle only at [[Gradle_Concepts#Configuration_Phase|configuration phase]]. If any of the configuration elements, such as the tag, must be computed dynamically at [[Gradle_Concepts#Execution_Phase|execution phase]], as result of another task execution, it could be done by pulling the DockerExtension instance and invoking the corresponding setter. This is an example of how to do that: {{Internal|Configuring com.palantir.docker tag at Execution Phase|Configuring com.palantir.docker tag at Execution Phase}}
 
 
The following parameters are available:


==name==
==name==
Line 136: Line 163:
</syntaxhighlight>
</syntaxhighlight>


name the name to use for this container, may include a tag
name the name to use for this container, may include a tag. This syntax is convenient if only one tag is applied. If more than one tags are used, there is an alternative syntax that uses the "tag" attribute. See [[#tag|tag]] below.
 
==tag==
 
Specifies one or more tags to be applied then when the container image is placed in the repository. The syntax of the tag configuration parameter is
<font size=-1>
...
tag <''task-name''> <''tag''>
...
</font>
where the task-name is used to create a new tagging task, named "dockerPush-''task-name''"
 
The following configuration:
 
<syntaxhighlight lang='groovy'>
docker {
  ...
  name com.example/my-container
 
  tag "-${version}", "${ecrRepositoryUri}:${version}"
  tag "-latest", "${ecrRepositoryUri}:latest"
  ...
}
</syntaxhighlight>
 
activated with:
<font size=-1>
gradle dockerTagsPush
</font>
will tag twice:
<font size=-1>
> Task :dockerPush'''-1.1-SNAPSHOT'''
The push refers to repository [777777777777.dkr.ecr.us-west-2.amazonaws.com/oa2aws]
b5399c55eada: Preparing
...
1.1-SNAPSHOT: digest: sha256:ad78e5752e1a819b6360bfb3ba5bfa4eb38fa8d88f6fb970d1b0efd483c6ab30 size: 1371
> Task :dockerPush'''-latest'''
The push refers to repository [673499572719.dkr.ecr.us-west-2.amazonaws.com/oa2aws]
b5399c55eada: Preparing
...
latest: digest: sha256: ad78e5752e1a819b6360bfb3ba5bfa4eb38fa8d88f6fb970d1b0efd483c6ab30 size: 1371
</font>
 
Also see [[#Tag_Handling|Tag Handling]] below.


==dockerfile==
==dockerfile==
The dockerfile to use for building the image. It defaults to <code>project.file('Dockerfile')</code>, which implies that the corresponding Dockerfile file exists in the project root, alongside <code>[[build.gradle]]</code>. It must be a file object.


The dockerfile to use for building the image. It defaults to project.file('Dockerfile') and must be a file object.
<syntaxhighlight lang='groovy'>
docker {
  ...
  dockerfile file('Dockerfile')
  ...
}
</syntaxhighlight>


If Dockerfile is maintain in <code>src/main/docker/Dockerfile</code>:
<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
docker {
docker {
   ...
   ...
   dockerfile file('Dockerfile')
   dockerfile file('src/main/docker/Dockerfile')
   ...
   ...
}
}
Line 152: Line 231:
==files==
==files==


An argument list of files to be included in the Docker build context, <font color=darkgray>evaluated per Project#files</font>
An optional comma-separated list of files to be included in the [[Docker_build#The_Build_Context|Docker build context]]. The path specified in the list should be resolvable with [[Gradle_File_Resolution|Project.files(...)]]. All files specified here are physically copied in the docker build context "build/docker".


<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
docker {
docker {
   ...
   ...
   bootJar.archivePath, './build/themyscira-openapi-with-aws-extensions.json'
   files bootJar.archivePath, './build/themyscira-openapi-with-aws-extensions.json', tasks.jar.outputs
   ...
   ...
}
}
</syntaxhighlight>
</syntaxhighlight>
Individual files can be specified on different lines:
<syntaxhighlight lang='groovy'>
docker {
  ...
  files bootJar.archivePath
  files './build/themyscira-openapi-with-aws-extensions.json'
  files tasks.jar.outputs
  ...
}
</syntaxhighlight>
If the "files" value is a directory, then only the directory content (not the directory itself) will be copied recursively in the docker build context "build/docker". A trailing slash in the source directory name will not make any difference.
<font color=darkgray>TODO: finish processing https://github.com/palantir/gradle-docker | Docker Configuration Parameters | "files"</font>


==buildArgs==
==buildArgs==
Line 195: Line 289:


More details about Docker build-time variables: {{Internal|Docker_build#Build-Time_Variables|Docker Build-Time Variables}}
More details about Docker build-time variables: {{Internal|Docker_build#Build-Time_Variables|Docker Build-Time Variables}}
=Tasks=
==docker==
Builds Docker image.
==dockerClean==
Cleans Docker build directory.
==dockerPrepare==
Prepares Docker build directory.
==dockerPush==
Pushes named Docker image to configured Docker registry.
==dockerTag==
Applies all tags to the Docker image.
==dockerTagsPush==
Pushes all tagged Docker images to configured Docker Hub.
==dockerfileZip==
Bundles the configured Dockerfile in a zip file
=Idiosyncrasies=
==com.palantir.docker and Distribution Plugin==
The output of dockerfileZip task is called ''project-name''.zip by default. When the plugin is used together with the [[Gradle_Distribution_Plugin#Overview|distribution plugin]], they break each other. The distribution plugin will output its zip file with the same name, which gets overwritten by the dockerfileZip. Workaround is the following build.gradle snippet:
<syntaxhighlight lang='groovy'>
dockerfileZip {
  baseName 'dockerfile'
}
</syntaxhighlight>
==RUN Executables Failure is Not Reported==
Under some non-elucidated circumstances, the Dockerfile RUN commands silently fail, but the build process is not aborted, leading to incomplete or invalid images. For example, using $(basename ...) failed silently.
==Tag Handling==
If no <code>tag</code> is explicitly specified in <code>docker {...}</code> configuration block, at the <code>name</code> is not postfixed with ":some-tag", <code>./gradlew docker</code> creates an image tagged with <code>latest</code> in the local docker cache. If <code>./gradlew dockerPush</code> is executed, the image is tagged with the current project version in the target registry/repository.


=Operations=
=Operations=


==<span id='Execute'></span>Build the Image and Place it in the Local Registry==
==<span id='Execute'></span>Build the Image and Place it in the Local Registry==
 
<font size=-1>
  gradle clean docker
  gradle clean docker
 
</font>
The command will build the container image and place into the local registry as "com.example/playground/myartifact" with the "latest" tag.
The command will build the container image and place into the local registry as "com.example/playground/myartifact" with the "latest" tag.


==="Normalized" build Task===
==="Normalized" build Task===


If the docker tasks belongs to a dedicated docker sub-project, that does not have a "build" task, the project can be "normalized" with a build task as follows:
If the docker tasks belongs to a dedicated docker sub-project named "docker-sub-project", that does not have a "build" task, the project can be "normalized" with a build task as follows:


<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
task build {
task build {
    dependsOn getTasksByName("docker", false).asList().get(0)
  dependsOn project.tasks.getByPath(":docker-sub-project:docker")
}
}
</syntaxhighlight>
</syntaxhighlight>
Line 220: Line 354:
<syntaxhighlight lang='groovy'>
<syntaxhighlight lang='groovy'>
docker {
docker {
  ...
  dependsOn build
   ...
   ...
}
}
tasks.getByPath('docker').dependsOn('build')
</syntaxhighlight>
</syntaxhighlight>


For more details, see: {{Internal|Gradle_Task#Task_Execution_Order|Task Execution Order}}
For more details, see: {{Internal|Gradle Task Dependencies and Ordering|Task Dependencies and Ordering}}


==Push to External Repository==
==Push to External Repository==


  gradle docker dockerPush
  gradle docker dockerPush
==Avoid Unzipping as a Container Build Operation==
When packaging a ZIP distribution, one of the steps is to unzip the distribution file. This is unnecessary, as Gradle build area already contains the unzipped contents. <font color=darkgray>Come up with the recipe to use that and avoid RUN unzip ... in the Dockerfile. Some ideas are available here: https://github.com/palantir/gradle-docker.</font>


=Execution Internals=
=Execution Internals=


The build context is a local directory similar to /var/lib/docker/tmp/docker-builder036476946.
The build context is a local directory similar to /var/lib/docker/tmp/docker-builder036476946.
=Idiosyncrasies=
==com.palantir.docker and Distribution Plugin==

Latest revision as of 04:54, 9 February 2022

External

Internal

Overview

The plugin is a wrapper around "docker build" command. It executes the command in the build/docker directory, relative to the project root. The command to be executed is:

docker build -t <name> .

where <name> is the container name, specified in the plugin configuration. The command's environment can be inspected running gradle with -d option. The actual docker build execution can be inspected by running gradle with -i option:

Starting process 'command 'docker''. Working directory: /Users/test/myProject/build/docker Command: docker build -t example.com/myImage:0.1.0 .
Successfully started process 'command 'docker''
Sending build context to Docker daemon  255.9MB
> ...
Step 1/3 : FROM example.com/open-jdk:1.0.0
 ---> f4e98563897e
Step 2/3 : COPY myApp.jar my-entrypoint ./
 ---> Using cache
 ---> 254e22a6581e
Step 3/3 : ENTRYPOINT ["/my-entrypoint"]
 ---> Using cache
 ---> 9c1b68c170ba
Successfully built 9c1b68c170ba
Successfully tagged example.com/myImage:0.1.0

Latest Version

https://plugins.gradle.org/m2/com/palantir/docker/com.palantir.docker.gradle.plugin/

Playground

https://github.com/ovidiuf/playground/tree/master/gradle/plugins/com.palantir.docker

build.gradle

Using plugin DSL:

plugins {
  id "com.palantir.docker" version "0.32.0"
}

Alternatively, use legacy format common section:

buildscript {
    ext {
        ...
        dockerGradleVersion = '0.21.0'
    }
    repositories {
        mavenCentral()
        maven {
            url "https://plugins.gradle.org/m2/"
        }
    }
    dependencies {
        ...
        classpath("gradle.plugin.com.palantir.gradle.docker:gradle-docker:${dockerGradleVersion}")
    }
}

ext {
    dockerNamespace = "com.example/playground"
}
...
apply plugin: 'com.palantir.docker'
...

Note that this format configures the build to pull the plugin code from https://plugins.gradle.org/m2/gradle/plugin/com/palantir/gradle/docker/gradle-docker, and only old version, up to 0.22.2, are available there.

Spring Boot Application Container

The docker plugin configuration that creates a SpringBoot application container. This configuration requires the common section, above.

docker {
    
  // bootJar.baseName resolves to the artifact name, without version and extension information
  name "${dockerNamespace}/${bootJar.baseName}"

  // by default the image will be tagged with "latest", to tag it with the project version use:
  // name "${dockerNamespace}/${bootJar.baseName}:${version}"

  dockerfile file('Dockerfile')

  // bootJar.archivePath resolves to the absolute path of the artifact file
  files bootJar.archivePath

  // bootJar.archiveName resolves to the artifact name, including version and extension
  buildArgs(['JAR_FILE': "${bootJar.archiveName}"])
}

tasks.getByPath('docker').dependsOn('build')

The corresponding Dockerfile:

FROM openjdk:8-jdk-alpine
ARG JAR_FILE
COPY ${JAR_FILE} /app.jar
RUN apk --no-cache add curl bash bind-tools
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar"]

Application Plugin Distribution Container

This is the docker plugin configuration that builds a container based on an application distribution built by the Application plugin. This configuration requires the common section, above.

docker {
    //
    // define the distribution file
    //
    String distributionBaseName = ((DistributionContainer)project.extensions.getByName("distributions")).getByName("main").baseName;
    File distributionZip = new File(new File(project.buildDir, "distributions"), distributionBaseName + ".zip")

    name "${dockerNamespace}/${distributionBaseName}:latest"

    dockerfile file('Dockerfile')

    files distributionZip

    buildArgs(['DISTRIBUTION_ZIP': distributionZip.name])
}

tasks.getByPath('docker').dependsOn('distZip')

The corresponding Dockerfile:

FROM openjdk:8-jdk-alpine
ARG DISTRIBUTION_ZIP
COPY ${DISTRIBUTION_ZIP} .
RUN apk --no-cache add curl bash bind-tools unzip; unzip ${DISTRIBUTION_ZIP}; rm ${DISTRIBUTION_ZIP}
ENTRYPOINT ["/oa2aws/bin/oa2aws"]

Configuration

The plugin configuration is applied in a docker {} script block. All these configuration elements correspond to a DockerExtension instance, and they can be configured in build.gradle only at configuration phase. If any of the configuration elements, such as the tag, must be computed dynamically at execution phase, as result of another task execution, it could be done by pulling the DockerExtension instance and invoking the corresponding setter. This is an example of how to do that:

Configuring com.palantir.docker tag at Execution Phase


The following parameters are available:

name

The container name. Includes the namespace and may include a tag. Example:

docker {
  ...
  name com.example/my-container:latest
  ...
}

name the name to use for this container, may include a tag. This syntax is convenient if only one tag is applied. If more than one tags are used, there is an alternative syntax that uses the "tag" attribute. See tag below.

tag

Specifies one or more tags to be applied then when the container image is placed in the repository. The syntax of the tag configuration parameter is

...
tag <task-name> <tag>
...

where the task-name is used to create a new tagging task, named "dockerPush-task-name"

The following configuration:

docker {
  ...
  name com.example/my-container
  
  tag "-${version}", "${ecrRepositoryUri}:${version}"
  tag "-latest", "${ecrRepositoryUri}:latest"
  ...
}

activated with:

gradle dockerTagsPush

will tag twice:

> Task :dockerPush-1.1-SNAPSHOT
The push refers to repository [777777777777.dkr.ecr.us-west-2.amazonaws.com/oa2aws]
b5399c55eada: Preparing
...
1.1-SNAPSHOT: digest: sha256:ad78e5752e1a819b6360bfb3ba5bfa4eb38fa8d88f6fb970d1b0efd483c6ab30 size: 1371

> Task :dockerPush-latest
The push refers to repository [673499572719.dkr.ecr.us-west-2.amazonaws.com/oa2aws]
b5399c55eada: Preparing
...
latest: digest: sha256: ad78e5752e1a819b6360bfb3ba5bfa4eb38fa8d88f6fb970d1b0efd483c6ab30 size: 1371

Also see Tag Handling below.

dockerfile

The dockerfile to use for building the image. It defaults to project.file('Dockerfile'), which implies that the corresponding Dockerfile file exists in the project root, alongside build.gradle. It must be a file object.

docker {
  ...
  dockerfile file('Dockerfile')
  ...
}

If Dockerfile is maintain in src/main/docker/Dockerfile:

docker {
  ...
  dockerfile file('src/main/docker/Dockerfile')
  ...
}

files

An optional comma-separated list of files to be included in the Docker build context. The path specified in the list should be resolvable with Project.files(...). All files specified here are physically copied in the docker build context "build/docker".

docker {
  ...
  files bootJar.archivePath, './build/themyscira-openapi-with-aws-extensions.json', tasks.jar.outputs
  ...
}

Individual files can be specified on different lines:

docker {
  ...
  files bootJar.archivePath
  files './build/themyscira-openapi-with-aws-extensions.json'
  files tasks.jar.outputs
  ...
}

If the "files" value is a directory, then only the directory content (not the directory itself) will be copied recursively in the docker build context "build/docker". A trailing slash in the source directory name will not make any difference.

TODO: finish processing https://github.com/palantir/gradle-docker | Docker Configuration Parameters | "files"

buildArgs

An argument map of string-to-string which will set --build-arg arguments to the docker build command and pass Docker build-time variables. It defaults to empty, which results in no --build-arg parameters.

Single argument:

docker {
  ...
  buildArgs(['DISTRIBUTION_FILE': distribution.name])
}

Multiple arguments:

docker {
  ...
  buildArgs(['DISTRIBUTION_FILE': distribution.name, 'DISTRIBUTION_IMAGE_DIR': distributionImageDir])
}

The arguments passed as such can then be used in the Dockerfile as follows:

...
ARG DISTRIBUTION_FILE
ARG DISTRIBUTION_IMAGE_DIR
COPY ${DISTRIBUTION_FILE} ${DISTRIBUTION_IMAGE_DIR}/${DISTRIBUTION_FILE}
...

More details about Docker build-time variables:

Docker Build-Time Variables

Tasks

docker

Builds Docker image.

dockerClean

Cleans Docker build directory.

dockerPrepare

Prepares Docker build directory.

dockerPush

Pushes named Docker image to configured Docker registry.

dockerTag

Applies all tags to the Docker image.

dockerTagsPush

Pushes all tagged Docker images to configured Docker Hub.

dockerfileZip

Bundles the configured Dockerfile in a zip file

Idiosyncrasies

com.palantir.docker and Distribution Plugin

The output of dockerfileZip task is called project-name.zip by default. When the plugin is used together with the distribution plugin, they break each other. The distribution plugin will output its zip file with the same name, which gets overwritten by the dockerfileZip. Workaround is the following build.gradle snippet:

dockerfileZip {
  baseName 'dockerfile'
}

RUN Executables Failure is Not Reported

Under some non-elucidated circumstances, the Dockerfile RUN commands silently fail, but the build process is not aborted, leading to incomplete or invalid images. For example, using $(basename ...) failed silently.

Tag Handling

If no tag is explicitly specified in docker {...} configuration block, at the name is not postfixed with ":some-tag", ./gradlew docker creates an image tagged with latest in the local docker cache. If ./gradlew dockerPush is executed, the image is tagged with the current project version in the target registry/repository.

Operations

Build the Image and Place it in the Local Registry

gradle clean docker

The command will build the container image and place into the local registry as "com.example/playground/myartifact" with the "latest" tag.

"Normalized" build Task

If the docker tasks belongs to a dedicated docker sub-project named "docker-sub-project", that does not have a "build" task, the project can be "normalized" with a build task as follows:

task build {
  dependsOn project.tasks.getByPath(":docker-sub-project:docker")
}

Declare docker Tasks' Dependencies on other Tasks

In most cases it is necessary to first build the artifacts that will be package in the container image. The task dependency can be expressed as follows:

docker {
  ...
}

tasks.getByPath('docker').dependsOn('build')

For more details, see:

Task Dependencies and Ordering

Push to External Repository

gradle docker dockerPush

Avoid Unzipping as a Container Build Operation

When packaging a ZIP distribution, one of the steps is to unzip the distribution file. This is unnecessary, as Gradle build area already contains the unzipped contents. Come up with the recipe to use that and avoid RUN unzip ... in the Dockerfile. Some ideas are available here: https://github.com/palantir/gradle-docker.

Execution Internals

The build context is a local directory similar to /var/lib/docker/tmp/docker-builder036476946.