Docker build: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(31 intermediate revisions by the same user not shown)
Line 21: Line 21:
Starting with Docker 1.10, only RUN, COPY and ADD create layers.
Starting with Docker 1.10, only RUN, COPY and ADD create layers.


As the Dockerfile instructions are evaluated in order, Docker inspects the [[Docker Concepts#Build_Cache|build cache]], looking for cached ''images'' that can be reused, instead of building a duplicate image from scratch. In matching commands to cached images, the following rules are followed:
Dockerfile is read and the instructions listed in it are evaluated in order. If the build cache is used, which is the default unless the [[#--no-cache|--no-cache=true]] option is specified as argument of the build command, the instructions are also looked up against the cache content and if a match is found, the instruction is not executed, but the cached layer is used instead. This usually speeds up builds, but some precautions need to be taken.
* Starting with a parent image that is already in the cache, the next instruction is compared against all cached child images derived from that parent image, to see if one of them was built using ''the exact same instruction''. If no such child image exists, the cache is invalidated.
* The literal instruction is used in comparison (presumably the cache maintains those instructions), with the exception of [[Dockerfile#ADD|ADD]] and [[Dockerfile#COPY|COPY]].
* In case of [[Dockerfile#ADD|ADD]], [[Dockerfile#COPY|COPY]], the contents of the files to be copied are examined and a checksum is calculated for each file. The last-modified and last-accessed times of the file(s) are not considered in these checksums. The corresponding files in cache are also checksummed and the checksums are compared. If anything changed in the file - content or metadata - the cache is invalidated.
* <tt>RUN yum -y update</tt> commands '''require''' --no-cache=true to be effective.
* Once the cache is invalidated, all subsequent Dockerfile commands generate new images and the cache is not used.


The build cache can be explicitly invalidated with:
For details, see: {{Internal|Docker Build Cache#Overview|Docker Build Cache}}
 
--no-cache=true


If a cached image was used, the build command output states that:
If a cached image was used, the build command output states that:
Line 37: Line 30:
   ---> '''Using cache'''
   ---> '''Using cache'''
   ---> 171da11cf0ef
   ---> 171da11cf0ef
The cache can be explicitly invalidated with the [[#--no-cache|--no-cache]] command line option is used.


=The Build Context=
=The Build Context=


The ''build context'' is a set of file located in a path, or at an URL, which are specified in the build command line. The commands in the [[Dockerfile]] are relative to the context.
The '''build context''' is a set of files located in a path, or at an URL, which are specified in the build command line:
<syntaxhighlight lang='bash'>
docker build ... <build-context>
docker build ... .
</syntaxhighlight>
 
The files specified in the [[Dockerfile]] instructions are relative to the context.


The URL may refer to Git repositories, pre-package TAR files and plain text files.
The URL may refer to Git repositories, pre-package TAR files and plain text files.
Line 52: Line 53:
{{External|https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg}}
{{External|https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg}}
{{External|https://docs.docker.com/engine/reference/builder/#arg}}
{{External|https://docs.docker.com/engine/reference/builder/#arg}}
{{External|https://docs.docker.com/engine/reference/builder/#environment-replacement}}
Docker allows build-time variables, that can be initialized in the <code>docker build</code> command line and used in the [[Dockerfile|Dockerfile]]. This mechanism is useful for specifying environment specific configuration elements, such as IP addresses, for example. Built-time variables should not be used to pass sensitive information, such as password, since the values can be retrieved with [[docker history]] command.
Build-time variables are introduced with the Dockerfile [[Dockerfile#ARG|ARG]] instruction.


Docker allows build-time variables, that can be initialized on <tt>docker build</tt> command line and used in the Dockerfile. This mechanism is useful for specifying environment specific configuration element (such as IP addresses, for example), and values that should not be maintained in a Dockerfile, such as password.
If there is more than one ARG, it should be listed on a separate line:


<font size=-2>
ARG DISTRIBUTION_ZIP
ARG DISTRIBUTION_VERSION
</font>
Using build-time variables is a two-step process:
Using build-time variables is a two-step process:


1. The variables should be declared in Dockerfile with [[Dockerfile#ARG|ARG]] as follows:
1. The variables should be declared in Dockerfile with [[Dockerfile#ARG|ARG]] as follows:
<font size=-2>
ARG <name>[=<default value>]
</font>
Example:
<syntaxhighlight lang='bash'>
ARG VAR1
ARG VAR2='something'
</syntaxhighlight>


ARG=<name>[=<default value>]
The ARG instruction defines a variable that can be initialized at runtime, in the <code>docker build</code> command line, as shown below. One or more build-time variables may be defined, one per line. If user specifies a build argument that was not defined in the Dockerfile, the build outputs a warning.


The ARG instruction defines a variable that can be initialized at runtime, in the <tt>docker build</tt> command line, as shown below. One or more build-time variables may be defined, one per line. If user specifies a build argument that was not defined in the Dockerfile, the build outputs a warning.
Once the variable has been defined with an <code>ARG</code> instruction, it can be used in other <code>Dockerfile</code> instructions, like for example in:
<syntaxhighlight lang='bash'>
ARG COLOR
RUN echo "${COLOR}" > /tmp/color
</syntaxhighlight>
If "COLOR" is not declared with <code>ARG</code>, it will not be resolved when referred with <code>${COLOR}</code>.


{{Warn| It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc. Build-time variable values are visible to any user of the image with the [[docker history]] command.}}
{{Warn| It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc. Build-time variable values are visible to any user of the image with the [[docker history]] command.}}


2. The variables should be initialized on <tt>docker build</tt> command line as follows:
2. The variables should be initialized on <code>docker build</code> command line as follows:
 
<syntaxhighlight lang='bash'>
docker build ... --build-arg VAR1=value1 --build-arg VAR2=value2 ...
docker build ... --build-arg COLOR=blue --build-arg VAR1=value1 --build-arg VAR2=value2 ...
</syntaxhighlight>


=Options=
=Options=
Line 101: Line 125:
==--no-cache==
==--no-cache==


Do not use the cache while building the image. See more here: [[#The_Build_Process|The Build Process]].
<tt>--no-cache</tt> instructs the build process to not use the cache while building the imageProcess]].


==--build-arg==
<syntaxhighlight lang='bash'>
docker build --no-cache=true ...
</syntaxhighlight>


See [[Docker_build#Build-Time_Variables|Built-Time Variables]] above.
<tt>--no-cache=true</tt> and <tt>--no-cache</tt> are equivalent.


=Multi-Stage Build=
For more details on how build cache works see: {{Internal|Docker Build Cache|Docker Build Cache}}


{{External|https://docs.docker.com/engine/userguide/eng-image/multistage-build/}}
==--build-arg==


{{External|https://blog.alexellis.io/mutli-stage-docker-builds/}}
See [[Docker_build#Build-Time_Variables|Built-Time Variables]] above.


A more efficient replacement for the [[Docker Concepts#Builder_Pattern|builder pattern]].
==--pull==


The general syntax involves adding FROM additional times within the Dockerfile and naming build stages. Whichever is the last FROM statement is the final base image. To copy artifacts and outputs from intermediate images use COPY --from=<base_image_name>:
Force pulling the base image even if it locally present.


<syntaxhighlight lang='Docker'>
=Multi-Stage Build=
FROM something AS my_builld
{{Internal|Docker_Concepts#Multi-Stage_Build|Docker Concepts &#124; Multi-Stage Build}}
 
# This results in a single layer image
FROM alpine:latest 
COPY --from=my_builld  ...
</syntaxhighlight>


=Image Recipes=
=Image Recipes=


* [[Building a Container that Loops]]
* [[Building a Container that Loops]]

Latest revision as of 20:45, 23 September 2024

External

Internal

Overview

docker build expects a Dockerfile and a context and uses the information provided to produce a Docker image.

docker build [options] <path>|<url>|-

The default location for the Dockerfile is the current directory, and the path is "."

The image such produced is placed in the local registry, using the repository name and the image tag provided with -t command line option. Note that if no -t option is use, the image will still be built, but it will be "dangling". For more details see -t command line option.

The Build Process

Starting with Docker 1.10, only RUN, COPY and ADD create layers.

Dockerfile is read and the instructions listed in it are evaluated in order. If the build cache is used, which is the default unless the --no-cache=true option is specified as argument of the build command, the instructions are also looked up against the cache content and if a match is found, the instruction is not executed, but the cached layer is used instead. This usually speeds up builds, but some precautions need to be taken.

For details, see:

Docker Build Cache

If a cached image was used, the build command output states that:

Step 2/5 : COPY ./loop /opt/loop
 ---> Using cache
 ---> 171da11cf0ef

The cache can be explicitly invalidated with the --no-cache command line option is used.

The Build Context

The build context is a set of files located in a path, or at an URL, which are specified in the build command line:

docker build ... <build-context>
docker build ... .

The files specified in the Dockerfile instructions are relative to the context.

The URL may refer to Git repositories, pre-package TAR files and plain text files.

.dockerignore

.dockerignore

Build-Time Variables

https://docs.docker.com/engine/reference/commandline/build/#set-build-time-variables---build-arg
https://docs.docker.com/engine/reference/builder/#arg
https://docs.docker.com/engine/reference/builder/#environment-replacement

Docker allows build-time variables, that can be initialized in the docker build command line and used in the Dockerfile. This mechanism is useful for specifying environment specific configuration elements, such as IP addresses, for example. Built-time variables should not be used to pass sensitive information, such as password, since the values can be retrieved with docker history command.

Build-time variables are introduced with the Dockerfile ARG instruction.

If there is more than one ARG, it should be listed on a separate line:

ARG DISTRIBUTION_ZIP
ARG DISTRIBUTION_VERSION

Using build-time variables is a two-step process:

1. The variables should be declared in Dockerfile with ARG as follows:

ARG <name>[=<default value>]

Example:

ARG VAR1
ARG VAR2='something'

The ARG instruction defines a variable that can be initialized at runtime, in the docker build command line, as shown below. One or more build-time variables may be defined, one per line. If user specifies a build argument that was not defined in the Dockerfile, the build outputs a warning.

Once the variable has been defined with an ARG instruction, it can be used in other Dockerfile instructions, like for example in:

ARG COLOR
RUN echo "${COLOR}" > /tmp/color

If "COLOR" is not declared with ARG, it will not be resolved when referred with ${COLOR}.


It is not recommended to use build-time variables for passing secrets like github keys, user credentials etc. Build-time variable values are visible to any user of the image with the docker history command.

2. The variables should be initialized on docker build command line as follows:

docker build ... --build-arg COLOR=blue --build-arg VAR1=value1 --build-arg VAR2=value2 ...

Options

-t, --tag

Even if the option is named "tag", it actually specifies the target image repository URL, ignoring the registry host name - the image is always placed in the local registry. In one of its simplest variants, the URL can be just a tag, indeed. Most commonly, the URL contains the namespace and the repository name, or just the repository name. For more details about a repository URL, see: URL.

docker build -t novaordis/centos-loop:latest .

If no name/tag information is provided, there is no default: the image may be stored with no repository and no tag, just with an image ID. An "unnamed" images is called "dangling":


REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
<none>              <none>              4cfda3233bb6        5 seconds ago       204MB

Alternatively, if the only Dockerfile instruction is FROM, or ..., the name and the tag will be inferred from the base image.

If the name/tag combination already exists in the repository, the image they designate will be "unnamed" (dangled) and the new image that has just been built replaces it.

--rm

--rm=true

Remove intermediate containers after a successful build (default true).

-f

Instructs the build process to use a Dockerfile other than the ./Dockerfile.

docker build -f ./my.Dockerfile -t something .

--no-cache

--no-cache instructs the build process to not use the cache while building the imageProcess]].

docker build --no-cache=true ...

--no-cache=true and --no-cache are equivalent.

For more details on how build cache works see:

Docker Build Cache

--build-arg

See Built-Time Variables above.

--pull

Force pulling the base image even if it locally present.

Multi-Stage Build

Docker Concepts | Multi-Stage Build

Image Recipes