Docker Storage Operations: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(27 intermediate revisions by the same user not shown)
Line 12: Line 12:


=Container-Generated Data Storage Operations=
=Container-Generated Data Storage Operations=
==Listing Existing Volumes==
  docker volume ls
The output returns the driver and volume name for each volume <font color=darkgray>in use</font>.
==Details about a Specific Volume==
If you have the volume name (or obtain the volume name with <tt>docker volume ls</tt>):
docker volume inspect <''volume-name''>
==Mounting a Volume==


If the container has volume mount points specified with [[Dockerfile#VOLUME|VOLUME]] in its original Dockerfile, those mount points must be bound to paths on the native host when the container is created. This is done with [[Docker_run#-v.2C_--volume|--v|--volume]] or [[Docker_run#--mount|--mount]] (recommended) command line options, as follows. Note that the mount succeeds even if no [[Dockerfile#VOLUME|VOLUME]] is declared in Dockerfile.
If the container has volume mount points specified with [[Dockerfile#VOLUME|VOLUME]] in its original Dockerfile, those mount points must be bound to paths on the native host when the container is created. This is done with [[Docker_run#-v.2C_--volume|--v|--volume]] or [[Docker_run#--mount|--mount]] (recommended) command line options, as follows. Note that the mount succeeds even if no [[Dockerfile#VOLUME|VOLUME]] is declared in Dockerfile.


==--mount==
===--mount===


The --mount flag allows mounting data volumes, host directories and tmpfs mounts in a container. --mount supports most of the options supported by [[Docker_Storage_Operations#-v.7C--volume|-v|--volume]], with some exceptions:
The --mount flag allows mounting data volumes, host directories and tmpfs mounts in a container. --mount supports most of the options supported by [[Docker_Storage_Operations#-v.7C--volume|-v|--volume]], with some exceptions:
Line 25: Line 39:
  docker run ... --mount type=volume,source=/data-volumes/postgresql,destination=var/lib/pgsql/data,volume-label="postgres" ...
  docker run ... --mount type=volume,source=/data-volumes/postgresql,destination=var/lib/pgsql/data,volume-label="postgres" ...


Also, a named volume can be used:
Also, a <span id='Named_Volume'></span>[[Docker_Storage_Concepts#Named_Volume|named volume]] can be used:


  docker run --mount '''type=volume''','''source=<''volume-name''>''',destination=<''container-mount-point''>,volume-label="<''some-label''>" ...
  docker run --mount '''type=volume''','''source=<''volume-name''>''',destination=<''container-mount-point''>,volume-label="<''some-label''>" ...
  docker run ... --mount type=volume,source=external-storage,destination=var/lib/pgsql/data,volume-label="postgres" ...
  docker run ... --mount type=volume,source=external-storage,destination=var/lib/pgsql/data,volume-label="postgres" ...
<span id='Anonymous_Volume'></span>[[Docker_Storage_Concepts#Anonymous_Volume|Anonymous volumes]] can be created, and mounted with (note there is no "source"):
docker run --mount '''type=volume''',destination=<''container-mount-point''> ...
Bind mounts are created with:


  docker run ... --mount '''type=bind''','''src=<''native-host-path''>''',dst=<''container-mount-point''> ...
  docker run ... --mount '''type=bind''','''src=<''native-host-path''>''',dst=<''container-mount-point''> ...
Line 40: Line 60:
Also, the mount point must have sufficient permissions. For more details on native host path permissions, see [[Docker_Storage_Concepts#UID.2FGID_Mapping|Native Host Path Permissions]].
Also, the mount point must have sufficient permissions. For more details on native host path permissions, see [[Docker_Storage_Concepts#UID.2FGID_Mapping|Native Host Path Permissions]].


==-v|--volume==
===-v|--volume===


  docker run ... -v <''native-host-path''>:<''container-mount-point''> ...
  docker run ... -v <''native-host-path''>:<''container-mount-point''> ...
docker run ... -v $(pwd)/build:/build oa2aws:latest ... /build/swagger.json
==Creating a Volume==
{{External|https://docs.docker.com/engine/reference/commandline/volume_create/}}
The command creates a new volume that containers can consume and store data in. If a name is not specified, Docker generates a random name. Volumes thus created survive a Docker server reboot, meaning they will be re-mounted on reboot.
docker volume create [''volume-name'']
===Creating a NFS Volume===
docker volume create \
  --driver local \
  --opt type=nfs \
  --opt o=addr=192.168.1.1,rw \
  --opt device=:<''path-on-the-remote-nfs-server''> \
  <''volume-name''>
docker volume create \
    --driver local \
    --opt type=nfs \
    --opt o=addr=192.168.1.40,rw \
    --opt device=:/volume1/dovecot \
  rackstation-dovecot
The volume is mounted in /var/lib/docker/volumes/''volume-name''/_data.
Errors of type:
<syntaxhighlight lang='bash'>
docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/nfstest/_data': error while mounting volume with options: type='nfs' device=':/volume1/nfstest' o='addr=192.168.1.4,rw': connection refused.</syntaxhighlight>
were fixed in the past by [[Linux_NFS_Installation#Install_Client_Packages|installing the client library on the docker host]].
====NFS Volume Troubles====
{{Warn|Attempting to use an NFS volume mounted by the Docker daemon with a "docker volume create" command similar to the one shown above leads to errors:}}
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): fcntl() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): mail_index_wait_lock_fd() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available
The errors do to show up if the NFS storage is mounted at the Docker host level and [[#-v.7C--volume|-v option]] is used with the container.
==Verifying Whether a Volume Exists==
<syntaxhighlight lang='bash'>
if docker volume inspect <volume-name> >/dev/null 2>&1; then
  echo "volume exists";
else
  echo "volume does not exist";
fi
</syntaxhighlight>
==Getting Information on a Volume==
docker volume ls [''volume-name'']
docker volume inspect <''volume-name''>
==Remove a Volume==
docker volume rm
==Remove Unused Local Volumes==
docker volume prune

Latest revision as of 10:25, 20 March 2019

External

Internal

Overview

Container-Generated Data Storage Operations

Listing Existing Volumes

 docker volume ls

The output returns the driver and volume name for each volume in use.

Details about a Specific Volume

If you have the volume name (or obtain the volume name with docker volume ls):

docker volume inspect <volume-name>

Mounting a Volume

If the container has volume mount points specified with VOLUME in its original Dockerfile, those mount points must be bound to paths on the native host when the container is created. This is done with --v|--volume or --mount (recommended) command line options, as follows. Note that the mount succeeds even if no VOLUME is declared in Dockerfile.

--mount

The --mount flag allows mounting data volumes, host directories and tmpfs mounts in a container. --mount supports most of the options supported by -v|--volume, with some exceptions:

  • --mount allows specifying a volume driver and volume driver options per volume, without creating the volume in advance. In contrast, docker run --volume allows specifying a single volume driver, which is shared by all volumes, using the --volume-driver flag.
  • --mount allows specifying custom metadata ("labels") for a volume, before the volume is created.
  • --mount does not allow relabeling a volume with Z or z flags, which are used for selinux labeling.
docker run --mount type=volume,source=<native-host-path>,destination=<container-mount-point>,volume-label="<some-label>" ...
docker run ... --mount type=volume,source=/data-volumes/postgresql,destination=var/lib/pgsql/data,volume-label="postgres" ...

Also, a named volume can be used:

docker run --mount type=volume,source=<volume-name>,destination=<container-mount-point>,volume-label="<some-label>" ...
docker run ... --mount type=volume,source=external-storage,destination=var/lib/pgsql/data,volume-label="postgres" ...

Anonymous volumes can be created, and mounted with (note there is no "source"):

docker run --mount type=volume,destination=<container-mount-point> ...

Bind mounts are created with:

docker run ... --mount type=bind,src=<native-host-path>,dst=<container-mount-point> ...
docker run ... --mount type=bind,src=/data-volumes/postgresql,dst=/var/lib/pgsql/data ...

When --mount with type=bind is used, the native-host-path must refer to an existing path on the host. The path will not be created if it does not exist, and the command will fail:

docker run --rm -it --mount type=bind,src=/some/path/that/does/not/exist,dst=/blah test-image
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist.

Also, the mount point must have sufficient permissions. For more details on native host path permissions, see Native Host Path Permissions.

-v|--volume

docker run ... -v <native-host-path>:<container-mount-point> ...
docker run ... -v $(pwd)/build:/build oa2aws:latest ... /build/swagger.json

Creating a Volume

https://docs.docker.com/engine/reference/commandline/volume_create/

The command creates a new volume that containers can consume and store data in. If a name is not specified, Docker generates a random name. Volumes thus created survive a Docker server reboot, meaning they will be re-mounted on reboot.

docker volume create [volume-name]

Creating a NFS Volume

docker volume create \
  --driver local \
  --opt type=nfs \
  --opt o=addr=192.168.1.1,rw \
  --opt device=:<path-on-the-remote-nfs-server> \
  <volume-name>
docker volume create \
   --driver local \
   --opt type=nfs \
   --opt o=addr=192.168.1.40,rw \
   --opt device=:/volume1/dovecot \
 rackstation-dovecot

The volume is mounted in /var/lib/docker/volumes/volume-name/_data.

Errors of type:

docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/nfstest/_data': error while mounting volume with options: type='nfs' device=':/volume1/nfstest' o='addr=192.168.1.4,rw': connection refused.

were fixed in the past by installing the client library on the docker host.

NFS Volume Troubles


Attempting to use an NFS volume mounted by the Docker daemon with a "docker volume create" command similar to the one shown above leads to errors:

dovecot: May 21 03:16:12 Error: IMAP(ovidiu): fcntl() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): mail_index_wait_lock_fd() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available

The errors do to show up if the NFS storage is mounted at the Docker host level and -v option is used with the container.

Verifying Whether a Volume Exists

if docker volume inspect <volume-name> >/dev/null 2>&1; then 
  echo "volume exists"; 
else 
  echo "volume does not exist"; 
fi

Getting Information on a Volume

docker volume ls [volume-name]
docker volume inspect <volume-name>

Remove a Volume

docker volume rm

Remove Unused Local Volumes

docker volume prune