Docker Storage Operations: Difference between revisions
(39 intermediate revisions by the same user not shown) | |||
Line 1: | Line 1: | ||
=External= | |||
* https://docs.docker.com/engine/reference/commandline/run/#add-bind-mounts-or-volumes-using-the---mount-flag | |||
=Internal= | =Internal= | ||
Line 9: | Line 13: | ||
=Container-Generated Data Storage Operations= | =Container-Generated Data Storage Operations= | ||
==Listing Existing Volumes== | |||
docker volume ls | |||
The output returns the driver and volume name for each volume <font color=darkgray>in use</font>. | |||
==Details about a Specific Volume== | |||
If you have the volume name (or obtain the volume name with <tt>docker volume ls</tt>): | |||
docker volume inspect <''volume-name''> | |||
==Mounting a Volume== | |||
If the container has volume mount points specified with [[Dockerfile#VOLUME|VOLUME]] in its original Dockerfile, those mount points must be bound to paths on the native host when the container is created. This is done with [[Docker_run#-v.2C_--volume|--v|--volume]] or [[Docker_run#--mount|--mount]] (recommended) command line options, as follows. Note that the mount succeeds even if no [[Dockerfile#VOLUME|VOLUME]] is declared in Dockerfile. | |||
===--mount=== | |||
The --mount flag allows mounting data volumes, host directories and tmpfs mounts in a container. --mount supports most of the options supported by [[Docker_Storage_Operations#-v.7C--volume|-v|--volume]], with some exceptions: | |||
* --mount allows specifying a volume driver and volume driver options per volume, without creating the volume in advance. In contrast, <tt>docker run --volume</tt> allows specifying a single volume driver, which is shared by all volumes, using the --volume-driver flag. | |||
* --mount allows specifying custom metadata ("labels") for a volume, before the volume is created. | |||
* --mount does not allow relabeling a volume with Z or z flags, which are used for selinux labeling. | |||
docker run --mount '''type=volume''','''source=<''native-host-path''>''',destination=<''container-mount-point''>,volume-label="<''some-label''>" ... | |||
docker run ... --mount type=volume,source=/data-volumes/postgresql,destination=var/lib/pgsql/data,volume-label="postgres" ... | |||
Also, a <span id='Named_Volume'></span>[[Docker_Storage_Concepts#Named_Volume|named volume]] can be used: | |||
docker run --mount '''type=volume''','''source=<''volume-name''>''',destination=<''container-mount-point''>,volume-label="<''some-label''>" ... | |||
docker run ... --mount type=volume,source=external-storage,destination=var/lib/pgsql/data,volume-label="postgres" ... | |||
<span id='Anonymous_Volume'></span>[[Docker_Storage_Concepts#Anonymous_Volume|Anonymous volumes]] can be created, and mounted with (note there is no "source"): | |||
docker run --mount '''type=volume''',destination=<''container-mount-point''> ... | |||
Bind mounts are created with: | |||
docker run ... --mount '''type=bind''','''src=<''native-host-path''>''',dst=<''container-mount-point''> ... | |||
docker run ... --mount type=bind,src=/data-volumes/postgresql,dst=/var/lib/pgsql/data ... | docker run ... --mount type=bind,src=/data-volumes/postgresql,dst=/var/lib/pgsql/data ... | ||
When --mount with type=bind is used, the native-host-path must refer to an existing path on the host. The path will not be created if it does not exist, and the command will fail: | |||
docker run --rm -it --mount type=bind,src=/some/path/that/does/not/exist,dst=/blah test-image | |||
docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist. | |||
Also, the mount point must have sufficient permissions. For more details on native host path permissions, see [[Docker_Storage_Concepts#UID.2FGID_Mapping|Native Host Path Permissions]]. | |||
===-v|--volume=== | |||
docker run ... -v <''native-host-path''>:<''container-mount-point''> ... | docker run ... -v <''native-host-path''>:<''container-mount-point''> ... | ||
docker run ... -v $(pwd)/build:/build oa2aws:latest ... /build/swagger.json | |||
==Creating a Volume== | |||
{{External|https://docs.docker.com/engine/reference/commandline/volume_create/}} | |||
The command creates a new volume that containers can consume and store data in. If a name is not specified, Docker generates a random name. Volumes thus created survive a Docker server reboot, meaning they will be re-mounted on reboot. | |||
docker volume create [''volume-name''] | |||
===Creating a NFS Volume=== | |||
docker volume create \ | |||
--driver local \ | |||
--opt type=nfs \ | |||
--opt o=addr=192.168.1.1,rw \ | |||
--opt device=:<''path-on-the-remote-nfs-server''> \ | |||
<''volume-name''> | |||
docker volume create \ | |||
--driver local \ | |||
--opt type=nfs \ | |||
--opt o=addr=192.168.1.40,rw \ | |||
--opt device=:/volume1/dovecot \ | |||
rackstation-dovecot | |||
The volume is mounted in /var/lib/docker/volumes/''volume-name''/_data. | |||
Errors of type: | |||
<syntaxhighlight lang='bash'> | |||
docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/nfstest/_data': error while mounting volume with options: type='nfs' device=':/volume1/nfstest' o='addr=192.168.1.4,rw': connection refused.</syntaxhighlight> | |||
were fixed in the past by [[Linux_NFS_Installation#Install_Client_Packages|installing the client library on the docker host]]. | |||
====NFS Volume Troubles==== | |||
{{Warn|Attempting to use an NFS volume mounted by the Docker daemon with a "docker volume create" command similar to the one shown above leads to errors:}} | |||
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): fcntl() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available | |||
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): mail_index_wait_lock_fd() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available | |||
The errors do to show up if the NFS storage is mounted at the Docker host level and [[#-v.7C--volume|-v option]] is used with the container. | |||
==Verifying Whether a Volume Exists== | |||
<syntaxhighlight lang='bash'> | |||
if docker volume inspect <volume-name> >/dev/null 2>&1; then | |||
echo "volume exists"; | |||
else | |||
echo "volume does not exist"; | |||
fi | |||
</syntaxhighlight> | |||
==Getting Information on a Volume== | |||
docker volume ls [''volume-name''] | |||
docker volume inspect <''volume-name''> | |||
==Remove a Volume== | |||
docker volume rm | |||
==Remove Unused Local Volumes== | |||
docker volume prune |
Latest revision as of 10:25, 20 March 2019
External
Internal
Overview
Container-Generated Data Storage Operations
Listing Existing Volumes
docker volume ls
The output returns the driver and volume name for each volume in use.
Details about a Specific Volume
If you have the volume name (or obtain the volume name with docker volume ls):
docker volume inspect <volume-name>
Mounting a Volume
If the container has volume mount points specified with VOLUME in its original Dockerfile, those mount points must be bound to paths on the native host when the container is created. This is done with --v|--volume or --mount (recommended) command line options, as follows. Note that the mount succeeds even if no VOLUME is declared in Dockerfile.
--mount
The --mount flag allows mounting data volumes, host directories and tmpfs mounts in a container. --mount supports most of the options supported by -v|--volume, with some exceptions:
- --mount allows specifying a volume driver and volume driver options per volume, without creating the volume in advance. In contrast, docker run --volume allows specifying a single volume driver, which is shared by all volumes, using the --volume-driver flag.
- --mount allows specifying custom metadata ("labels") for a volume, before the volume is created.
- --mount does not allow relabeling a volume with Z or z flags, which are used for selinux labeling.
docker run --mount type=volume,source=<native-host-path>,destination=<container-mount-point>,volume-label="<some-label>" ... docker run ... --mount type=volume,source=/data-volumes/postgresql,destination=var/lib/pgsql/data,volume-label="postgres" ...
Also, a named volume can be used:
docker run --mount type=volume,source=<volume-name>,destination=<container-mount-point>,volume-label="<some-label>" ... docker run ... --mount type=volume,source=external-storage,destination=var/lib/pgsql/data,volume-label="postgres" ...
Anonymous volumes can be created, and mounted with (note there is no "source"):
docker run --mount type=volume,destination=<container-mount-point> ...
Bind mounts are created with:
docker run ... --mount type=bind,src=<native-host-path>,dst=<container-mount-point> ... docker run ... --mount type=bind,src=/data-volumes/postgresql,dst=/var/lib/pgsql/data ...
When --mount with type=bind is used, the native-host-path must refer to an existing path on the host. The path will not be created if it does not exist, and the command will fail:
docker run --rm -it --mount type=bind,src=/some/path/that/does/not/exist,dst=/blah test-image docker: Error response from daemon: invalid mount config for type "bind": bind source path does not exist.
Also, the mount point must have sufficient permissions. For more details on native host path permissions, see Native Host Path Permissions.
-v|--volume
docker run ... -v <native-host-path>:<container-mount-point> ...
docker run ... -v $(pwd)/build:/build oa2aws:latest ... /build/swagger.json
Creating a Volume
The command creates a new volume that containers can consume and store data in. If a name is not specified, Docker generates a random name. Volumes thus created survive a Docker server reboot, meaning they will be re-mounted on reboot.
docker volume create [volume-name]
Creating a NFS Volume
docker volume create \ --driver local \ --opt type=nfs \ --opt o=addr=192.168.1.1,rw \ --opt device=:<path-on-the-remote-nfs-server> \ <volume-name>
docker volume create \ --driver local \ --opt type=nfs \ --opt o=addr=192.168.1.40,rw \ --opt device=:/volume1/dovecot \ rackstation-dovecot
The volume is mounted in /var/lib/docker/volumes/volume-name/_data.
Errors of type:
docker: Error response from daemon: error while mounting volume '/var/lib/docker/volumes/nfstest/_data': error while mounting volume with options: type='nfs' device=':/volume1/nfstest' o='addr=192.168.1.4,rw': connection refused.
were fixed in the past by installing the client library on the docker host.
NFS Volume Troubles
Attempting to use an NFS volume mounted by the Docker daemon with a "docker volume create" command similar to the one shown above leads to errors:
dovecot: May 21 03:16:12 Error: IMAP(ovidiu): fcntl() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available dovecot: May 21 03:16:12 Error: IMAP(ovidiu): mail_index_wait_lock_fd() failed with file /opt/dovecot/external/imap-data/ovidiu/Maildir/dovecot.index.log: No locks available
The errors do to show up if the NFS storage is mounted at the Docker host level and -v option is used with the container.
Verifying Whether a Volume Exists
if docker volume inspect <volume-name> >/dev/null 2>&1; then
echo "volume exists";
else
echo "volume does not exist";
fi
Getting Information on a Volume
docker volume ls [volume-name]
docker volume inspect <volume-name>
Remove a Volume
docker volume rm
Remove Unused Local Volumes
docker volume prune