Docker device-mapper Storage Backend
External
- https://docs.docker.com/storage/storagedriver/
- https://docs.docker.com/storage/storagedriver/device-mapper-driver/
- https://docs.docker.com/engine/reference/commandline/dockerd/#options-per-storage-driver
Internal
Overview
The Docker device-mapper Storage Backend is built upon the device mapper framework. The devicemapper driver expects access to a dedicated block device - meaning that only the Docker server process should have exclusive access to it. The driver will operate at block level, rather than file level. Because of that, this type of storage usually performs better than using a filesystem accessed via the operating system. Once configured with a devicemapper driver, the docker server cannot be reconfigured with another driver unless the containers already stored within it are discarded (or saved with docker save and re-imported).
devicemapper Mode
The devicemapper driver can be configured to run in one of the loop-lvm or direct-lvm modes.
loop-lvm mode uses a loopback device for storage, but this is a configuration that is only appropriate for testing. Loopback devices are slow, resource-intensive, and they can introduce race conditions. The only upside is that the setup is easier, so they may be used for testing. For the procedure of configuring loop-lvm mode, see https://docs.docker.com/storage/storagedriver/device-mapper-driver/#configure-loop-lvm-mode-for-testing.
direct-lvm uses an actual uses block device to create the thin pool. This mode is faster than using loopback devices, uses system resources more efficiently, and block devices can grow as needed.
Thin Pool
Info
At runtime, general statistics on the devicemapper storage driver can be obtained with:
docker info Storage Driver: devicemapper Pool Name: docker_vg-container--thinpool Pool Blocksize: 524.3 kB Base Device Size: 10.74 GB Backing Filesystem: xfs Data file: Metadata file: Data Space Used: 16.81 GB Data Space Total: 53.57 GB Data Space Available: 36.76 GB Metadata Space Used: 3.338 MB Metadata Space Total: 54.53 MB Metadata Space Available: 51.19 MB Thin Pool Minimum Free Space: 5.357 GB Udev Sync Supported: true Deferred Removal Enabled: true Deferred Deletion Enabled: true Deferred Deleted Device Count: 0 Library Version: 1.02.140-RHEL7 (2017-05-03) ...
Installation
Devicemapper direct-lvm Installation
This procedure is part of the Docker installation procedure.
Production systems using the devicemapper storage driver must use direct-lvm mode, which uses block devices to create a thin pool.
If this is an initial Docker installation, and just one block device will be used for storage, Docker can be configured to initialize and manage the single block device, without additional manual configuration.
The block device to be used by the storage backend must be configured externally. Examples:
After the device is configured and attached to the guest, add the following to daemon.json:
{
"storage-driver": "devicemapper",
"storage-opts": [
"dm.directlvm_device=/dev/vdb",
"dm.thinp_percent=95",
"dm.thinp_metapercent=1",
"dm.thinp_autoextend_threshold=80",
"dm.thinp_autoextend_percent=20",
"dm.directlvm_device_force=false"
]
}
Then restart docker:
systemctl stop docker systemctl start docker
The initialization results can be verified with docker info. The result should be similar with the output shown above.
The newly created thin pool device can be inspected with pvs/vgs/lvs:
pvs
PV VG Fmt Attr PSize PFree
/dev/vdb docker lvm2 a-- 46.56g <1.40g
vgs
VG #PV #LV #SN Attr VSize VFree
docker 1 1 0 wz--n- 46.56g <1.40g
lvs
LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert
thinpool docker twi-a-t--- 44.23g 0.04 0.02
Base Device Size
Base size is the maximum size that a image, and containers based on it, can grow to. Essentially, the size of the root filesystem of the container.
View Base Device Size
The docker engine-wide base image size is reported with docker info (see above). The container-specific base image size is reported with docker inspect {{.GraphDriver.Data.DeviceSize}}:
...
"GraphDriver": {
"Name": "devicemapper",
"Data": {
"DeviceId": "3315",
"DeviceName": "docker-253:3-12968682-7827556d545926857d669f5f9bb7fd69083e7e98636ff3f060d352223f4918ab",
"DeviceSize": "10737418240"
}
}
...
At runtime, the same amount can be viewed by attaching to the container and execute:
df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/docker-253:3-12968682-.... 10G 658M 9.4G 7% /
Modify Base Device Size
The docker engine instance wide value set with:
--storage-opt dm.basesize=20G
on docker engine's command line.
For a specific container, it can be set with:
docker run --storage-opt size=60
Apparently it can only be expanded, not shrunk - needs experimentation. Existing images will retain the old base device size, unless they are rmi/pull.
TODO
Device-mapper driver metadata is stored in /var/lib/docker/devicemapper/metadata. Docker state can be reset by stopping docker and by removing /var/lib/docker/devicemapper/metadata directory. This will destroy existing docker images and containers and all the docker metadata.
The thin pool.
Base device.
/var/lib/docker/devicemapper/metadata/base
{"device_id":1,"size":10737418240,"transaction_id":1,"initialized":true,"deleted":false}
What is metadata in this context? The data that is maintained in /var/lib/docker/devicemapper/metadata?
The concept of a snapshot of an image.
The concept of a devicemapper snapshot.
There seems to be a base device for each running container.
How are images and container stored.
A container in this context is a writable layer.
Layer.
Thin pool block and thin pool block size.