Docker Storage Concepts
External
- https://docs.docker.com/engine/userguide/storagedriver/selectadriver
- https://docs.docker.com/engine/userguide/storagedriver/
Internal
Image Storage
The Docker server stores the layers that form images into a dedicated storage system, called the storage backend, managed by a pluggable storage driver.
Storage Driver
The Docker storage driver handles details related to how various layers, including the container layer interact with each other and how the container image is exposed. Containers and the images they are created from are stored in Docker’s storage backend. The Docker server's storage backend communicates with the underlying Linux filesystem to build and manage the multiple image layers that combine to form a single image.
Some storage concepts, such as base device size, which essentially represents the container's root file system size, only apply to specific storage backend, device-mapper in this case, and they will be mentioned in the corresponding sections.
To determine what kind of storage backend is in use by a specific server instance, execute docker info. Storage backend details are provided as part of the result.
Where are images stored (docker images), and where are the running and stopped containers stored (docker ps)?
Available storage drivers:
devicemapper Storage Driver
overlayfs Storage Driver
This is the default storage driver a RHEL installation will default to. Ubuntu installations also prefer it, if they have 4.x kernels.
AUFS
BTRFS
Copy-on-Write (CoW) Strategy
All storage backend drivers provides a fast CoW (copy-on-write) system for image management. Copy-on-write is a strategy of sharing and copying files for maximum efficiency. If a file or a directory exists in a lower layer of the image, and another layer - including the writable layer - needs read access to it, it just uses the existing file from its original layer. If the file needs to be modified - either at build time, when the container is being built, or at run time, when the container is instantiated - the file is copied into the layer that needs the file, and modified.
For an overlay/overlay2 or AUFS drivers, the copy-on-write operation consists of:
- Search through the image layers for the file to update. The process starts at the newest layer and works down to the base layer, one layer at a time. When a result is found, it is added to a cache.
- Perform a copy_up operation on the first copy of the file that is found into the writable layer.
- Any modifications are made to this copy of the file, and the container cannot see the read-only copy of the file that exists in the lower layer.
A copy_up may incur a significant performance overhead, which depends on the storage driver in use. Large files, many layers and deep directory tree can make the impact more noticeable. However, the copy_up operation only occurs the first time a file is modified.