Kubernetes Storage Concepts
Internal
Overview
Kubernetes has a mature and feature-rich subsystem called the persistent volume subsystem. Regardless of where it comes from, storage is exposed to the Kubernetes cluster in the form of a volume.
Storage Providers
Storage is exposed to a Kubernetes cluster by storage providers.
The Kubernetes persistent volume subsystem supports, among others:
- iSCSI volumes
- SMB
- Enterprise storage arrays from vendors like EMC and NetApp
- NFS volumes
- object storage blobs
- Amazon Elastic Block Store block devices
- Azure File resources (AzureDisk).
- GCE Persistent Disks
Each storage provider has its own plugin that handles the details of exposing the storage to the Kubernetes cluster.
Storage Plugins
The terms "storage plugin" and "provisioner" are used interchangeably. "Provisioner" is used especially when dynamic provisioning is involved. "Driver" is a term that is also used here.
Old storage plugins used to be implemented as part of the main Kubernetes code tree (in-tree), which raised a series of problems, such as that all had to be open-source and their release cycle was tied to the Kubernetes release cycle.
Newer plugins are based on the Container Storage Interface (CSI) and can be developed out-of-tree.
Plugin Types
kuberentes.io/aws-ebs
Container Storage Interface (CSI)
Container Storage Interface (CSI) is an open standard that provides a clean interface for storage plugins and abstracts the internal Kubernetes storage details. CSI provides means so the external storage can be leveraged in a uniform way across multiple container orchestrators (not only Kubernetes).
API Resources
The persistent volume subsystem consists of the following three API resource types that allow applications to consume storage: persistent volumes, persistent volume claims and storage classes:
Persistent Volume (PV)
A single external storage volume can only be used by a single persistent volume.
Persistent Volume Manifest
Access Mode
A persistent volume can be bound in only one mode - it is not possible for a persistent volume to have a persistent volume claim bound to it in ROM mode and another persistent volume claim bound in RWM mode.
ReadWriteOnce (RWO)
This mode defines a persistent volume that can only be bound in read/write mode by a single persistent volume claim. An attempt to bind it via multiple persistent volume claims will fail. In general, block storage normally only supports RWO.
ReadWriteMany (RWM)
This mode defines a persistent volume that can be bound in read/write mode by multiple persistent volume claims. In general, file storage and object storage support RWM.
ReadOnlyMany (ROM)
This mode defines a persistent volume that can be bound in read only mode by multiple persistent volume claims.
Persistent Volume Claim (PVC)
A persistent volume claim is similar to a ticket that authorizes a pod to use a certain persistent volume. Once an application has a persistent volume claim, it can mount the respective volume into its pod.
Storage Class (SC)
Volume
Secrets may be exposed as files in dedicated volumes mounted in the pod.
Volume Type
- hostPath
- configMap
Dynamic Volume Provisioning
Dynamic volume provisioner https://kubernetes.io/docs/concepts/storage/dynamic-provisioning/