Kubernetes Cluster Configuration Concepts
Internal
Overview
Containerized applications can be configured with:
- command-line arguments, hardcoded into the container's manifest
- environment variables, hardcoded into the container's manifest
- mounting configuration files into containers through a volume
- ConfigMaps
- Secrets
ConfigMap
ConfigMaps are containers for storing configuration data. Other objects, for example pods, can access the data in a ConfigMap. ConfigMap data can be projected into a pod as an environment variable or as a file, part of a special volume mount. This mechanism decouples configuration lifecycle from the pod (pod image) lifecycle. ConfigMaps can be created from a manifest file, or imperatively on command line, by specifying values of individual key value pairs or entire files.
ConfigMap Overview
ConfigMap Projection
ConfigMap Operations
Secrets
Secret Overview
A secret is a Kubernetes mechanism that can be used to expose security-sensitive data to pods as environment variables or as files. The most common method to create a secret is to write the content into a file, then use the file as argument to the Kubernetes API to create the secret resource. It is also possible to create a secret entirely from a manifest.
Secret Details
A secret is a mechanism, backed by the Secret Kubernetes API resource, that allows applications running on a Kubernetes cluster to safely manage, store and access security-sensitive information such as passwords, OAuth tokens and ssh keys. This mechanism provides a better alternative to placing that information in a container image or in the pod metadata. An individual secret contains a small amount of data, limited to 1 MiB - this is to discourage creation of very large secrets that would exhaust API server and kubelet memory. Entire multi-line configuration files can be exposed as secrets.
A Secret instance contains two maps: the data map, which is used to store arbitrary key/value pairs, where the values are base64-encoded string, and stringData map, which is a field provided for convenience that allows to provide secret data as unencoded fields. "stringData" field allows putting a non-base64 encoded string directly into the secret, and the string will be encoded by Kubernetes when the Secret is created or updated. When a secret instance is queried, only the data map is exposed.
A secrets exists in a namespace and they can only be referenced by pods in the same namespace. Multiple pods can reference the same secret. A pod must explicitly reference a secret in its manifest to access it. If that does not happen, the system will not initialize the infrastructure that exposes the information to the pod.
Both users and the system can create secrets. For example, service accounts automatically create secrets which contain credentials for accessing the API server and they automatically expose these secrets to the pods. Using these secrets is the recommended method to securely access the API server.
Secret Projection
Secrets are consumed by applications, and they can are projected into pods in two ways: as files and as environment variables. However, secrets can also be used by other parts of the system, without being directly exposed to pods. A secret needs to be created before a pod that depends on it is scheduled: once a pod is scheduled, the kubelet will try to fetch the secret value. If the secret cannot be fetched because it does not exist, kubelet will periodically retry and will report the occurrence as an event. Once the secret is fetched, the kubelet will create and mount a volume containing it. None of the pod’s containers will start until all the pod’s volumes are mounted.
Secrets Projected as Files
One option is to project secrets as files in dedicated volumes mounted in the pod. When the pod deploys, each key/value entry of the secret's data map is exposed as an individual file under the volume associated with the secret. The the file name is given by the key and the file content is filled with decoded secret value corresponding to the key. For specific configuration details, see Consume a Secret as a File. By default, each key is projected as a file in the volume root. However, specific keys can be mapped onto arbitrary relative paths in the volume, and the permissions under which corresponding files are exposed can be configured individually. For more details, see Projection of Keys to Specific Paths with Specific Permissions. Many files (key/value pairs) can be packaged into one secret, or many secrets can be used, whichever is most convenient.
If the key name starts with a dot, it will be projected as a "hidden file".
When a secret already projected in a volume is updated, the projected keys are eventually updated as well. The kubelet periodically checks whether the mounted secret is fresh. However, it is using its local cache for getting the current value of the secret, so depending on the cache configuration, the propagation delay of the fresh value to the pod can be as long as kubelet sync period + cache propagation delay, which depends on the cache type. Note that a secret propagated via a subPath volume mount will not receive updates. For more details on kubelet cache see:
Secrets Projected as Environment Variables
Another option would be to project secrets as environment variables in the pod's container environments. This option does not require declaring and mounting a volume, but just simply referencing a secret's key - each key/value pair declared in the secret is projected as an individual environment variable. The secret's keys appear as normal environment variables containing the base-64 decoded values of the secret value, inside the container's environment.
If a key is not a valid environment variable, that key will be skipped, and an event generated, but the pod will be allowed to start.
Secret Types
Opaque
kubernetes.io/service-account-token
Accessible on pods as /var/run/secrets/kubernetes.io/serviceaccount.
imagePullSecrets
Image pull secrets are essentially a combination of registry, username, and password. TODO:
- https://kubernetes.io/docs/concepts/configuration/secret/#using-imagepullsecrets
- https://kubernetes.io/docs/concepts/containers/images/#specifying-imagepullsecrets-on-a-pod
- https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/#add-imagepullsecrets-to-a-service-account
Secrets Operations
- Create a Secret with kubectl CLI
- Create a Secret from a manifest
- Consume a Secret as a File
- Consume a Secret as an Environment Variable
Secrets TODO
- Automatic mounting of manually creates secrets. Manually created secrets, like those for accessing a GitHub account can be automatically attached to pods based on their service account. https://kubernetes.io/docs/tasks/inject-data-application/podpreset/
- Pod with ssh keys: https://kubernetes.io/docs/concepts/configuration/secret/#use-case-pod-with-ssh-keys
- Generating Secrets https://github.com/kubernetes-sigs/kustomize/blob/master/examples/secretGeneratorPlugin.md