Kubernetes Concepts: Difference between revisions

From NovaOrdis Knowledge Base
Jump to navigation Jump to search
 
(379 intermediate revisions by the same user not shown)
Line 6: Line 6:


* [[Kubernetes#Subjects|Kubernetes]]
* [[Kubernetes#Subjects|Kubernetes]]
* [[Kubernetes_API_Resources_Concepts|API Resource Concepts]]
* [[Docker Concepts]]
* [[Docker Concepts]]
* [[OpenShift Concepts]]
* [[OpenShift Concepts]]
Line 11: Line 12:
=Overview=
=Overview=


Kubernetes is an ''container orchestration platform'', offering the ability of orchestrate [[Docker_Concepts#Container|Docker containers]] across multiple hosts. It manages containers in a clustered environment. It orchestrates containers at scale, defines application topologies, handles parts of the container networking, manages container state and schedules containers across hosts.
Kubernetes is an Apache 2.0 Open Source container orchestration platform, or a container orchestrator.


=Node=
A high level abstraction often used for Kubernetes is "data center OS". The primary use case for Kubernetes consists of containerized cloud-native applications, which are applications that are made from a set of small autonomous services (microservices) that communicate with each other. Kubernetes helps deploying, scaling up, scaling down, performing updates and rollbacks of these services, materialized as a set of containers. It does that at scale. It abstracts out details such as what specific compute nodes or physical storage volumes are allocated to applications.


A node is a ''Linux container host''.  
<span id='Control_Plane'></span>Kubernetes instances are known as <span id='Cluster'></span>[[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster|clusters]]. All management interactions with a Kubernetes cluster are performed by sending REST requests into an [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|API Server]]. The API Server is responsible with managing and exposing the state of the cluster. The state of the cluster is internally stored by a [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster_Store|cluster store]] control plane system service, which is currently implemented by [[etcd]]. The [[#Control_Loop|control loops]] essential to the [[#Declarative_versus_Imperative_Approach|declarative model]] implemented by Kubernetes are driven by various specialized controllers under the supervision of the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Controller_Manager|controller manager]]. The workloads are dispatched by the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Scheduler|scheduler]]. All these components - the API Server, cluster store, controllers, scheduler, cloud controller manager - are collectively known as the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Control_Plane|control plane]] and are executed on [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Master_Node|master nodes]].  Externally, the state can be accessed and modified with specialized tools, of which the most common is a command line client named [[kubectl]]. The control plane, the API server and API server-related details, such as [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Admission_Controllers|admission controllers]], are discussed in [[Kubernetes_Control_Plane_and_Data_Plane_Concepts|Control Plane and Data Plane Concepts]].


It is based on RHEL or Red Hat Atomic and provides a runtime environment where applications run inside [[#Container|containers]], which are contained in [[#Pod|pods]] assigned by the [[#Master|master]]. Nodes are orchestrated by [[#Master|masters]].  
Application workloads are deployed as [[Kubernetes_Pod_and_Container_Concepts#Pod|pods]] on a set of [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Worker_Node|worker nodes]] that constitute the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Data_Plane|data plane]]. Each node runs a <span id='Container_Runtime'></span>[[Kubernetes_Container_Runtime_Concepts#Container_Runtime|container runtime]], usually Docker. However, support for other container runtimes is available, via [[Kubernetes_Container_Runtime_Concepts#Container_Runtime_Interface_.28CRI.29|Container Runtime Interface (CRI)]]. Container runtime details are discussed in [[Kubernetes_Container_Runtime_Concepts|Container Runtime Concepts]].


Nodes can be organized into many different topologies.
Worker nodes are used to run workloads, deployed as <span id='Pod'></span>[[Kubernetes Pod and Container Concepts|pods]]. Pods are scheduled to nodes and then they are closely monitored. A pod is a wrapper that allows one or more containers to run on Kubernetes and it is the atomic unit of deployment in Kubernetes. Pods come and go - if a pod dies, it is not resurrected, but its failure is detected by the lack of response from configured [[Kubernetes Container Probes|probes]] that test expected container behavior and, depending on configuration, another pod may be scheduled as replacement. In consequence, the IP address of an individual pod cannot be relied on. Pod, containers, probes and pod scheduling are discussed in [[Kubernetes Pod and Container Concepts#Overview|Pod and Container Concepts]].  


[[OpenShift Concepts#The_node_Daemon|A node daemon]] runs on node each node.
Because the pods and their IPs are ephemeral, Kubernetes introduces an additional mechanism aimed at providing stable access point to a set of equivalent pods that belong to the same application: a <span id='Service'></span>[[Kubernetes Service Concepts#Service|service]]. A service can be thought of stable networking access for a continuously changing set of pods. A service's IP address and port can be relied on to not change for the life of the service. All live pods represented by a service at a moment in time are known as service "endpoints" - in fact, there is a Kubernetes resource representing a live pod within the context of a service, and it is called [[Kubernetes_Service_Concepts#Endpoints|endpoint]]. There are several types of services: ClusterIP, NodePort and LoadBalancer. The association between services and pods is loose - it is established logically by the service's selector, which is a label-based mechanism: a pod "belongs" to a service if the service's selector matches the pod's labels. Services are explained at length in the [[Kubernetes Service Concepts|Service Concepts]] section and selectors in [[Kubernetes_Selector_Concepts|Selector Concepts]]. A layer 7 complement to services, named Ingress, is available. Ingresses are discussed in [[Kubernetes Ingress Concepts|Ingress Concepts]].


=Pod=
A pod by itself has no built-in resilience: if it fails for any reason, it is gone. <span id='Higher_Level_Pod_Controller'></span>A higher level primitive - the [[Kubernetes_Workload_Resources#Deployment|deployment]] - is used to manage a set of pods from a high availability perspective: the deployment insures that a specific number of equivalent pods is always running, and if one of more pods fail, the deployment brings up replacement pods. The deployment relies on an intermediary concept - the [[Kubernetes_Workload_Resources#ReplicaSet|ReplicaSet]]. Deployments are used to implement rolling updates and rollbacks. There are higher-level pod controllers that manage sets of pods in different ways: [[Kubernetes_Workload_Resources#DaemonSet|DaemonSets]] and [[Kubernetes_Workload_Resources#StatefulSet|StatefulSets]]. Individual pods can be managed as [[Kubernetes_Workload_Resources#Job|Jobs]] or [[Kubernetes_Workload_Resources#CronJob|CronJobs]]. The pod controllers are discussed in [[Kubernetes_Workload_Resources#Overview|Kubernetes Workload Resources]].


{{External|https://kubernetes.io/docs/concepts/workloads/pods/pod/}}
Most Kubernetes resources can be logically grouped in namespaces. Services, pods, secrets, any many others are all namespaced. New namespaces can be created administratively, and all Kubernetes clusters come with a "default" namespace. There are resource types that cannot be allocated to namespaces, and those are named "cluster-level" resources. More details about namespaces are available in [[Kubernetes Namespace Concepts#Overview|Namespace Concepts]].


All Kubernetes resources can be annotated with labels and annotations, which are aimed at facilitating creating loose associations between resources. For more details see [[Kubernetes Labels and Annotations|Kubernetes Labels and Annotations]].


A pod runs one or more [[Docker Concepts#Container|containers]], deployed together on one host, as a single unit. A pod cannot span hosts.
A Kubernetes cluster exposes external storage to pods with three API resources: [[Kubernetes_Storage_Concepts#PV|PersistentVolumes]], [[Kubernetes_Storage_Concepts#PVC|PersistentVolumeClaims]] and [[Kubernetes_Storage_Concepts#SC|StorageClasses]], which are part of the [[Kubernetes_Storage_Concepts#Persistent_Volume_Subsystem|persistent volume subsystem]]. The actual storage is made available to Kubernetes by [[Kubernetes_Storage_Concepts#Storage_Plugin|storage plugins]], also known as provisioners, which should abide by the [[Kubernetes_Storage_Concepts#Container_Storage_Interface_.28CSI.29|Container Storage Interface (CSI)]]. All these are explained at length in [[Kubernetes Storage Concepts|Storage Concepts]].


The pod contains collocated applications that are relatively tightly coupled and run with a ''shared context''. Within that context, an application may have individual [[Linux cgroups|cgroups]] isolation applied. A pod models an application-specific ''logical host'', containing applications that in a pre-container world would have run on the same physical or virtual host.
Every pod in the Kubernetes cluster has its own IP address, which is directly routable to every other pod. The stable IP addresses provided by services are resolvable by the cluster's internal DNS service, as described in the [[Kubernetes_DNS_Concepts|DNS Concepts]]. This and other networking-related aspects are explained in [[Kubernetes Networking Concepts|Networking Concepts]].


Each pod has an IP address and can be assigned persistent storage [[#Volume|volumes]]. In consequence, all containers in a pod share the IP address, the [[#Volume|volumes]] and other resources allocated to the pod.  
Configuration can be exposed to pods specialized resources such as [[Kubernetes_Cluster_Configuration_Concepts#ConfigMap|ConfigMaps]] and [[Kubernetes_Cluster_Configuration_Concepts#Secret|Secrets]]. Kubernetes is exposing pod information to the containers running inside the pod through files, which are projected in the container by a mechanism known as the [[Kubernetes Downward API Concepts|Downward API]]. More details about configuration are available in [[Kubernetes Cluster Configuration Concepts|Configuration Concepts]].


The pod is the ''smallest unit'' that can be defined, deployed and managed. Kubernetes orchestrates pods.
Kubernetes security system ensures that the API sever is only accessed by authenticated identities, and the access is limited to resources that are supposed to be accessible to the authenticated identity, and also ensuring that the applications running in containers only access the node and network resources that are supposed to access, and nothing more. These aspects are discussed in [[Kubernetes Security Concepts#Overview|Security Concepts]].


Complex applications can be made of any pods.
Resources managed by Kubernetes are subject to <span id='Kubernetes_Policies'></span>[[Kubernetes Policies|policies]]: [[Kubernetes_Resource_Management_Concepts#Limit_Ranges|Limit Ranges]], [[Kubernetes_Resource_Management_Concepts#Resource_Quotas|Resource Quotas]] and specifically for pods, [[Kubernetes Pod Security Policy Concepts|Pod Security Policies]]. More details on resource management are available in [[Kubernetes Resource Management Concepts|Resource Management Concepts]].


<font color=red>'''Verify this applies to Kubernetes as well, not only OpenShift''':
Application health monitoring, resource consumption monitoring and scaling decisions require metrics to be collected and analyzed. Kubernetes facilitates metrics collection from containers, pods, services and other resources via metric pipelines. Metrics and metric pipelines are discussed in [[Metrics_in_Kubernetes#Overview|Metrics in Kubernetes]].


Pods are treated as ''static'', and cannot be changed while they are running. To change a pod, the current pod must be terminated, and a new one recreated with a modified base image, configuration, etc.
Pods can be automatically scaled up and down based on interpretation of their performance characteristic or resource consumption. Kubernetes provides a built-in autoscaling mechanism. For more details see [[Kubernetes_Autoscaling_Concepts#Overview|Autoscaling Concepts]].


Pods do not maintain state, they are expendable.
Kubernetes has built-in extension capabilities, allowing for [[Kubernetes_Custom_Resources|custom resources]] and registering multiple APIs servers via the [[Kubernetes Aggregation Layer#Overview|aggregation layer]]. A specific kind of extension are the [[Kubernetes_Operators_Concepts|operators]]. More details are available in [[Extending Kubernetes]].


Pods must not created or managed directly, but by controllers.
<span id='spec_and_status'></span>Spec and status: inputs and outputs.
</font>


{{Internal|OpenShift Concepts#Pod|OpenShift Pod}}
=Declarative versus Imperative Approach=
The preferred style while operating Kubernetes is to use a declarative model: Kubernetes likes to manage its resources declaratively, where we describe how we want our application to look - the '''desired state''' - in a set of YAML files, named [[Kubernetes Manifests#Overview|manifests]], POST these files into the Kubernetes [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|API Server]] with [[kubectl apply]] or other tools and wait for the changes to be applied. The [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Controller_Manager|controller manager]] and specialized controllers check whether the '''current state''' matches the desired state, and if the states do not match, they act to reconcile them, which usually happens after a short delay. <span id='Control_Loop'></span>This pattern is referred to as '''control loop'''. A "control loop" is a design pattern for distributed software that allows to define state declaratively and employ a controller to bring the current state to the desired state. It typically obtains the desired state, repeatedly observes the current state, determines differences and, if differences exist, reconciles differences. The terms "control loop" are used interchangeably with "watch loop" and "reconciliation loop".


==Pod Lifecycle==
Step-by-step, the declarative model works as follows:
# The desired state of the application is declared in the [[Kubernetes Manifests|manifest]] file.
# The manifest file is POSTed into the  [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#API_Server|API Server]], usually with [[kubectl]] command.
# The API server authenticates and authorizes the request and then validates the manifest.
# The API server stores the state - as desired state - in the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Cluster_Store|cluster store]].
# The API server identifies the [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Controller_Manager|controller]] responsible with enforcing and monitoring the state.
# The controller in charge implements the desired state, by adjusting the current state to match it.
# <span id='Current_State'></span>The [[Kubernetes_Control_Plane_and_Data_Plane_Concepts#Controller_Manager|controller manager]]'s control loops monitor the current state and make sure that it does not diverge from the desired state. The current state of a resource can be obtained from the cluster with [[Kubectl#Output_in_YAML_Format|CLI commands]].
# If the current state of the cluster diverges from the desired state, the cluster control plane will perform whatever tasks are necessary to bring those states in sync.


* A pod is defined.
This model is the opposite of the traditional imperative model, where a precise sequence of specific commands are issued to explicitly adjust the state. In other words, in the declarative model we tell the cluster how things should look, as opposite to telling it how to adjust the state.
* A pod is assigned to run on a node.
* The pod runs until its containers exit or the pod is removed.


==Pod Name==
Also see: {{Internal|Infrastructure_as_Code_Concepts#Declarative_Infrastructure_Languages|Declarative Infrastructure Languages}}


Pod must have an unique name in their namespace. The pod definition can specify a base name and use "generateName" attribute to append random characters at the end of the base name, thus generating an unique name.
=Subjects=


==Pod Definition File==
* [[Kubernetes_Control_Plane_and_Data_Plane_Concepts|Control Plane and Data Plane Concepts]]
* <span id='Container_Runtime_Concepts'></span>[[Kubernetes Container Runtime Concepts|Container Runtime Concepts]]
* [[Kubernetes Pod and Container Concepts|Pod and Container Concepts]]
** [[Kubernetes_Scheduling,_Preemption_and_Eviction_Concepts|Kubernetes Scheduling, Preemption and Eviction Concepts]]
* [[Kubernetes Service Concepts|Service Concepts]]
* [[Kubernetes Ingress Concepts#Overview|Ingress Concepts]]
* [[Kubernetes Selector Concepts|Selector Concepts]]
* [[Kubernetes Workload Resources#Overview|Kubernetes Workload Resources]]
* [[Kubernetes Namespace Concepts|Namespace Concepts]]
* [[Kubernetes Labels and Annotations|Labels and Annotations Concepts]]
* [[Kubernetes Storage Concepts|Storage Concepts]]
* [[Kubernetes Networking Concepts|Networking Concepts]]
** [[Kubernetes DNS Concepts|DNS Concepts]]
* [[Kubernetes Cluster Configuration Concepts|Cluster Configuration Concepts]]
** [[Kubernetes_Downward_API_Concepts#Overview|Downward API Concepts]]
* [[Kubernetes Security Concepts|Security Concepts]]
* [[Kubernetes Resource Management Concepts|Resource Management Concepts]]
** [[Kubernetes Policies]]
* [[Metrics in Kubernetes|Metrics in Kubernetes]]
* [[Kubernetes Autoscaling Concepts|Autoscaling Concepts]]
* [[Extending Kubernetes]]
** [[Kubernetes Aggregation Layer|Aggregation Layer]]
** [[Kubernetes Custom Resources|Custom Resources]]
** [[Kubernetes Operators Concepts|Operators]]
* <span id='API_Resources'></span>[[Kubernetes API Resources Concepts|API Resources Concepts]]
** [[Kubernetes_API_Resources_Concepts#Names|Kuberentes Names]]
** [[Kubernetes Manifests|Resource Manifests]]
* [[Kubernetes Patterns|Kubernetes Patterns]]


{{Internal|Pod Definition File|Pod Definition File}}
=Kubernetes Flavors=
 
* [[Managed Kubernetes]]
=Storage=
* [[OpenShift]]
 
* [[kind]]
==Volume==
 
{{Internal|OpenShift Concepts#Volume|OpenShift Volume}}
 
=etcd=
 
A distributed key/value datastore for state within the environment.
 
{{Internal|Etcd Concepts#Overview|etcd}}
 
=Scheduler=
 
Scheduling is essentially the master's main function: when a user decides to create a pod, the master determines where to do this - this is called ''scheduling''. The scheduler is a component that runs on master and determines the best fit for running [[#Pod|pods]] across the environment. The scheduler also spreads pod replicas across nodes, for application HA. The scheduler reads data from the pod definition and tries to find a nod that is a good fit based on configured policies. The scheduler does not modify the pod, it creates a binding that ties the pod to the selected node.
 
The scheduler is deployed as a container (referred to as an ''infrastructure container'').
 
{{Internal|OpenShift Concepts#Scheduler|OpenShift Scheduler}}
 
=Namespace=
 
{{Internal|OpenShift Concepts#Namespace|OpenShift Namespace}}
 
=Policies=
 
Policies are rules that specify which users can and cannot specify actions on objects ([[#Pods|pods]], [[#Service|services]], etc.).
 
{{Internal|OpenShift Concepts#Policies|OpenShift Policies}}
 
=Service=
 
A service represents a group of pods (which may come and go) and provides the permanent IP, hostname and port for other applications to use. A ''service resource'' is an abstraction that defines a logical set of pods and a policy that is used to access the pods. The service layer is how applications communicate with one another.
 
{{Internal|OpenShift Concepts#Service|OpenShift Service}}
 
=Router=
 
=Relationship between Service and Router=
 
=API=
 
{{Internal|OpenShift Concepts#API|OpenShift API}}
 
=Label=
 
Labels are simple key/value pairs that can be used to group and select arbitrarily related objects. Most Kubernetes objects can include labels in their metadata.
 
{{Internal|OpenShift Concepts#Label|OpenShift Label}}
 
=Selector=
 
A set of labels.
 
{{Internal|OpenShift Concepts#Selector|OpenShift Selector}}
 
=Replication Controller=
 
A component that insures a specified number of pod replicas defined in the environment state are running at all times. If pods exit or are deleted, the replication controller instantiates more pods up to desired number. If there are more pods running than desired, the replication controller deletes as many as necessary. It is NOT the replication controller's job to perform autoscaling based on load or traffic.
 
{{Internal|OpenShift Concepts#Replication_Controller|OpenShift Replication Controller}}

Latest revision as of 21:06, 22 March 2024

External

Internal

Overview

Kubernetes is an Apache 2.0 Open Source container orchestration platform, or a container orchestrator.

A high level abstraction often used for Kubernetes is "data center OS". The primary use case for Kubernetes consists of containerized cloud-native applications, which are applications that are made from a set of small autonomous services (microservices) that communicate with each other. Kubernetes helps deploying, scaling up, scaling down, performing updates and rollbacks of these services, materialized as a set of containers. It does that at scale. It abstracts out details such as what specific compute nodes or physical storage volumes are allocated to applications.

Kubernetes instances are known as clusters. All management interactions with a Kubernetes cluster are performed by sending REST requests into an API Server. The API Server is responsible with managing and exposing the state of the cluster. The state of the cluster is internally stored by a cluster store control plane system service, which is currently implemented by etcd. The control loops essential to the declarative model implemented by Kubernetes are driven by various specialized controllers under the supervision of the controller manager. The workloads are dispatched by the scheduler. All these components - the API Server, cluster store, controllers, scheduler, cloud controller manager - are collectively known as the control plane and are executed on master nodes. Externally, the state can be accessed and modified with specialized tools, of which the most common is a command line client named kubectl. The control plane, the API server and API server-related details, such as admission controllers, are discussed in Control Plane and Data Plane Concepts.

Application workloads are deployed as pods on a set of worker nodes that constitute the data plane. Each node runs a container runtime, usually Docker. However, support for other container runtimes is available, via Container Runtime Interface (CRI). Container runtime details are discussed in Container Runtime Concepts.

Worker nodes are used to run workloads, deployed as pods. Pods are scheduled to nodes and then they are closely monitored. A pod is a wrapper that allows one or more containers to run on Kubernetes and it is the atomic unit of deployment in Kubernetes. Pods come and go - if a pod dies, it is not resurrected, but its failure is detected by the lack of response from configured probes that test expected container behavior and, depending on configuration, another pod may be scheduled as replacement. In consequence, the IP address of an individual pod cannot be relied on. Pod, containers, probes and pod scheduling are discussed in Pod and Container Concepts.

Because the pods and their IPs are ephemeral, Kubernetes introduces an additional mechanism aimed at providing stable access point to a set of equivalent pods that belong to the same application: a service. A service can be thought of stable networking access for a continuously changing set of pods. A service's IP address and port can be relied on to not change for the life of the service. All live pods represented by a service at a moment in time are known as service "endpoints" - in fact, there is a Kubernetes resource representing a live pod within the context of a service, and it is called endpoint. There are several types of services: ClusterIP, NodePort and LoadBalancer. The association between services and pods is loose - it is established logically by the service's selector, which is a label-based mechanism: a pod "belongs" to a service if the service's selector matches the pod's labels. Services are explained at length in the Service Concepts section and selectors in Selector Concepts. A layer 7 complement to services, named Ingress, is available. Ingresses are discussed in Ingress Concepts.

A pod by itself has no built-in resilience: if it fails for any reason, it is gone. A higher level primitive - the deployment - is used to manage a set of pods from a high availability perspective: the deployment insures that a specific number of equivalent pods is always running, and if one of more pods fail, the deployment brings up replacement pods. The deployment relies on an intermediary concept - the ReplicaSet. Deployments are used to implement rolling updates and rollbacks. There are higher-level pod controllers that manage sets of pods in different ways: DaemonSets and StatefulSets. Individual pods can be managed as Jobs or CronJobs. The pod controllers are discussed in Kubernetes Workload Resources.

Most Kubernetes resources can be logically grouped in namespaces. Services, pods, secrets, any many others are all namespaced. New namespaces can be created administratively, and all Kubernetes clusters come with a "default" namespace. There are resource types that cannot be allocated to namespaces, and those are named "cluster-level" resources. More details about namespaces are available in Namespace Concepts.

All Kubernetes resources can be annotated with labels and annotations, which are aimed at facilitating creating loose associations between resources. For more details see Kubernetes Labels and Annotations.

A Kubernetes cluster exposes external storage to pods with three API resources: PersistentVolumes, PersistentVolumeClaims and StorageClasses, which are part of the persistent volume subsystem. The actual storage is made available to Kubernetes by storage plugins, also known as provisioners, which should abide by the Container Storage Interface (CSI). All these are explained at length in Storage Concepts.

Every pod in the Kubernetes cluster has its own IP address, which is directly routable to every other pod. The stable IP addresses provided by services are resolvable by the cluster's internal DNS service, as described in the DNS Concepts. This and other networking-related aspects are explained in Networking Concepts.

Configuration can be exposed to pods specialized resources such as ConfigMaps and Secrets. Kubernetes is exposing pod information to the containers running inside the pod through files, which are projected in the container by a mechanism known as the Downward API. More details about configuration are available in Configuration Concepts.

Kubernetes security system ensures that the API sever is only accessed by authenticated identities, and the access is limited to resources that are supposed to be accessible to the authenticated identity, and also ensuring that the applications running in containers only access the node and network resources that are supposed to access, and nothing more. These aspects are discussed in Security Concepts.

Resources managed by Kubernetes are subject to policies: Limit Ranges, Resource Quotas and specifically for pods, Pod Security Policies. More details on resource management are available in Resource Management Concepts.

Application health monitoring, resource consumption monitoring and scaling decisions require metrics to be collected and analyzed. Kubernetes facilitates metrics collection from containers, pods, services and other resources via metric pipelines. Metrics and metric pipelines are discussed in Metrics in Kubernetes.

Pods can be automatically scaled up and down based on interpretation of their performance characteristic or resource consumption. Kubernetes provides a built-in autoscaling mechanism. For more details see Autoscaling Concepts.

Kubernetes has built-in extension capabilities, allowing for custom resources and registering multiple APIs servers via the aggregation layer. A specific kind of extension are the operators. More details are available in Extending Kubernetes.

Spec and status: inputs and outputs.

Declarative versus Imperative Approach

The preferred style while operating Kubernetes is to use a declarative model: Kubernetes likes to manage its resources declaratively, where we describe how we want our application to look - the desired state - in a set of YAML files, named manifests, POST these files into the Kubernetes API Server with kubectl apply or other tools and wait for the changes to be applied. The controller manager and specialized controllers check whether the current state matches the desired state, and if the states do not match, they act to reconcile them, which usually happens after a short delay. This pattern is referred to as control loop. A "control loop" is a design pattern for distributed software that allows to define state declaratively and employ a controller to bring the current state to the desired state. It typically obtains the desired state, repeatedly observes the current state, determines differences and, if differences exist, reconciles differences. The terms "control loop" are used interchangeably with "watch loop" and "reconciliation loop".

Step-by-step, the declarative model works as follows:

  1. The desired state of the application is declared in the manifest file.
  2. The manifest file is POSTed into the API Server, usually with kubectl command.
  3. The API server authenticates and authorizes the request and then validates the manifest.
  4. The API server stores the state - as desired state - in the cluster store.
  5. The API server identifies the controller responsible with enforcing and monitoring the state.
  6. The controller in charge implements the desired state, by adjusting the current state to match it.
  7. The controller manager's control loops monitor the current state and make sure that it does not diverge from the desired state. The current state of a resource can be obtained from the cluster with CLI commands.
  8. If the current state of the cluster diverges from the desired state, the cluster control plane will perform whatever tasks are necessary to bring those states in sync.

This model is the opposite of the traditional imperative model, where a precise sequence of specific commands are issued to explicitly adjust the state. In other words, in the declarative model we tell the cluster how things should look, as opposite to telling it how to adjust the state.

Also see:

Declarative Infrastructure Languages

Subjects

Kubernetes Flavors