Kubernetes Horizontal Pod Autoscaler

External

• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/
• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/
• https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md
• https://github.com/kubernetes/community/blob/master/contributors/design-proposals/autoscaling/hpa-v2.md
• https://towardsdatascience.com/kubernetes-hpa-with-custom-metrics-from-prometheus-9ffc201991e

Internal

• Kubernetes Autoscaling Concepts
• Metrics in Kubernetes
• metrics-server

Overview

One of the basic Kubernetes features is the ability to manually horizontally scale pods up and down by increasing or decreasing the desired replica count field on the pod controller. Automatic scaling is built in top of that. Horizontal pod autoscaling is the automatic increase or decrease the number of pod replicas managed by a higher level controller that supports scaling, such as deployments, replica sets and stateful sets. The autoscaling mechanism aims for a target value for one or more metrics. By increasing the number of pods, the average value per pod for the autoscaling target value should come down. Metrics appropriate for autoscaling are those metrics whose average value decreases linearly with the increase of pod replica count.

How it Works

The scaling is performed by a horizontal pod controller and it is configured by a HorizontalPodAutoscaler Kubernetes API resource, which enables and configures the horizontal pod autoscaler. For an horizontal pod autoscaler to work correctly, a source of metrics, most commonly resource metrics, must be deployed. In case of resource metrics, the autoscaler uses the corresponding resource request as a reference, so if some of the containers do not have the relevant resource request set, the autoscaler will not take any action for that metric. This does not apply for custom and external metrics. The simplest source of resource metrics is the metrics server.

The controller periodically reads the appropriate metrics APIs, as specified in the HorizontalPodAutoscaler resource definition, and obtains metrics for the pods it monitors. To be able to access metrics APIs, --horizontal-pod-autoscaler-use-rest-clients must be set to true, the API aggregation layer must be enabled, and the corresponding APIs (resource metrics API, custom metrics API or external metrics API) must be registered. The query cycle period is set with --horizontal-pod-autoscaler-sync-period.

The set of pods to watch is provided by the higher level pod controller the autoscaler is associated with. The autoscaler calculates the number of replicas required to meet the target metric, as described in the Autoscaling Algorithm section, below. If there is a mismatch, the controller adjusts the "replicas" field through the Scale sub-resource of the pod controller. The target pod controller is not aware of the autoscaler. In what it is concerned, anybody, including the autoscaler, may update the replica count.

Each component involved in autoscaling gets metrics from its source periodically. The end effect is that it takes a while for metrics to be propagated and a rescaling action to be performed.

Autoscaling can be performed for one metric or multiple metrics. If multiple metrics are involved, the controller computes the required number of pods for each metric, and picks the larger value.

Utilization vs. raw values. If a target utilization value is set, the controller calculates the utilization value as a percentage of the equivalent resource request on the containers in each pod. Utilization values can only be used with resource metrics, because only for resource metrics we can set requests. If a target raw value is set, the raw metric values are used directly. The controller then takes the mean or average(?) of the utilization or the raw value across all targeted pods, and produces a ratio used to scale the number of desired replicas. If per-pod custom metrics are used in autoscaling, the controller works with raw values, not utilization values.

For object metrics and external metrics, a single metric is fetched and it compared to the target value to produce a ratio. In autoscaling/v2beta2 API version, the value can optionally be divided by the number of pods before the comparison is made.

Horizontal Pod Autoscaler Controller

The horizontal pod autoscaler controller is part of the cluster's controller manager process. The controller periodically adjusts the number of replicas.

Autoscaling Algorithm

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#algorithm-details

For a specific metric, the goal of the algorithm is to compute the number of replicas that will bring the average value of the metric as close to the target value as possible. The tolerance can be set with --horizontal-pod-autoscaler-tolerance. The input is a set of metrics, one for each pod in case of resource metrics or pod custom metrics, or just one metric for object or external metrics. The output is an integer, which represents the target number of pod replicas.

The algorithm also needs to make sure that the autoscaler does not thrash around when the metric value is unstable and changes rapidly. One parameter that can be used to control this behavior is --horizontal-pod-autoscaler-downscale-stabilization.

The autoscaler will at most double the number of replicas in a single operation, if more that two current replicas exist. If only one replica exists, there is no such limitation. The autoscaler also has a limit of how soon a subsequent autoscale operation can occur after the previous one. By default, a scale-up will occur only if no rescaling event occurred in the last three minutes and a scale-down is performed every five minutes.

Horizontal Pod Autoscaler Resource

The HorizontalPodAutoscaler Kubernetes API resource is deployed as any other Kubernetes resource by posting a manifest to the API server. The resource determines the behavior of the controller. The HorizontalPodAutoscaler minReplicas field cannot be set to zero, so the autoscaler will never scale down to zero pods, as currently idling and un-idling is not supported. The resource can be changed after deployment, the changes will be detected and acted upon.

HorizontalPodAutoscaler Manifest

HorizontalPodAutoscaler Manifest

CPU-based Scaling

The CPU-based scaling algorithm uses the guaranteed CPU amount (the CPU requests) when determining the CPU utilization of the pod. This means the pod needs to have the CPU requests set, either directly or through a LimitRange object, to be eligible for autoscaling. CPU average utilization is the container's actual CPU usage divided by its CPU request value.

Also see:

• Create a CPU-based horizontal pod autoscaler with CLI
• Create a CPU-based horizontal pod autoscaler with Metadata

Memory-based Scaling

Memory-based autoscaling is much more problematic than CPU-based autoscaling. The main reason is because after scaling up, the old pods would somehow need to be forced to release memory. This needs to be done by the application itself, it cannot be done by the system. All the system could do is to kill and restart the application, hoping that it would use less memory than before.

Custom Metrics-based Scaling

Autoscaling solutions based on custom metrics are natively supported by v2beta2 Horizontal Pod Autoscalers.

The most common custom metrics-based scaling involves Prometheus-based metrics collection and a Prometheus adapter that exposes collected metrics over the Kubernetes Custom Metrics API and External Metrics APIs. The Prometheus adapter will act as an extension API server, operating at the Kubernetes aggregation layer, so the aggregation layer needs to be enabled in the cluster for the solution to work.

Prometheus Adapter for Kubernetes Metrics APIs

External Metrics-based Scaling

Autoscaling solutions based on external metrics are natively supported by v2beta2 Horizontal Pod Autoscalers.

Also see:

External Metrics

TODO:

• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#autoscaling-on-metrics-not-related-to-kubernetes-objects

Configurable Scaling Behavior

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#support-for-configurable-scaling-behavior

Scaling behavior can be configured with the HorizontalPodAutoscaler resource "behavior" field. Behaviors are specified separately for scaling up, in "scaleUp" section and scaling down, in "scaleDown" section. A stabilization window can be specified for both directions.

TODO:

• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#scaling-policies
• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#stabilization-window
• https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale/#default-behavior

Horizontal Pod Autoscaler Status Conditions

https://kubernetes.io/docs/tasks/run-application/horizontal-pod-autoscale-walkthrough/#appendix-horizontal-pod-autoscaler-status-conditions

• AbleToScale
• ScalingActive
• ScalingLimited

Playground

https://github.com/ovidiuf/playground/tree/master/kubernetes/horizontal-pod-autoscaler

Operations

• Create a CPU-based horizontal pod autoscaler with CLI
• Create a CPU-based horizontal pod autoscaler with Metadata
• Horizontal Pod Autoscaler troubleshooting