Metrics in Kubernetes

Internal

• Kubernetes Resource Management Concepts
• Kubernetes Concepts
• Kubernetes Autoscaling Concepts

Overview

Application health monitoring, resource consumption monitoring and scaling decisions require metrics collection and analysis. Kubernetes facilitates metrics collection from containers, pods, services and the overall cluster via metric pipelines.

Metric Pipeline

A metric pipeline is a solution that allows metrics collection, propagation, possibly storage, and publishing. In Kubernetes, application monitoring does not depend on a single monitoring solution. Kubernetes allows for different types of metric pipelines: resource metrics pipelines and full metrics pipelines.

Resource Metrics Pipeline

https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/#resource-metrics-pipeline

A resource metrics pipeline provides a limited set of metrics that are consumed by the horizontal pod autoscaler or kubectl top utility. The metrics are collected by the lightweight, short-term, in-memory metrics-server and exposed via the metrics.k8s.io Resource Metrics API. Also see:

Kubernetes Metrics Server

Full Metrics Pipeline

https://kubernetes.io/docs/tasks/debug-application-cluster/resource-usage-monitoring/#full-metrics-pipeline

A full metrics pipeline provides access to richer metrics than the resource metrics pipeline. The monitoring pipeline fetches metrics from the kubelet and then exposed them to Kubernetes via an adapter by implementing either the custom.metrics.k8s.io or external.metrics.k8s.io APIs.

Metrics

Resource Metrics

A resource metric is a numeric quantity that tracks either the CPU or memory consumed by containers and pods. By default, the only two supported resource metrics are the CPU utilization and the memory consumed by a container. These resources do not change names from cluster to cluster and they should be available as long the Resource Metrics API is available.

Resource Metrics API

metrics.k8s.io

The API is served at /metrics/resource/v1beta1 on the kubelet's authenticated and read-only ports.

TODO:

• https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/resource-metrics-api.md
• https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md

Custom Metrics

Aside from resource metrics, there are two other types of metrics, both of which are considered custom metrics: pod metrics and object metrics. Custom metrics track resources used by Kubernetes objects (pods or otherwise).

Pod Metrics

Pod metrics track resources that describe pods.

Object Metrics

Object metrics track resources describe different objects in the same namespace, instead of describing pods.

Custom Metrics API

custom.metrics.k8s.io

Monitoring systems like Prometheus expose application-specific metrics to the Horizontal Pod Autoscaler controller via the Custom Metrics API.

External Metrics

External Metrics API

external.metrics.k8s.io

Sources of Metrics

Kubernetes Metrics Server

Kubernetes Metrics Server

Prometheus

Prometheus

Notes

• More details about resource metrics API.
• Metrics: raw values, utilization values.
• Metric server: https://kubernetes.io/docs/tasks/debug-application-cluster/resource-metrics-pipeline/, https://github.com/kubernetes-sigs/metrics-server
• Custom metrics API: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/custom-metrics-api.md
• External metrics API: https://github.com/kubernetes/community/blob/master/contributors/design-proposals/instrumentation/external-metrics-api.md