OpenShift Resource Management Concepts

External

• https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-guide-compute-resources
• https://docs.openshift.com/container-platform/latest/admin_guide/quota.html
• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/resource_management_guide/

Internal

• OpenShift Concepts
• Linux Resource Management
• Kubernetes Resource Management Concepts

Overview

OpenShift provides API-level support for establishing and enforcing resource quotas. The runtime monitors resource usage and intervenes when the quotas are reached or exceeded. Resource quotas can be set up and managed on the following type of resources: the quantity of objects that can be created per project, the amount of compute resources (requests and limits on CPU and memory) consumed by the project and individually by project entities such as pods and containers, the amount of storage consumed by project entities. Opaque integer resources can also be set and monitored.

Resource monitoring and consumption is important because it insures that no projects are using more resources that is appropriate for the cluster size. Primarily, resource constraints are set by the cluster administrators, but developers can also set request and limits on compute resources.

Quota

A resource quota specifies constraints that limit aggregate resource consumption per project, and they are set by cluster administrators. The resource quota for a project is defined by a ResourceQuota object. Resource quotas per cluster can be managed with ClusterResourceQuota. The resource quota limits:

• the quantity of objects, per type, that can be created in a project:
  • Pods ("pods") - the total number of pods in a non-terminal state.
  • ConfigMaps ("configmaps")
  • Replication Controllers ("replicationcontrollers")
  • Secrets ("secrets")
  • Services ("services")
  • Image Streams ("openshift.io/imagestreams")
  • Resource Quotas ("resourcequotas")
• the total amount of compute resources consumed by the project. Note that if a quota has a value specified for "requests.cpu" or "requests.memory", then it requires that every incoming container make an explicit request for those resources. The same rule applies for "limits.cpu" and "limits.memory":
  • CPU Requests ("cpu" and "requests.cpu" are equivalent) - the sum of CPU requests across all pods in a non-terminal state.
  • CPU Limits ("limits.cpu") - the sum of CPU limits across all pods in a non-terminal state.
  • Memory Requests ("memory" and "requests.memory" are equivalent) - the sum of memory requests across all pods in a non-terminal state.
  • Memory Limits ("limits.memory") - the sum of CPU limits across all pods in a non-terminal state.
• the total amount of storage consumed by the project:
  • Persistent Volume Claims ("persistentvolumeclaims")
  • "requests.storage" - across all persistent volume claims in the project, the sum of storage requests cannot exceed this value.
  • "gold.storageclass.storage.k8s.io/persistentvolumeclaims", "gold.storageclass.storage.k8s.io/requests.storage"
  • "silver.storageclass.storage.k8s.io/persistentvolumeclaims", "silver.storageclass.storage.k8s.io/requests.storage"
  • "bronze.storageclass.storage.k8s.io/persistentvolumeclaims", "bronze.storageclass.storage.k8s.io/requests.storage"

Quota Scope

The compute resource quotas can be restricted only to pods in a non-terminal state within a certain scope. Each quota can have an associated set of scopes, and the quota will only measure usage for a resource if it matches the intersection of enumerated scopes. The scope can designate the pod's quality of service ("BestEffort", "NotBestEffort"), or type ("NonTerminating", "Terminating").

apiVersion: v1
kind: ResourceQuota
spec:
  ...
  scopes:
  - BestEffort 2

A "BestEffort" scope restricts quota to limiting the number of pods ("pods").

A "Terminating", "NotTerminating" and "NotBestEffort" scope restricts a quota to tracking the following resources: "pods", "requests.memory"/"memory", "limits.memory" "requests.cpu"/"cpu", "limits.cpu".

Quota Enforcement

After a quota is first declared on a project, the system restricts the ability to create new resources that may exceed the quota until usage statistics are calculated. Once the usage statistics are updated, content can be created or modified, but only if by doing so quota is not exceeded. If the quota is exceeded, the action will be denied and an error message will be returned. The quota will be modified immediately after creation/update. When a resource is deleted, the quota is decremented during the next full recalculation of quota statistics per project.

Quota Operations

All quotas for a project can be obtained with oc get quota, and information about individual quotas can be obtained with oc describe quota. Quotas can also be viewed from the web console, in the project's "Quota" page. Quotas can be created with oc create.

Limit Range

https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#dev-limit-ranges

A limit range enumerates compute resource constraints in a project a the pod/container/image/image stream/persistent volume claim level, and specifies the amount of resources that a pod/container/image/image stream/persistent volume claim can consume. Limit ranges are defined by the LimitRange object. All resource creation and modification requests are evaluated against each limit range in the project and rejected if the resource request is outside limits. If the resource does not set an explicit value, and if the constraint supports a default value, the default value is applied to the resource. Limit ranges are set by the cluster administrators and are project-scoped.

The limits are accessed with:

oc get limits [-n project-name]

Opaque Integer Resources

https://docs.openshift.com/container-platform/latest/dev_guide/compute_resources.html#opaque-integer-resources-dev

Compute Resources for System Daemons

https://kubernetes.io/docs/tasks/administer-cluster/reserve-compute-resources/

Handling Out of Resource Events

https://access.redhat.com/documentation/en-us/openshift_container_platform/3.7/html/cluster_administration/admin-guide-handling-out-of-resource-errors

Access to Resource Specification from Pods

...
       - env:
         - name: FLUENTD_CPU_LIMIT
           valueFrom:
             resourceFieldRef:
               containerName: fluentd-elasticsearch
               divisor: "0"
               resource: limits.cpu
         - name: FLUENTD_MEMORY_LIMIT
           valueFrom:
             resourceFieldRef:
               containerName: fluentd-elasticsearch
               divisor: "0"
               resource: limits.memory
...