Linux cgroups

External

• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/html/resource_management_guide/ch01
• https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/html/resource_management_guide/chap-introduction_to_control_groups
• https://www.kernel.org/doc/Documentation/cgroup-v1/cgroups.txt
• https://en.wikipedia.org/wiki/Cgroups
• http://man7.org/linux/man-pages/man7/cgroups.7.html
• https://docs.docker.com/config/containers/runmetrics/#control-groups

Internal

• Docker Concepts

Overview

cgroups is a Linux kernel feature that allows allocation of resources (CPU, system memory, network bandwidth, or a combination of these) among user-defined groups of processes running on the system. cgroups not only track groups of processes, but they also expose metrics about CPU, memory and block I/O usage.

cgroups are exposed through a pseudo-filesystem available at /sys/fs/cgroup (older systems expose it at /cgroup). The sub-directories of the cgroup pseudo-filesystem root correspond to different cgroups hierarchies: cpu, freezer, blkio.

This command returns a list of the cgroups that are mounted:

cat /proc/mounts | grep cgroup

The control groups subsystems known to the system are available in /proc/cgroups:

#subsys_name	hierarchy	num_cgroups	enabled
cpuset	6	13	1
cpu	4	89	1
cpuacct	4	89	1
memory	8	89	1
devices	3	83	1
freezer	10	13	1
net_cls	5	13	1
blkio	11	89	1
perf_event	2	13	1
hugetlb	9	13	1
pids	7	13	1
net_prio	5	13	1

cgroups are organized hierarchically, child cgroups inheriting certain attributes from their parent group. Many different hierarchies of cgroups can exist simultaneously on a system. Each hierarchy is attached to one or more subsystem, where a subsystem represents a single resource like CPU time or memory.

To figure out what cgroups a process belongs to, look at /proc/<pid>/cgroup: the cgroup is shown as a path relative to the root of the hierarchy mount point. "/" means the process has not been assigned to a group, while "/lxc/something" means the process is member of a container named "something".

cgroups can be configured via the cgconfig service.

cgroups Subsystems

These subsystems are also known as "controllers":

blkio

Sets limits on input/output access from and to block devices.

cpu

Uses the scheduler to provide cgroup tasks access to the CPU.

cpuacct

Generates automatic reports on CPU resources.

More:

https://docs.docker.com/config/containers/runmetrics/#cpu-metrics-cpuacctstat

cpuset

Assigns individual CPUs and memory nodes to tasks in a cgroup.

memory

Memory metrics are found in the "memory" cgroup. To enable memory control group, add the following kernel command-line parameters:

cgroup_enable=memory swappacount=1

The metrics are available in "memory.stat".

More details:

https://docs.docker.com/config/containers/runmetrics/#metrics-from-cgroups-memory-cpu-block-io

net_cls

Tags network packets with a tag identifier (classid) that allow the Linux traffic controller (tc) to identify packets.

ns

The namespace subsystem.

Operations

The recommended location for cgroup hierarchies:

/sys/fs/cgroup