cgroups v2 Basics for Sysadmins

Introduction¶

cgroups (control groups) are how Linux accounts for and limits CPU, memory, and I/O per group of processes. Containers use them; systemd uses them for every service. Knowing the basics lets you cap a memory-hungry service or find which unit is eating the box.

Concepts¶

cgroups v2 replaced v1's separate per-controller hierarchies with one unified hierarchy mounted at /sys/fs/cgroup. Every process is in exactly one cgroup. Controllers (cpu, memory, io, pids) are enabled per subtree.

systemd is the cgroup manager on modern distros. It creates:

• system.slice/ — system services, each foo.service a cgroup
• user.slice/user-1000.slice/ — logged-in users and their sessions
• machine.slice/ — VMs and containers

So nginx.service is already system.slice/nginx.service with its own accounting. You just set limits on it.

Hands-on¶

See usage per unit¶

systemd-cgtop                    # like top, but by cgroup (CPU, memory, IO)
systemctl status nginx.service   # shows the cgroup + current Memory/CPU/Tasks
systemd-cgls                     # the tree of cgroups and their processes
cat /sys/fs/cgroup/system.slice/nginx.service/memory.current

Cap a service¶

sudo systemctl edit myapp.service

[Service]
# Memory: throttle then OOM-kill the cgroup (not the whole box)
MemoryHigh=1500M          # start reclaiming aggressively above this
MemoryMax=2G              # hard limit — OOM-kills within the cgroup if exceeded

# CPU: 150% = 1.5 cores worth
CPUQuota=150%
CPUWeight=50             # relative share under contention (default 100)

# IO
IOWeight=50
IOReadBandwidthMax=/dev/sda 50M

# Process count (fork-bomb guard)
TasksMax=512

sudo systemctl daemon-reload
sudo systemctl restart myapp.service
systemctl show myapp.service -p MemoryMax -p CPUQuota

MemoryMax is the safety valve: a leaking service gets OOM-killed inside its own cgroup instead of the kernel killing something random on the host.

Limit a whole slice¶

Group related services and cap them together:

# /etc/systemd/system/batch.slice
[Slice]
CPUQuota=200%
MemoryMax=4G

Then Slice=batch.slice in each service. Everything in the slice shares the 4G / 2-core budget.

Transient limits (no unit file)¶

sudo systemd-run --scope -p MemoryMax=500M -p CPUQuota=50% ./heavy-import.sh

Verification and troubleshooting¶

# Was a service OOM-killed within its cgroup?
journalctl -u myapp.service | grep -i "oom\|killed"
cat /sys/fs/cgroup/system.slice/myapp.service/memory.events   # oom, oom_kill counts

# Is it being CPU-throttled?
cat /sys/fs/cgroup/system.slice/myapp.service/cpu.stat        # nr_throttled, throttled_usec

• MemoryMax set but service still takes down the box — you set MemoryHigh only (that just throttles), or the process is using shared memory / page cache attributed elsewhere, or an old kernel on cgroups v1. Check cat /sys/fs/cgroup/system.slice/*/memory.current.
• CPUQuota ignored — the cpu controller isn't enabled for that subtree. systemd enables it when any unit in the slice requests a CPU property; cat /sys/fs/cgroup/system.slice/cgroup.controllers should list cpu.
• Service restarts in a loop after hitting MemoryMax — it's being OOM-killed and Restart= brings it back. Fix the leak or raise the limit; add StartLimitIntervalSec/StartLimitBurst to stop the loop.
• systemd-cgtop shows a slice eating CPU but no obvious service — systemd-cgls to expand it; often user.slice (a runaway user process, a broken cron) or machine.slice (a container).
• Still on cgroups v1 — stat -fc %T /sys/fs/cgroup says tmpfs (v1) vs cgroup2fs (v2). Add systemd.unified_cgroup_hierarchy=1 to the kernel cmdline; most distros since ~2021 are v2 by default.

Related tools and reading¶

• On-site: SAR Report Analyzer, Linux commands reference.
• Related posts: How systemd unit files work, Understanding Linux load average.