>_ IzzyLab

Hands-on task

This task is step 9 of the course “Containers from the inside: namespaces and cgroups”

The index rebuild on short rations: cpu.max

medium~10 min

The server is rebuilding the search index with /opt/reindex/run.sh. It takes a whole core, and the API on the same server has started to slow down. You can't stop or restart the rebuild: it takes about an hour, and after a restart it begins from scratch.

Restrict the already running process with cgroup v2: create a group reindex and make the rebuild get no more than 20% of one core. The process must keep running — with the same PID.

What you'll practise:
  • cgroups
  • cgroup v2
  • cpu.max
  • throttling

Topic breakdown

The cpu controller in cgroup v2 limits the CPU with a quota: cpu.max takes <quota> <period> in microseconds, and in each period the whole group may spend no more than the quota. 50000 100000 is half of one core, 200000 100000 is two cores, max 100000 means no limit. Having used up its quota, the group waits for the next period to begin — this is called throttling, and you can see it in cpu.stat (nr_throttled, throttled_usec). A process can be moved into a group on the fly by writing its PID to cgroup.procs: the kernel starts applying the limit from the next period, with nothing restarted. docker run --cpus and limits.cpu in Kubernetes write exactly this file.

What's checked

  1. The rebuild is running and hasn't been restarted
  2. The rebuild is moved into the group reindex
  3. The group reindex's limit is at most 20% of a core
  4. The kernel really throttles the rebuild, and it keeps running

What it looks like