cgroups and CPU: how Linux splits the processor between groups of processes

Why group processes at all

Assume one CPU and two applications.

Application A:

A1

Application B:

B1
B2
B3
B4
B5
B6
B7
B8
B9

If the scheduler looked only at individual tasks and each of them had the same share, we would have ten competitors in total.

In a very simplified model:

A1 ~ 10% CPU

B1..B9 ~ 90% CPU

So application B gets nine times more CPU purely because it created nine tasks.

Sometimes that is exactly what we want.

But often it is not.

We may want to say:

application A = one group
application B = another group

and first split the CPU between the groups:

A ~ 50%
B ~ 50%

and only then distribute group B's share among its nine tasks.

That is one of the problems cgroups solve.

The Linux scheduler supports group scheduling: tasks can be grouped, and CPU time distributed between groups instead of treating every process as one entry in a flat list of equal competitors.

What a cgroup is

cgroup stands for control group.

Put simply:

a cgroup is a kernel mechanism that lets you place a set of processes into a hierarchical group and apply resource management rules to that group.

The kernel describes cgroups as a mechanism for organising processes hierarchically and distributing resources along that hierarchy in a controlled way.

Imagine:

root
├── application-a
│   ├── process A1
│   └── process A2
│
└── application-b
    ├── process B1
    ├── process B2
    ├── process B3
    └── process B4

The processes are still ordinary Linux tasks.

A cgroup does not create a new kind of process.

What it does give the kernel is extra information:

A1 and A2 belong to one group

B1..B4 belong to another

On that basis a resource controller can apply rules to the whole group.

A cgroup is not only about CPU

The cgroup mechanism itself is mainly responsible for:

organising processes
+
hierarchy

It is the controllers that implement behaviour for specific resources.

So we can have control over, among others:

CPU
memory
I/O
pids

In this article we care only about:

cpu controller

Its job is to influence how groups of tasks share the processor.

Without cgroups: the scheduler sees tasks

Imagine four CPU-bound tasks:

A1
A2
B1
B2

Under identical conditions we can simplify the situation to:

A1 ~ 25%
A2 ~ 25%
B1 ~ 25%
B2 ~ 25%

In total:

A ~ 50%
B ~ 50%

Now application B creates six more workers:

A1
A2

B1
B2
B3
B4
B5
B6
B7
B8

If all tasks competed on the same terms, application B would end up with a much larger share of the CPU.

That shows why the number of threads should not automatically define an application's share of the machine.

With cgroups we can split CPU between applications first

We create:

root
├── group-A
│   ├── A1
│   └── A2
│
└── group-B
    ├── B1
    ├── B2
    ├── B3
    ├── B4
    ├── B5
    ├── B6
    ├── B7
    └── B8

And we set both groups as equals.

Then we can get this split:

group-A ~ 50%
group-B ~ 50%

Only inside group A do:

A1
A2

divide its share of the CPU between themselves.

Likewise, B's eight tasks divide group-B's share.

So the scheduler can reason hierarchically:

CPU
 |
 +-- 50% -> group-A
 |           |
 |           +-- A1
 |           +-- A2
 |
 +-- 50% -> group-B
             |
             +-- B1
             +-- B2
             +-- ...

That is the essence of group scheduling.

CPU weight

Modern cgroup v2 exposes a CPU parameter:

cpu.weight

The default value is:

100

and the available range is:

1 .. 10000

cpu.weight defines the group's relative weight against its active siblings.

Assume:

group-A:
cpu.weight = 100

group-B:
cpu.weight = 100

If both groups need CPU all the time, we expect roughly:

A : B
1 : 1

That is:

A ~ 50%
B ~ 50%

Let us change the configuration:

A = 100
B = 200

Now the ratio is:

1 : 2

so under full competition, roughly:

A ~ 33%
B ~ 67%

Weight is therefore a proportion.

Not an amount of CPU.

Weight does not say "how much CPU you get"

This is important.

cpu.weight = 100

does not mean:

100% CPU

Nor:

1 CPU

Nor:

100 ms CPU

Weight on its own says almost nothing without knowing the competition.

If:

A = 100
B = 100

we get about:

50% : 50%

If we add:

C = 100

we get about:

33% : 33% : 33%

We did not change A's configuration.

And yet its share changed.

That is why weight should always be read as:

my weight
relative to the weights of other active groups

"Active" is very important here

Assume again:

A weight = 100
B weight = 100

But workload B is currently doing nothing.

Then A is not artificially capped at 50%.

It can use the available CPU.

The kernel describes weight-based distribution as work-conserving: only groups that can actually use the resource at a given moment take part in the split.

So:

A wants CPU
B idle

may give:

A ~ 100%
B ~ 0%

despite identical weights.

Weights start defining a proportion only when real competition appears.

Hierarchy matters

cgroups form a tree.

That means the CPU split does not have to happen globally between all groups at once.

Example:

root
├── A weight=100
│
└── B weight=100
    ├── B1 weight=100
    └── B2 weight=100

First A and B compete:

A
B

If both are active:

A ~ 50%
B ~ 50%

Only then is B's share divided between:

B1
B2

so roughly:

B1 ~ 25% of the whole CPU
B2 ~ 25% of the whole CPU

Finally:

A  ~ 50%
B1 ~ 25%
B2 ~ 25%

And this is exactly why you cannot take two cpu.weight values sitting in different places of the tree and compare them directly.

Weight matters mainly between sibling cgroups, that is, groups competing at the same level of the hierarchy.

Historically: cpu.shares

If you have worked with older systems or cgroup v1, instead of:

cpu.weight

you may have met:

cpu.shares

The idea was similar.

An example from the kernel documentation:

multimedia = 2048 shares
browser    = 1024 shares

means a relative proportion of:

2 : 1

between those groups while competing for CPU.

There is no need to go deeper into the differences between v1 and v2 yet.

At this stage all that matters is:

cgroup v1 -> cpu.shares
cgroup v2 -> cpu.weight

Both describe a group's relative share of the CPU.

Mental model

Without cgroups we can think:

scheduler
   |
   +-- task A
   +-- task B
   +-- task C
   +-- task D

With CPU cgroups:

scheduler
   |
   +-- group A
   |     |
   |     +-- task A1
   |     +-- task A2
   |
   +-- group B
         |
         +-- task B1
         +-- task B2
         +-- task B3

The scheduler still executes concrete tasks.

But cgroups let us tell the kernel:

before you start looking at individual processes, treat them as groups and take the rules assigned to those groups into account.

For CPU, one of the most basic such rules is relative weight.

The most important things to remember:

cgroup = a group of processes

cpu controller = CPU rules for that group

cpu.weight = the group's relative weight

weight != limit

weight matters under competition

the split is hierarchical

That is enough for this stage.

We have not capped maximum CPU yet, we have not introduced quota, and we are not talking about Kubernetes yet.

We simply taught the kernel to look at many processes as one group.