CPU Contention: when a process wants CPU and does not get it

The simplest case

We have one logical CPU:

CPU 0

and one CPU-bound task:

A

The task always has work.

So the scheduler can execute it practically all the time:

CPU 0 -> A

We have:

demand   = 1 CPU
capacity = 1 CPU

There is no contention.

Let us add a second identical task:

A
B

Both would like to execute all the time.

Their combined demand is:

2 CPU

but the available capacity is still:

1 CPU

At any given moment the CPU can execute only one of them:

CPU 0 -> A

B -> runnable, waiting

A moment later the scheduler may switch:

CPU 0 -> B

A -> runnable, waiting

And so on:

A -> B -> A -> B -> A -> B

That is the basic form of CPU contention.

Contention starts at demand > capacity

The simplest model:

CPU demand <= CPU capacity
    |
    v
no need to wait for CPU


CPU demand > CPU capacity
    |
    v
some runnable tasks have to wait

On a machine with four logical CPUs:

capacity = 4 CPU

If we have four constantly runnable single-threaded tasks:

A
B
C
D

all of them can execute in parallel:

CPU 0 -> A
CPU 1 -> B
CPU 2 -> C
CPU 3 -> D

Let us add:

E

Now we have:

5 runnable tasks
4 CPU

At least one task has to wait at any moment.

Not because it is throttled.

Not because it is sleeping.

It is ready to execute.

There simply is no CPU for it.

A runnable task may be executing nothing

This is the most important thing about contention.

A task in the runnable state tells the scheduler:

I am ready,
give me a CPU

But runnable does not mean:

I am currently executing instructions

Example:

time -------------------------------------->

task A:
RUNNING | RUNNING | RUNNING | RUNNING

task B:
WAIT    | WAIT    | RUNNING | WAIT

During the WAIT moments task B may still be runnable.

It is not waiting on:

disk
network
a timer
an application lock

It is waiting purely for the scheduler.

That is the time lost to CPU contention.

Runqueue

Runnable tasks have to be represented by the scheduler somewhere.

Simplified, you can think of a runqueue:

CPU 0

running:
    A

runnable:
    B
    C
    D

The more runnable work there is per unit of available capacity, the more time tasks may spend waiting.

And it is not only about the number of processes.

A single process may have:

32 runnable threads

and generate large CPU demand on its own.

From the scheduler's point of view what matters is tasks that are ready to execute.

CPU usage does not show the whole demand

Assume an application that, with no constraints, could use:

2 CPU

The node is overloaded, though, and the scheduler can give it on average:

1 CPU

Monitoring will show roughly:

CPU usage = 1 CPU

But that does not mean the application only needed one CPU.

The real situation may have been:

CPU demand = 2 CPU
CPU received = 1 CPU
CPU waiting = 1 CPU worth of work

Of course "waiting = 1 CPU" is a mental model here, not a direct kernel counter.

What matters most is the distinction:

usage = the CPU time the workload got

demand = the CPU time the workload wanted

If the scheduler did not run the task, the work it did not do will never appear as CPU usage.

An example from a Kubernetes node

Assume a node:

8 CPU

The running workloads generate in total:

4 CPU demand

We have plenty of free capacity:

demand   = 4
capacity = 8

There is no global contention.

Now a peak arrives:

Workload A wants 3 CPU
Workload B wants 3 CPU
Workload C wants 4 CPU
Workload D wants 2 CPU

In total:

demand = 12 CPU

on:

capacity = 8 CPU

We are short of:

4 CPU

That does not mean Linux suddenly "creates" extra CPUs.

It means:

some runnable work will wait

The scheduler will distribute the eight available CPUs among the competing tasks according to its own rules and the cgroup weights.

Contention does not require a limit

This is very important after the previous article.

Assume:

resources:
  requests:
    cpu: "1"

with no:

limits:
  cpu:

So the workload has no CPU bandwidth limit.

In cgroup v2:

cpu.max = max ...

It therefore cannot be throttled for exhausting its own quota.

But it can still wait for CPU.

If the node has:

8 CPU

and the runnable workloads want a total of:

16 CPU

then physically all that work cannot be executed at once.

No limit does not mean:

I will always get as much CPU as I want

It only means:

I have no hard bandwidth ceiling of my own

Throttling and contention look similar from the application's side

In both cases a task may have work and not be executing instructions.

Throttling

task runnable
     |
     v
CPU may be available
     |
     v
quota exhausted
     |
     v
task cannot execute

Contention

task runnable
     |
     v
quota available / no limit
     |
     v
the CPU is executing other tasks
     |
     v
the task waits

The application-level effect may be similar:

higher latency
lower throughput

But the cause is completely different.

More threads do not create more CPU

Assume an application on a node with:

4 CPU

The application creates:

4 worker threads

If nothing else competes, all of them can execute in parallel:

T1 -> CPU 0
T2 -> CPU 1
T3 -> CPU 2
T4 -> CPU 3

Adding more:

T5
T6
T7
T8

does not double the physical capacity.

Now eight threads compete for four CPUs.

At any given moment:

4 -> running
4 -> runnable and waiting

simplified, of course, because the scheduler keeps switching tasks.

Higher concurrency can therefore increase:

runnable pressure

without increasing the available compute power.

Scheduler latency

Assume a task that becomes runnable at:

t = 0 ms

but the scheduler only starts it at:

t = 8 ms

We have:

0 ms                8 ms
|--------------------|
      runnable
       waiting

And only later:

8 ms       10 ms
|-----------|
  running

The task needed:

2 ms CPU time

but the operation took at least:

10 ms wall-clock

of which:

8 ms

was waiting for a chance to execute.

In practice we call that time scheduler latency or runqueue latency.

And it is one of the most direct effects of CPU contention.

Why contention hurts latency-sensitive workloads in particular

A batch workload can get less CPU and simply finish later.

For a request-response service the situation is worse.

Assume an HTTP request that needs:

3 ms CPU time

Without contention:

~3 ms execution

On an overloaded node:

5 ms waiting
3 ms running
4 ms waiting

we get:

12 ms

even though the application still used only:

3 ms CPU time

The CPU work did not change.

What changed is when the process was allowed to do it.

That is why contention shows up in latency very quickly.

Mental model

CPU contention comes down to a single condition:

runnable CPU demand
>
available CPU capacity

Then:

the task has work
      |
      v
runnable
      |
      v
the CPU is taken by the competition
      |
      v
the task waits
      |
      v
the scheduler eventually picks the task
      |
      v
running

The key distinction against the previous article:

THROTTLING:
you cannot use the CPU
because you exhausted your quota

CONTENTION:
you may use the CPU,
but others want it right now too

In both cases the application may stop doing work.

Only the reason is different.