Requests, Limits and QoS: how Kubernetes classifies Pods

BestEffort

The simplest case:

containers:
- name: app
  image: example/app

No:

requests.cpu
limits.cpu
requests.memory
limits.memory

The Pod gets the class:

BestEffort

For a Pod to be BestEffort, none of its containers may have a CPU or memory request or limit.

That does not mean:

the Pod cannot use CPU

Quite the opposite.

If the node has spare compute power, BestEffort can use it.

For example:

node = 16 CPU

other workloads use = 4 CPU

BestEffort workload wants = 8 CPU

it may really get those 8 CPU if nothing else gets in its way.

So BestEffort does not mean:

little CPU

It rather means:

no declared CPU request
no CPU limit

Burstable

Now let us set:

resources:
  requests:
    cpu: 500m

There is no limit.

The Pod does not meet the criteria for Guaranteed, but it does have a resource configuration.

So it lands in:

Burstable

A Pod goes into the Burstable class when it:

is not Guaranteed

and has at least one CPU or memory request or limit.

Example:

resources:
  requests:
    cpu: 500m
  limits:
    cpu: "2"

will also be a typical Burstable case, assuming the rest of the Pod's configuration does not meet the conditions for Guaranteed.

In a CPU context that means we can have:

request = 500m
limit   = 2 CPU

that is:

scheduler accounting = 0.5 CPU

relative CPU weight = derived from 500m

maximum bandwidth = 2 CPU

QoS does not replace requests and limits.

It is a classification built on top of them.

Guaranteed

Now this configuration:

resources:
  requests:
    cpu: "1"
    memory: 1Gi
  limits:
    cpu: "1"
    memory: 1Gi

If every container in the Pod has positive CPU and memory requests and limits, and for each resource:

request == limit

the Pod gets the class:

Guaranteed

So for CPU:

requests.cpu = limits.cpu

but that alone is not enough.

Memory has to meet the matching conditions too.

That is an important detail.

This configuration:

resources:
  requests:
    cpu: "1"
  limits:
    cpu: "1"

does not automatically mean:

Guaranteed

if we did not also satisfy the memory requirements.

QoS is a classification of the whole Pod, not of CPU alone.

Guaranteed does not mean dedicated CPU

The name may suggest:

Guaranteed
=
I have a guaranteed physical core

No.

A Pod with:

resources:
  requests:
    cpu: "1"
    memory: 1Gi
  limits:
    cpu: "1"
    memory: 1Gi

may be Guaranteed and still run on shared CPUs.

From the point of view of the mechanisms we already know:

request = 1 CPU
limit   = 1 CPU

means:

the scheduler accounts for 1 CPU of capacity

the workload has a CPU weight derived from the request

the workload has a bandwidth limit matching 1 CPU

It does not mean:

CPU #7 belongs to this Pod alone

Exclusive CPUs require the additional CPU Manager mechanism and matching node configuration. Kubernetes documents that Guaranteed Pods may qualify for exclusive CPUs under the static policy, but the Guaranteed class alone does not create such an assignment.

Three Pods on one CPU

Assume one CPU and three workloads.

Pod A - BestEffort

resources: {}

Pod B - Burstable

resources:
  requests:
    cpu: 500m

Pod C - Guaranteed

resources:
  requests:
    cpu: "1"
    memory: 1Gi
  limits:
    cpu: "1"
    memory: 1Gi

You should not think:

Guaranteed always first
Burstable next
BestEffort last

The Linux scheduler does not interpret Kubernetes QoS names that way.

The runtime works with cgroup mechanisms.

Pod B has a CPU weight derived from its request:

500m

Pod C has a weight derived from:

1 CPU

and additionally has a CPU bandwidth limit of:

1 CPU

Pod A has no declared CPU request, so its position under contention will be correspondingly weaker, according to the cgroup configuration the kubelet generated.

This is not classic scheduler priority:

Guaranteed > Burstable > BestEffort

A QoS class on its own is not an equivalent of nice, of scheduler priority, or of a CPU priority queue.

Most importantly: QoS is a result of configuration

A good mental model:

requests + limits
       |
       v
Kubernetes computes the QoS class
       |
       v
Guaranteed / Burstable / BestEffort

Not the other way round.

We do not configure directly:

qosClass: Guaranteed

Kubernetes computes the class from the resource configuration.

You can see it for example with:

kubectl get pod <pod> -o jsonpath='{.status.qosClass}'

and get:

Guaranteed

or:

Burstable

or:

BestEffort

The class is assigned when the Pod is created and stays its class for the Pod's lifetime; current in-place resize mechanisms do not let a resource change move it to a different QoS class.

One trap: a limit with no explicit request

Assume:

resources:
  limits:
    cpu: "1"

with no:

requests:
  cpu:

You should not automatically assume:

CPU request = 0

If a request was not given, Kubernetes may set the request to the value of the limit. As a result the container can end up with:

request = 1 CPU
limit   = 1 CPU

That matters both for the scheduler and for determining QoS.

So when analysing a running Pod it is worth looking at the final resource configuration, not only at the fragment of the manifest you remember from the repository.

QoS does not change what a request or a limit means

This matters, so that we do not create another abstraction where there is none.

For Burstable:

requests:
  cpu: 500m
limits:
  cpu: "2"

still:

500m

is the request, and:

2 CPU

is the bandwidth limit.

For Guaranteed:

requests:
  cpu: "1"
limits:
  cpu: "1"

these are still the same two mechanisms.

Their values simply happen to be equal.

QoS does not create a new way of scheduling CPU.

Mental model

Put simply:

BestEffort

request = none
limit   = none

Burstable

any CPU/memory configuration
that does not satisfy Guaranteed

Guaranteed

for every container:
CPU request = CPU limit > 0
Memory request = Memory limit > 0

In a CPU context, though, what matters most is:

QoS class
!=
CPU priority class

and:

Guaranteed
!=
dedicated CPU

A request still governs declared capacity and relative weight.

A limit still governs maximum CPU bandwidth.

QoS is a Kubernetes classification layer built on those settings, not a third, independent mechanism for handing out CPU.