Networking in Kubernetes

The Kubernetes network model

Kubernetes assumes that every Pod can have its own IP address.

This matters, because it lets Pods talk to each other without manually mapping ports at the host level.

In simplified form:

Pod A: 10.0.1.10
Pod B: 10.0.2.15

Pod A can send traffic to Pod B.

How that traffic actually travels through the network depends on the implementation.

CNI: Container Network Interface

Kubernetes doesn't implement the whole network by itself.

Instead it relies on the CNI (Container Network Interface) standard.

CNI defines how the runtime and the cluster integrate with a network plugin.

Popular options are:

• Calico,
• Cilium,
• Flannel.

A CNI plugin can be responsible, among other things, for:

• assigning an IP address,
• creating interfaces,
• routing,
• configuring routes,
• network policies.

The scope depends on the specific solution.

Why CNI exists

Just as CRI separates Kubernetes from a specific container runtime, CNI separates Kubernetes from a specific network implementation.

This is an important architectural pattern.

Kubernetes states:

a Pod should have a working network

but doesn't impose a single way to achieve it.

Service and the virtual address

Pods get created and deleted.

Their IP addresses can change.

That makes connecting an application directly to a specific Pod IP inconvenient.

Kubernetes introduces the Service object, which provides a stable access point to a group of endpoints.

For example:

Service IP
   |
   v
Pod A
Pod B
Pod C

The Service IP is a logical address. Something still has to make traffic sent to that address actually reach one of the Pods.

kube-proxy

Classically this role is played by kube-proxy.

The process runs on the node and configures the operating system's networking so that traffic to a Service is directed to the right endpoints.

Depending on the mode and environment, it can use mechanisms such as:

• iptables,
• previously also IPVS.

Important:

kube-proxy usually doesn't act like a classic application proxy that every packet passes through.

Often its job is to set up rules in the kernel.

eBPF and the newer datapath

In solutions such as Cilium, some functions can be implemented with eBPF.

In that model the classic kube-proxy can be replaced with its own datapath.

For architecture basics, though, what matters is not which technology is used, but which function has to be fulfilled:

Service IP
-> has to be translated
-> into the right endpoint

CNI vs kube-proxy

This is a very common source of confusion.

Put simply:

CNI
-> provides Pod connectivity

kube-proxy / eBPF datapath
-> implements Service behavior

They are not two interchangeable components.

They touch the same domain, networking, but solve different problems.

What to remember

Kubernetes networking can be reduced to three questions:

1. How does a Pod get an IP?
2. How does a Pod talk to another Pod?
3. How does traffic to a Service reach the right endpoint?

CNI mostly answers the first two.

kube-proxy or an alternative datapath answers the third.