Migrating from ingress-nginx to Gateway API

Starting point

A simple Ingress:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: web
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "20m"
spec:
  ingressClassName: nginx
  tls:
    - hosts:
        - example.com
      secretName: example-com-tls
  rules:
    - host: example.com
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: web
                port:
                  number: 80

Some of this is standard:

host
path
backend Service
TLS secret

Some of it is specific to ingress-nginx:

ssl-redirect
proxy-body-size

If we only rewrite the structure, we can lose behavior.

If we carry everything over one to one, we can preserve behavior we no longer want.

New model

In Gateway API, the traffic entry point and the HTTP route are separate resources.

Gateway:

apiVersion: gateway.networking.k8s.io/v1
kind: Gateway
metadata:
  name: public
  namespace: platform
spec:
  gatewayClassName: public
  listeners:
    - name: https
      hostname: example.com
      port: 443
      protocol: HTTPS
      tls:
        certificateRefs:
          - name: example-com-tls

HTTPRoute:

apiVersion: gateway.networking.k8s.io/v1
kind: HTTPRoute
metadata:
  name: web
spec:
  parentRefs:
    - name: public
      namespace: platform
  hostnames:
    - example.com
  rules:
    - matches:
        - path:
            type: PathPrefix
            value: /
      backendRefs:
        - name: web
          port: 80

This is not a perfect equivalent for every Ingress.

It is a new split of responsibilities.

The platform can keep Gateway in the platform namespace.

The application can keep HTTPRoute next to its Service.

What to check before migration

Start by inventorying annotations.

Not the number of Ingress objects.

Annotations.

This list usually decides how hard the migration will be:

rewrite-target
use-regex
configuration-snippet
server-snippet
auth-url
auth-signin
proxy-read-timeout
proxy-send-timeout
proxy-body-size
ssl-redirect
backend-protocol
canary

Mark each item as one of three categories:

standard Gateway API
implementation extension
remove or redesign

The third category matters the most.

A migration is a good moment to discover configuration that appeared by accident and no one can explain anymore.

ingress2gateway as a starting point

ingress2gateway can generate Gateway and HTTPRoute from existing Ingress resources.

That is useful.

But read the output as an audit report, not as a production-ready manifest.

Warnings matter most for things that do not map cleanly:

configuration-snippet
proxy-body-size
timeouts
regex path matching
URL normalization

If the tool does not migrate something, that is not necessarily a tool bug.

Sometimes it tells you the old behavior depended on a specific controller.

Migrating next to the old controller

Gateway API can run next to ingress-nginx.

That should be the default plan.

ingress-nginx keeps running
the new gateway controller runs next to it
the new Gateway gets its own address
routes are tested without changing production DNS

Check first:

TLS
HTTP -> HTTPS redirect
Host header
path matching
rewrite
timeouts
large request body
headers
metrics
logs
traces

Move traffic only after that.

The simplest rollback is one that does not require recreating the old controller.

The old controller should still be alive when DNS or load balancer traffic is moved.

What not to change silently

Do not change path matching semantics without a test.

Do not assume regex works the same way.

Do not assume timeouts mean the same thing.

Do not carry configuration-snippet into a random extension only because there is somewhere to paste it.

Do not remove a body size limit without checking that the application and proxy defaults agree.

These are small differences until they hit a production request.

Definition of done

The migration is not done when kubectl apply accepts the new manifests.

It is done when you can say:

we know which GatewayClass handles traffic
we know which HTTPRoutes replaced the old Ingresses
we know which annotations became standard API
we know which behaviors are implementation extensions
we know what we intentionally did not migrate
we know how to roll back

Only then is removing ingress-nginx cleanup, not faith.