Revalidation and ISR: static output that expires

A cache needs a notion of freshness

Assume we generated a product page:

<h1>Mechanical Keyboard</h1>
<p>129 zł</p>

At 12:00 the price changes to 119 zł.

If we do nothing, the cache still returns:

129 zł

So we have to define when the result stops being fresh.

The simplest model:

generated at: 12:00
TTL: 60s

12:00:30 -> fresh
12:01:10 -> stale

From that moment the system knows the artifact should be refreshed.

The only question is: when?

Synchronous regeneration

The most obvious strategy:

request
  |
  v
cache stale?
  | yes
  v
render new version
  |
  v
store
  |
  v
return

That gives a fresh result, but the first request after expiry pays the full render cost.

So we have a problem similar to deferred generation:

most users -> fast

one unlucky user -> slow

In many systems we can do better.

Stale-while-revalidate

Instead of blocking the user on regeneration, we can return the old result and refresh it in the background.

request
  |
  v
cache stale?
  | yes
  v
return stale HTML
  +
start regeneration

The next request will already get the new version.

An example:

12:01:10
request A
-> gets 129 zł
-> triggers regeneration

12:01:11
new HTML generated: 119 zł

12:01:12
request B
-> gets 119 zł

That is the classic trade-off:

freshness vs latency.

We accept a short window of staleness in exchange for a fast response.

Where ISR fits in

Incremental Static Regeneration can be treated as one concrete implementation of that model.

Not as a magical rendering category, but as a combination of:

precomputed HTML
+
cache
+
expiration
+
regeneration

That matters, because it stops ISR from looking like yet another framework shortcut.

The mechanism is very old.

Similar strategies have existed for years in CDNs, reverse proxies and cache systems.

The frontend simply started applying them directly to rendering artifacts.

A commercial example: a product catalogue

Assume a large store.

A product description changes once every few days.

The price a few times a day.

Stock levels can change every few seconds.

If the whole document had a single TTL, we would have to pick a compromise:

TTL 24h
-> great cache
-> potentially a very old price

TTL 5s
-> a fresh price
-> constant regeneration of the whole page

That shows an important problem:

different data has different freshness requirements.

Treating the whole HTML as a single cache unit does not always make sense.

That problem will later lead to more granular rendering models.

A curiosity

TTL does not mean:

after 60 seconds the cache automatically generates a new page.

Most often it only means:

after 60 seconds the current entry should no longer be treated as fresh.

Regeneration can start only on the next request.

If a page gets no traffic for an hour, there is no reason to update it.

That is another example of lazy computation:

don't refresh until somebody cares

Thanks to that the cost of the system can scale with real traffic rather than with the theoretical number of pages.

Time-based revalidation is only one model

TTL is simple, but it has a downside.

If a product changed a second after regeneration, we may serve the old document for almost the entire TTL.

The alternative is event-driven invalidation.

CMS update
  |
  v
invalidate /products/42
  |
  v
next request regenerates

Or:

database update
  |
  v
webhook
  |
  v
purge cache

That lets us keep a long TTL, or even no classic expiration, and still react immediately to a data change.

The cost is more system complexity.

We have to know:

which artifacts depend on the changed data?

Invalidation quickly becomes a dependency graph problem

Assume product 42 appears on:

/products/42
/category/keyboards
/search?q=keyboard
/promotions/summer

Changing the product price may mean invalidating all of those pages.

So the relation is no longer:

data -> page

but:

data
├── page A
├── page B
├── page C
└── page D

In a larger application a dependency graph appears between data and cached artifacts.

That is why invalidation is harder than caching itself.

The cache knows what it stores.

It does not always know why the result looks the way it does.

What if many requests try to regenerate at once

Assume a popular product.

The cache has just expired.

At the same moment 500 requests arrive.

A naive implementation:

500 requests
  |
  v
500 regenerations
  |
  v
database cries

That is the classic cache stampede.

The system should usually let one instance perform the regeneration:

request A -> regeneration

request B ─┐
request C ─┼─> stale response
request D ─┘

or make the remaining requests wait for the same result.

It is the same concurrency problem we saw with deferred generation, except now it comes back periodically.

Revalidation changes how we think about rendering

At the start we had a simple choice:

render now

or:

render during build

Revalidation introduces a third model:

render
  |
  v
reuse
  |
  v
become stale
  |
  v
render again

Rendering becomes a process of periodically materialising a result.

And that is a far more interesting mental model than:

ISR is SSG refreshed every now and then.

What we really have is a cache system with controlled inconsistency.

But we are still waiting for the whole document

Even if we cache and regenerate HTML beautifully, another limitation remains.

Assume a page needs:

product          -> 20 ms
reviews          -> 80 ms
recommendations  -> 500 ms

If the server wants to produce a complete document before sending the response, the whole request can wait for the slowest fragment.

fast data ───────────┐
medium data ─────────┼─> wait -> HTML
slow data ───────────┘

And yet the user could already see the product name and price.

We do not have to wait for everything.

We can start sending the document before the render of the whole tree finishes.

That is how we arrive at Streaming SSR.