HTTP Caching: Cache-Control, ETag and Validation

By the end of this lesson you will be able to

• Explain why HTTP caching is the cheapest performance and scaling tool you have, and what a fresh hit, a 304 and a full 200 each cost.
• Tell private caches (the browser) from shared caches (proxies, gateways, CDNs) and say which responses each may store.
• Work out whether a stored response is fresh from max-age, s-maxage, Expires, Age and heuristic freshness.
• Choose the right Cache-Control directives, and explain exactly how no-cache, no-store, private, public, must-revalidate and immutable differ.
• Revalidate with ETags (strong and weak) and Last-Modified, and read every header of a 304.
• Design cache keys with Vary without falling into the Vary: User-Agent trap, and serve stale content on purpose with stale-while-revalidate and stale-if-error.
• Ship deploys that never strand users on old JavaScript, using hashed file names.
• Write a caching policy for HTML, static assets, public APIs and personal data, cache authenticated responses safely, and use ETag with If-Match to stop lost updates in an API.

#Where this lesson fits

Lesson 1.1 introduced Cache-Control, ETag and the 304 status in one line each. Lesson 1.8 showed the same idea in DNS: a TTL decides how long an answer may be reused, and “propagation” is just caches expiring. Lesson 1.12 and lesson 1.13 showed proxies and Nginx caching responses, and the public, immutable header on fingerprinted files. Lesson 2.2 explains which methods are cacheable and lesson 2.3 how headers and compression work.

This lesson is the full protocol: how any HTTP cache, from your browser to a CDN, decides whether it may store a response, how long it may reuse it, how it checks it again cheaply, and how to set the headers so users get speed without ever seeing the wrong data. What a CDN is inside, cache-aside with Redis, eviction policies and stampedes belong to Chapters 23 and 24; here we stay with what HTTP itself says.

#Why HTTP caching

HTTP caching means keeping a copy of a response close to the user, in the browser or in a proxy or CDN, and reusing it for later requests instead of asking the origin server again. Three outcomes are possible, and their costs are very different:

A browser on a 4G phone in Dhaka might wait 150 ms for each round trip to a server in Singapore. A page that reuses 40 of its 50 files from the cache simply does not make those 40 requests. For the business the effect is double: users see faster pages, and every hit is a request your servers and your bandwidth bill never see. If a CDN answers 95% of requests, the origin handles only 5%; raising the hit ratio from 90% to 95% halves the origin’s load.

The decision every cache makes is the one the notes draw:

                  [ Browser / client cache ]
                              |
                      (Is the copy fresh?)
                       /              \
                    (Yes)             (No: stale)
                     /                    \
       [ Use it: memory / disk ]   [ Conditional request ]
                                   (If-None-Match: ETag, or
                                    If-Modified-Since: date)
                                            |
                                   (Server validation)
                                    /               \
                       (304 Not Modified)      (200 OK + new body)
                       reuse the stored body   replace the copy

#Where caches live: private and shared

The HTTP standard (RFC 9111) recognises two kinds of cache, and almost every rule depends on which one you are talking about.

A private cache may store responses meant for one person, such as your profile page. A shared cache must never do that, because the next person through it would receive your page. That single rule explains private, public, s-maxage and most caching security bugs.

Two more facts about the browser’s cache:

• It uses memory and disk. Small, recently used responses are kept in memory for the current session; DevTools labels them “(memory cache)” or “(disk cache)”. The notes say files are kept “on the device’s disk”, which is only half of it.
• It is partitioned. Since 2020–2021, Chrome, Firefox and Safari key the cache by the top-level site as well as the URL, so a file cached while visiting a.example is not reused on b.example. This stops sites from probing your history through cache timing, and it means “everyone already has jQuery from the public CDN cached” is no longer true.

#What may be cached at all

• Methods: in practice only GET and HEAD responses are cached. A POST response is only reusable when it says so explicitly and is almost never cached (lesson 2.2 covers safe and cacheable methods).
• Status codes: with explicit freshness any response can be stored. Without it, only codes the standard calls heuristically cacheable may be stored: 200, 203, 204, 206, 300, 301, 308, 404, 405, 410, 414 and 501. So a 404 with no headers can be cached, and a 301 is cached hard by browsers.
• The default key is the method plus the full URL (scheme, host, path and query string). Two URLs that differ only in a tracking parameter are two different entries.

#Freshness: how long a copy may be reused

A stored response is fresh while its age is less than its freshness lifetime, and stale after that. A fresh copy is reused without contact; a stale copy is normally revalidated first. Stale does not mean wrong or deleted: the cache keeps it, because a cheap 304 may bring it back to life.

#max-age and Expires

Cache-Control: max-age=3600
Expires: Mon, 02 Nov 2026 12:00:00 GMT

• max-age=N gives the lifetime in seconds, counted from when the response was generated. max-age=3600 is one hour; max-age=31536000 (the notes’ example) is one year, the conventional “forever”.
• Expires gives an absolute date and time, always in GMT with the correct weekday. It is the HTTP/1.0 way and depends on the server’s and client’s clocks agreeing. An invalid value such as Expires: 0 means “already expired”.
• If both are present, max-age always wins, in every cache. (The notes say “usually”; the standard says always.)

Both describe freshness, not how long the copy is kept. A cache may delete a fresh entry early when it needs space, and it may keep a stale entry for a long time in the hope of revalidating it.

#s-maxage: a separate lifetime for shared caches

s-maxage=N applies only to shared caches and overrides max-age (and Expires) there. Browsers ignore it. The common pattern is a short browser lifetime and a longer CDN lifetime:

Cache-Control: max-age=60, s-maxage=600

Why? You can purge a CDN in seconds after a change, but you can never reach into users’ browsers. So the copy you can delete gets the long lifetime. Many CDNs also accept their own headers for this, such as CDN-Cache-Control (a standard since 2022) or Surrogate-Control, which they read and strip; Chapter 24 covers them.

#Age: time already spent in other caches

When a shared cache serves a stored response it adds an Age header: how many seconds since the response was generated or last validated by the origin. The next cache counts that time too:

current age = time the copy has been in this cache + the Age it arrived with

So a response with max-age=60 that arrives from a CDN with Age: 40 has only 20 seconds of freshness left in the browser. If it arrives with Age: 300, it is stale on arrival and the browser will revalidate every time it is used: a surprising cost of combining a short max-age with a long s-maxage.

#Heuristic freshness: when the server says nothing

If a response has no max-age, s-maxage or Expires but has a Last-Modified date, the standard allows caches to guess a lifetime. The usual rule is 10% of the time since the last modification: a file last changed 10 days ago is treated as fresh for about a day. That is how a forgotten header turns into “users still see the old file a day after the fix”. Always send an explicit Cache-Control, even if it is only no-cache.

#Cache-Control directives

Cache-Control is a comma-separated list of directives, mostly sent on responses:

Requests can carry Cache-Control too. A hard reload sends Cache-Control: no-cache to force a full fetch; max-age=0 asks for revalidation; only-if-cached asks for a stored copy or nothing. The old Pragma: no-cache header is an HTTP/1.0 leftover that the current standard deprecates; you will still see browsers send it alongside Cache-Control for compatibility.

#Validation: ETag and Last-Modified

When a stored response is stale, the cache does not throw it away. It sends a conditional request that says “here is what I have; send a new body only if it changed”. To make that possible the original response must carry a validator.

#ETag and If-None-Match

An ETag (entity tag) is an opaque label for one version of a resource:

HTTP/1.1 200 OK
Cache-Control: max-age=3600
ETag: "hash_v123"

Later the cache asks:

GET /logo.png HTTP/1.1
If-None-Match: "hash_v123"

If the current ETag is still "hash_v123", the server answers 304 Not Modified with headers but no body. Otherwise it answers 200 OK with the new body and a new ETag. Either way only one round trip is spent.

What makes a good ETag:

• Any string that changes whenever the body changes. A hash of the content is common, as the notes say, but a database row version or an update counter ("v42") works just as well and is cheaper than hashing a large response.
• The same on every server. Behind a load balancer, three servers must produce the same ETag for the same file. Old Apache defaults mixed in the file’s inode number, which differs per machine, so every request landing on another server got a full 200 instead of a 304.
• Strong or weak. A plain "abc" is a strong validator: byte-for-byte identical. W/"abc" is a weak one: “semantically the same”, for example the same page with a different timestamp in a comment. If-None-Match compares weakly, so both work for caching, but range requests and If-Match need strong ETags. Nginx turns a strong ETag into a weak one when it compresses the response on the fly, because the bytes are no longer identical.

#Last-Modified and If-Modified-Since

The older, time-based validator:

Last-Modified: Wed, 21 Oct 2026 07:28:00 GMT
...
If-Modified-Since: Wed, 21 Oct 2026 07:28:00 GMT

The server returns 304 if the resource has not changed since that moment. It works without any extra state, but it has one-second resolution: two changes in the same second, a deploy that restores old file times, or servers whose file times differ all make it lie. If a request carries both If-None-Match and If-Modified-Since, the server must use the ETag. Send ETags; keep Last-Modified as a fallback (it also feeds heuristic freshness and is useful to crawlers).

#What a 304 really says

A 304 does not mean “your copy was fresh”: a fresh copy would not have been checked at all. It means “your stale copy is still correct”. The 304 has no body, but it does carry headers (ETag, Cache-Control, Expires, Vary, Date), and the cache updates the stored headers from it and restarts the freshness clock. That is how you can change the lifetime of cached files without resending them.

The conditional headers, all in one place:

#Cache keys and Vary

A cache files each response under a key, by default the method and the full URL. But one URL can have several correct answers: a Brotli-compressed and an uncompressed app.js, an English and a Bengali page. The Vary response header lists the request headers that must also match before a stored response can be reused:

Vary: Accept-Encoding

Now the cache stores one entry per distinct Accept-Encoding value and gives each client the body it can read.

• Vary: Accept-Encoding is the one almost every compressed response needs (lesson 2.3 covers gzip, Brotli and zstd). Forgetting it lets a shared cache hand Brotli bytes to a client that never asked for them. CDNs normalise this header to a few values so it does not create many variants.
• Vary: User-Agent is a trap. There are thousands of distinct User-Agent strings, so the cache splits into thousands of tiny entries and the hit ratio collapses. If you must serve different HTML to phones, let the CDN map the User-Agent to a device class (or use the Sec-CH-UA-Mobile client hint) and vary on that, or better, make one responsive page.
• Vary: Cookie makes the cache keep one copy per cookie value, which for logged-in users means one copy per user: correct, but nearly useless in a shared cache. Mark such responses private instead.
• Vary: Accept-Language is fine if the CDN normalises it to the languages you support; otherwise every browser’s slightly different language list is a new variant.
• Vary: * means “never reuse this”, which is a clumsy way to write no-store.

Most CDNs also let you shape the key directly: drop tracking parameters (utm_*, fbclid), sort the query string, or add a header or cookie to the key. That is CDN configuration, taught in Chapter 24; the HTTP-level tool is Vary.

#Serving stale on purpose

Two directives (RFC 5861) let a cache use a stale copy deliberately, trading a little freshness for speed or availability:

Cache-Control: max-age=60, stale-while-revalidate=300, stale-if-error=86400

• stale-while-revalidate=300: for 300 seconds after the copy goes stale, the cache may answer immediately with the stale copy and refresh it in the background. Users never wait for the origin’s render time; the cache pays it once. Chromium browsers and Firefox support it, and so do the major CDNs.
• stale-if-error=86400: if revalidation fails (a 500, 502, 503 or 504, or no answer), the cache may keep serving the stale copy for up to a day. Your home page stays up while the database is down. CDNs such as Cloudflare, Fastly and CloudFront honour it; most browsers do not, so treat it as a shared-cache tool.
• must-revalidate does the opposite: it forbids serving stale even when the origin is unreachable. Use it where a wrong answer is worse than an error (a stock level used to take payment, a feature flag that disables a dangerous path).

The Age header on such a response tells you how old it really is. Stale-while-revalidate also softens the cache stampede, where many requests for one expired key hit the origin at once; the full treatment, with locks and request coalescing, is in Chapter 23.

#Cache busting with hashed file names

Long lifetimes and correct updates seem to conflict: cache a file for a year and users cannot get a new version; cache it for a minute and you lose most of the benefit. The answer is to never change a file under the same URL:

1. The build tool names each static file after a hash of its content: app.a83f92.js, style.72bc1.css. Change one line and the name changes.
2. Those files are served with Cache-Control: public, max-age=31536000, immutable: cached for a year and never revalidated, because a given name can never have different content.
3. The HTML is short-lived: no-cache (with an ETag), or a few seconds to minutes in the CDN with a purge on deploy. The HTML is the only thing that changes, and it points to the new file names.
4. On a deploy the browser revalidates the HTML, sees new names, and fetches only the files that actually changed. Unchanged files keep their names and stay cached.
5. Keep old files online for days after a deploy. Users with an open tab, and HTML still cached somewhere, still ask for them.

With a fixed name like /app.js and a long lifetime, a deploy produces new HTML with old JavaScript in returning users’ browsers, and nothing you do on the server can fix it until their copy expires. Query-string versions (app.js?v=42) work in browsers but some proxies refuse to cache URLs with query strings, and they are easy to forget to bump; content hashes in the file name are the robust choice. immutable matters mainly in Firefox and Safari: since 2017 Chrome no longer revalidates subresources on a normal reload, so it ignores the directive.

#What to cache: a policy per kind of response

There is no single right header; there is a right header per kind of response. A good default set:

Think about who is allowed to see the response first (decides private, public, no-store), then how stale it may be (decides lifetimes and stale-*), then how it is updated (fixed URL with revalidation, or hashed name).

#Caching authenticated and personalised responses safely

The worst caching bug is not a slow page; it is a page delivered to the wrong person. The rules:

1. Responses that depend on who is asking are private (or no-store). Never rely on a CDN guessing that a page is personal.
2. A request with an Authorization header may not be stored by a shared cache unless the response explicitly allows it with public, s-maxage or must-revalidate. Do not add those to personal responses.
3. Never let a shared cache store a response with Set-Cookie. The HTTP standard does not forbid it, but if a cached response carries one user’s session cookie, everyone who gets that copy is logged in as that user. Nginx and most CDNs refuse to cache such responses by default; “cache everything” rules and custom proxies often do not.
4. Do not cache by file extension alone. Web cache deception tricks a CDN into storing /account/x.css, which a lenient router renders as the account page.
5. Split the page. Cache the public shell (layout, product data) in the CDN and fetch the personal parts (name, cart count) with a separate private API call. That keeps the hit ratio high without risk.
6. On logout, Clear-Site-Data: "cache" asks the browser to drop its cached data for the site, and no-store on sensitive pages stops the back button from showing them afterwards on a shared computer.

#Common caching bugs

#ETag beyond caching: If-Match and lost updates

The same ETag that makes 304 possible also solves a classic API problem. Two people load the same record, both edit it, both save: the second save silently overwrites the first. This is the lost update.

With optimistic concurrency the client sends back the ETag it loaded:

PUT /products/7 HTTP/1.1
If-Match: "7"
Content-Type: application/json

{"price": 120, "title": "Mug"}

• If the current version is still "7", the server applies the change and returns a new ETag.
• If someone else changed it in the meantime, the server returns 412 Precondition Failed and changes nothing. The client reloads, shows the user the new state, and retries.
• A server that wants to forbid blind overwrites returns 428 Precondition Required for writes without If-Match.
• If-None-Match: * on a PUT means “create only if it does not exist”, a safe way to claim a name or write an object once; Amazon S3 supports both conditional writes.

On the server the check and the write must be one atomic step, typically UPDATE … WHERE id = 7 AND version = 7 and checking that one row changed. No lock is held while the user thinks, which is why it is called optimistic. API design with these headers continues in Chapter 26, and transaction isolation in the database chapters.

#Other browser caches, briefly

• Back/forward cache (bfcache). When you press back, browsers can restore the previous page from memory, complete with its JavaScript state, instantly. This is separate from the HTTP cache. Pages that use unload handlers, or keep a WebSocket open, may be excluded, and pages sent with Cache-Control: no-store have long been excluded too (Chrome has started allowing some of them). If back navigation feels slow, Chrome DevTools has a bfcache test under Application.
• Service worker cache. A site’s own script can intercept requests and answer from the Cache API, which is how offline web apps and “network first, fall back to cache” strategies work. It does whatever the script says, so its bugs are your bugs.
• Memory cache and preload. Resources fetched with <link rel="preload"> or already used on the current page are reused from memory, even for a moment after they would normally expire.

#HTTP caching in system design

• Cache at the edge first. Every request a browser or CDN answers never reaches your load balancer, app servers or database. Static assets at a 99% hit ratio are the cheapest scaling win there is.
• Hit ratio is the metric. Origin load = total requests × (1 − hit ratio). Watch it per path; Vary, cookies and query strings are the usual reasons it is low.
• Short in browsers, longer in the CDN, purge on change. Browsers cannot be purged, CDNs can. s-maxage and the CDN-specific headers exist for this.
• Design APIs to be cacheable. Separate public reads from personal ones (/products/7 vs /me/cart), return ETags, and use GET for reads so caches can help (lesson 2.2).
• Choose availability on purpose. stale-while-revalidate hides slow renders, stale-if-error hides outages, must-revalidate refuses both. Decide per endpoint.
• Caching is also a security boundary. Any shared cache in front of personal data must be configured as carefully as an access-control list.
• Interview angles: “How do you make sure users get the new JavaScript after a deploy?” (hashed names, short HTML). “no-cache vs no-store?” “Why is our CDN hit ratio low?” (Vary, cookies, query strings). “How would you stop two admins overwriting each other?” (ETag + If-Match, 412). The cache-aside, write-through and eviction questions are Chapter 23.

#Try it yourself

# See the caching headers of any URL (HEAD request)
curl -sI https://www.example.com/ | grep -iE 'cache-control|etag|last-modified|expires|age|vary'

# Revalidate by hand: copy the ETag from above and send it back
curl -sI https://www.example.com/ -H 'If-None-Match: "84238dfc8092e5d9c0dac8ef93371a07"'
#   HTTP/2 304 means your copy is still good

# Same with a date
curl -sI https://www.example.com/ -H 'If-Modified-Since: Mon, 02 Nov 2026 00:00:00 GMT'

# Is a CDN serving from cache? Look for age, x-cache or cf-cache-status (hit / miss / expired / revalidated)
curl -sI https://www.cloudflare.com/ | grep -iE 'cf-cache-status|age|cache-control'

# Ask for Brotli, then for nothing, and compare content-encoding and vary
curl -sI https://www.example.com/ -H 'Accept-Encoding: br'
curl -sI https://www.example.com/

# Optimistic concurrency against your own API
curl -i -X PUT https://api.example.com/products/7 -H 'If-Match: "7"' \
  -H 'Content-Type: application/json' -d '{"price":120}'
#   412 Precondition Failed if someone changed it first

Setting the policy in Nginx (lesson 1.13 shows the full config):

location /assets/ {                       # hashed files
    add_header Cache-Control "public, max-age=31536000, immutable";
}
location = /index.html {                  # the page that names them
    add_header Cache-Control "no-cache";
    etag on;
}
location /api/me {                        # personal
    add_header Cache-Control "private, no-cache";
    proxy_pass http://app;
}

In the browser, open DevTools → Network, reload, and look at the Size column: “(memory cache)” and “(disk cache)” are fresh hits, a few hundred bytes with status 304 is a revalidation. Tick Disable cache to see the page as a first-time visitor, and compare a normal reload with a hard reload (Ctrl+Shift+R). Application → Back/forward cache tests whether a page is eligible for bfcache.

#Summary

Key takeaways

• HTTP caching reuses responses near the user. A fresh hit costs nothing, a 304 costs one small round trip, a 200 costs the full body. Raising a CDN hit ratio from 90% to 95% halves origin load.
• Private caches (the browser) may store one user’s data; shared caches (proxies, gateways, CDNs) must never. The browser cache uses memory and disk and is partitioned by site.
• A copy is fresh while its age is below its lifetime: s-maxage (shared only) beats max-age, which always beats Expires; Age counts time already spent upstream; with no lifetime caches may guess 10% of the time since Last-Modified.
• no-cache = store but check every time; no-store = never store; private = browser only; public = rarely needed; must-revalidate = never stale, error instead; immutable = never revalidate.
• Validation: ETag + If-None-Match (preferred; strong or W/ weak; identical on every server) or Last-Modified + If-Modified-Since (one-second resolution) → 304 Not Modified with updated headers and no body.
• Vary adds request headers to the cache key: always Accept-Encoding for compressed responses; never raw User-Agent or Cookie in a shared cache.
• stale-while-revalidate serves instantly while refreshing; stale-if-error keeps a site up through an outage (mostly in CDNs).
• Hashed file names with max-age=31536000, immutable, plus short-lived HTML, make deploys safe; keep old assets online for days.
• Personal responses: private or no-store, never Set-Cookie in a shared cache, never cache by extension alone.
• ETag + If-Match turns lost updates into 412 Precondition Failed; 428 demands a precondition; If-None-Match: * creates only once.

Self-check: HTTP caching31 questions