Your servers are in Mumbai. A user in São Paulo requests a 2MB image.
That request travels roughly 14,000km each way. Light in fibre covers about 200,000 km/s, so the round trip has a physical floor of around 140ms — before your server does any work at all. Add TCP and TLS handshakes, each needing their own round trips, and you are at half a second before the first byte of image data moves.
No amount of optimisation fixes this. You cannot make light faster.
The only way to win is to not make the journey. Put a copy of the image in São Paulo.
What a CDN is
A CDN is a network of servers — edge locations or points of presence — distributed across the world, holding copies of your content. Users are routed to the nearest one, usually by DNS or anycast routing.
- Users are routed to a nearby edge cache.
- A cache miss at the edge falls back to the origin, once per cold object.
Describe this diagram
Components
- users (Client)
- edge (Content delivery network)
- origin (Service)
Connections
- users sends to edge
- edge sends to origin
The first Brazilian user to request the image causes the edge to fetch it from your origin once. Every subsequent Brazilian user is served locally in perhaps 15ms instead of 500ms.
The benefits beyond speed
Origin offload is the one people underrate. If 95% of requests are served at the edge, your origin sees 5% of the traffic. That is not a small optimisation — it is the difference between needing fifty servers and needing three.
Absorbing traffic spikes. A product launch or a viral post hits hundreds of edge servers, not your origin. CDNs are the cheapest form of scaling available.
DDoS resilience. A distributed attack is spread across the CDN's global capacity, which is vastly larger than yours.
TLS termination at the edge. The expensive handshake happens near the user, cutting several round trips.
What to cache, and what not to
The natural fit is content that is identical for everyone and changes rarely: images, video, CSS, JavaScript, fonts, downloads.
The instinct is that anything user-specific cannot be cached. Modern CDNs are more capable than that:
Static assets — cache aggressively, for a year, using the versioning trick below.
Public API responses — a product catalogue or public profile can be cached for seconds or minutes. Even a 10-second TTL absorbs an enormous share of traffic during a spike.
Personalised HTML — generally not cacheable, but the page shell often is, with user-specific parts loaded separately.
Anything authenticated or private — do not cache, and be careful. A misconfigured cache key that serves one user's account page to another is a serious, and genuinely common, incident.
Cache keys: where the bugs live
The cache key decides whether two requests count as "the same request". By default it is the URL — which is why the private-data leak above happens: two users requesting /account have the same URL.
The Vary header tells the CDN which request headers also form part of the key:
Vary: Accept-Encoding, Accept-LanguageNow Brazilian Portuguese and English versions are cached separately.
Invalidation, and the trick that avoids it
"There are only two hard things in computer science: cache invalidation and naming things."
You cache logo.png for a year. You change the logo. Users see the old one for a year.
Two options.
Purge, telling the CDN to drop the object. It works, but takes seconds to minutes to propagate globally, and heavy purging is a sign of an underlying design problem.
Versioned URLs, which sidestep invalidation entirely. Include a content hash in the filename:
<script src="/static/app.a3f9c2e1.js"></script>Change the file, the hash changes, the URL changes, and it is a different object the CDN has never seen. No invalidation needed. Every build tool does this automatically, and it is why you can set a one-year TTL with total confidence.
The general principle is worth remembering: immutable content is trivially cacheable. Make things immutable and give them new names when they change.
For the HTML that references those assets, use a short TTL or no-cache, so users pick up new asset URLs promptly.
Cache headers you actually need
Cache-Control: public, max-age=31536000, immutable # hashed assets
Cache-Control: public, max-age=0, s-maxage=60 # API: browser no, CDN 60s
Cache-Control: private, no-store # never cacheTwo distinctions carry most of the weight:
public versus private. private means a browser may cache it but shared caches must not. Getting this wrong on authenticated pages is the leak described above.
max-age versus s-maxage. max-age applies to browsers, s-maxage to shared caches like the CDN. This pair lets you keep browsers fresh while still offloading your origin.
Also worth knowing: stale-while-revalidate lets the edge serve a slightly stale copy instantly while fetching a fresh one in the background. Users get speed; you get freshness shortly after.
The thundering herd
A subtle failure mode. A popular object's TTL expires. A thousand requests arrive at that edge in the same second, all miss, and all forward to your origin simultaneously — which promptly falls over.
The mitigations are worth naming in an interview:
- Request coalescing — the edge sends one request to origin and holds the others until it returns. Most good CDNs do this by default.
- Stale-while-revalidate — serve the old copy while one request refreshes.
- Jittered TTLs — vary expiry slightly so many objects do not expire in the same instant.
Check your understanding
Common mistakes
Caching authenticated responses publicly. The most serious CDN mistake there is. Audit Cache-Control on anything behind a login.
No CDN for API responses. Teams cache images and leave a public endpoint serving identical JSON to millions of users straight from origin. Even a few seconds of TTL is transformative.
Forgetting the origin still needs protection. A cache miss storm, or an attacker requesting random URLs that never cache, reaches you directly. Rate limit at the origin too.
Ignoring cache hit rate. It is the single metric that tells you whether the CDN is working. A low rate usually means over-broad Vary headers or missing TTLs.
Assuming a CDN fixes a slow origin. It only helps cacheable content. A slow, personalised, uncacheable page is unaffected.
Practice
Design the caching for a news site: a homepage that changes every few minutes, article pages that rarely change, per-user 'saved articles' pages, and images. For each, give a cache policy and a cache key, and say what breaks if you get it wrong.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Decide first, for each one, whether two different users may be served the same bytes.
Show a strong answer
- Images — cache aggressively at the edge. Immutable, content-addressed filenames with a very long TTL. Key on the URL alone. Getting this wrong wastes bandwidth but is not dangerous; getting the filename scheme wrong means stale images after a change.
- Article pages — cache at the edge with a short-to-medium TTL, plus stale-while-revalidate. Key on URL and, if you serve translations, the language. Serving a two-minute-old article is invisible to readers; serving one that never updates after a correction is not, so pair a TTL with an explicit purge on republish.
- Homepage — cache for a short TTL with revalidation. It changes every few minutes and receives the most traffic, so this is where caching earns the most. Key on URL plus any geography or edition that changes the content. The failure to design for here is the thundering herd: when the entry expires, thousands of requests miss simultaneously and all hit the origin. Serve stale while one request refreshes.
- Saved articles — never cached in a shared cache. Mark it explicitly private and uncacheable. Key does not apply because it must not be stored at the edge at all. Getting this wrong is a data leak, not a performance bug.
The organising question is the one above: may two users be served the same bytes? If no, it must not enter a shared cache, and nothing about TTLs makes that safe.
Your origin sees a sharp traffic spike every hour exactly on the hour, large enough to cause errors. Explain the mechanism and give two fixes with different trade-offs.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
What do a million cache entries created at the same moment have in common?
Show a strong answer
The mechanism is synchronised expiry. Entries created at the same moment with the same TTL expire at the same moment. Every edge simultaneously has a miss, and every miss becomes an origin request — so the origin receives, in one instant, the traffic it would normally have received spread across an hour. This is the thundering herd, and it is self-reinforcing: the resulting slowness means each request holds a connection longer, so the pile-up grows.
Fix one — serve stale while revalidating. On expiry, the edge returns the stale copy immediately and refreshes in the background, with only one request per edge allowed through to the origin. Trade-off: users briefly see slightly out-of-date content, which is fine for a homepage and unacceptable for anything that must be exact.
Fix two — jitter the TTL. Instead of exactly 3600 seconds, use 3600 plus a random spread of a few minutes. Expiry is now smeared across a window rather than concentrated at one instant. Trade-off: it does nothing for a genuinely popular single object — a million requests for the same URL still converge — so it addresses the aggregate spike but not per-key stampede. Request coalescing at the edge, where concurrent misses for one key produce one origin fetch, is what covers that case.
In practice you want both, and they compose well.
Key takeaways
- Distance has a physical floor — a round trip across the world costs 100ms+ regardless of your code. The only fix is to not make the trip.
- Beyond speed, a CDN provides origin offload, spike absorption and DDoS resilience — often the difference between three servers and fifty.
- The cache key decides what counts as the same request. Too narrow leaks other users' data; too broad destroys your hit rate.
- Versioned, hashed filenames make invalidation unnecessary — immutable content is trivially cacheable.
- Use
s-maxagefor the CDN andmax-agefor browsers, andstale-while-revalidatefor instant responses with prompt freshness. - Watch for the thundering herd when popular objects expire; request coalescing and jittered TTLs prevent it.
- Cache public API responses too, not just images.
Next: the same idea — keep the answer close and avoid the expensive trip — applied inside your own datacentre.
Further reading
- RFC 9111, HTTP Caching — freshness, validation, and the rules a shared cache must obey.
- RFC 5861, HTTP Cache-Control Extensions for Stale Content —
stale-while-revalidateandstale-if-error. - Jeffrey Mogul et al., "Potential Benefits of Delta Encoding and Data Compression for HTTP" (SIGCOMM 1997) — early measurement of what caching actually saves.
- Erik Nygren, Ramesh Sitaraman and Jennifer Sun, "The Akamai Network: A Platform for High-Performance Internet Applications" (ACM SIGOPS, 2010) — how a large CDN is actually built.
- Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapter 22 — cascading failures, including herd effects.