The Read-Heavy Assumption

"It's read-heavy, so we'll put a cache in front of it." That sentence appears in nearly every system design discussion, usually within the first ten minutes, and it is almost never followed by a question. It has the rhythm of a conclusion, so everyone treats it as one and moves on to the next box.

It's often true. It's just never checked, and the cases where it's false are exactly the cases where the design fails.

Why it earned its status

The assumption is popular because it's usually directionally right and cheap to act on.

Most user-facing systems really do skew hard towards reads. A post is written once and read many times. A product is listed once and viewed constantly. And access is rarely uniform: a small fraction of items typically accounts for a large fraction of requests, which is precisely the property that makes caching work. A cache that holds a modest slice of the key space can serve a majority of traffic, and there is no other single change with that leverage for that little effort.

So the heuristic is a good one. The problem is that it's applied as a conclusion rather than a hypothesis, and four things routinely break it.

Write amplification

The read/write ratio you quote is measured at the API. The ratio that matters is the one at the storage layer, and a fan-out design makes those two numbers wildly different.

Consider a feed built by writing to each follower's timeline. One user posts — that's a single write at the API. If they have a thousand followers, that's a thousand writes underneath. Add a search index update, a denormalised counter, a notification row and the invalidation of every cached view that included the item, and a "read-heavy" system is doing enormous write work that never appears in the ratio anyone quoted.

The skew makes it worse rather than better. Follower counts are wildly unequal, so the amplification factor for the heaviest accounts is orders of magnitude above the median, and those accounts are also the ones posting most. The system with a hundred-to-one read/write ratio at the edge can be write-bound at the disk.

Hot keys

The same skew that makes caching effective concentrates load onto individual keys, and a cache does not spread that concentration — it relocates it.

One key receiving a large share of all reads sits on one cache node, on one shard, served by one connection pool. Scaling the cache horizontally doesn't help, because adding nodes doesn't move a single key. And the failure mode is nasty: when that key expires, every concurrent request misses simultaneously and hits the database at once. You've built a mechanism that turns one expiry into a synchronised thundering herd against your least prepared component.

Caching hot keys is right. It just needs an answer for what happens at the moment of expiry, and "we'll set a TTL" is not one.

Personalisation

This is the one that quietly destroys hit rates, and it usually arrives after launch.

A cache works because many requests share a key. The moment the response depends on who is asking — a personalised ranking, a locale, a permission set, an experiment bucket, a feature flag combination — the key has to include all of it. Cardinality multiplies with every dimension, and hit rate collapses towards the fraction of users who happen to share a bucket.

You end up with a cache that mostly misses. That's worse than no cache: you've added a network hop to the read path, a memory bill, an invalidation problem and a new failure mode, in exchange for a hit rate that doesn't cover any of it.

The way out is to cache the shared parts and compose at request time — the item bodies, the rendered fragments, the reference data — and accept that the personalised assembly is not cacheable. That's a different design from "put a cache in front of it", and it only appears if someone asks the question early.

Read-your-own-writes

The user most likely to read a record in the seconds after it changes is the person who changed it. They are also the only user guaranteed to know the value is wrong.

So the first bug report after the cache ships is someone editing their profile and seeing the old version. The fix is usually to bypass the cache for the writer for a window, or to invalidate on write and eat the miss. Both are reasonable. Both mean your most engaged users — the ones taking actions rather than browsing — are the ones your cache serves least, which is worth knowing before you size it.

Checking it instead of asserting it

None of this needs a research project. It needs an afternoon.

Count the storage operations a single logical write causes. Not the API calls — the actual writes, including index updates, denormalised copies and invalidations. Multiply by the fan-out of your heaviest entities, not the median ones.

Compare distinct keys to request volume. Take a day of reads and count unique keys. If the two numbers are close, no cache can help you, because nothing is being asked for twice.

Estimate the hit rate before building. Sort keys by request count, take as many as fit your memory budget, and see what share of total requests they cover. That share is roughly your ceiling on hit rate, and you can compute it from a log file.

Then work out what the hit rate is worth. This is where the arithmetic gets uncomfortable. Suppose the database read is a third of your end-to-end latency and you achieve an 80% hit rate: you've removed about 0.8 × 1/3, a little over a quarter of the total. Real, but not the transformation the diagram implied — and if the read is only a tenth of the time, an excellent cache buys you 8%.

Ask what a cold cache does. After a deploy, a restart, or an eviction wave, every request misses at once. If your database cannot serve full traffic unaided for a few minutes, the cache is not an optimisation, it's a dependency — and it belongs in your availability calculation, not your performance one.

Say something falsifiable instead

Replace the sentence. Rather than "it's read-heavy so we'll add a cache", try: a large share of reads target a small share of keys, staleness of about a minute is acceptable for this data, so a cache of this size should reach a useful hit rate, and here's what happens when it's cold.

Every clause in that version can be wrong, which is the point. If you can't fill the blanks in, you haven't established that the workload is read-heavy — you've repeated a phrase that usually is.

This is also the cheapest thing to build intuition for by watching rather than reasoning. Put a cache in front of a database in the ArchiDepth canvas, run traffic where every request asks for a different key, and the hit rate tells you immediately that the component is doing nothing. Then concentrate the traffic on a handful of keys and watch the database load fall away. The gap between those two runs is the entire assumption, made visible in about a minute.