Design Instagram

This is the first case study, so we will work it exactly as you should in an interview — following the five phases from the interview template rather than jumping to a diagram.

The goal is not to memorise this architecture. It is to see how requirements and numbers produce an architecture.

Phase 1: Clarify

The prompt "design Instagram" is deliberately enormous. Narrow it.

Functional requirements — the small set that defines the product:

1. Upload a photo with a caption.
2. Follow other users.
3. View a feed of photos from people you follow.
4. Like and comment.

Explicitly out of scope, so you protect your time: stories, direct messages, reels, search, advertising, recommendations.

Non-functional requirements — these shape the architecture far more than the feature list:

• Read-heavy. People scroll far more than they post.
• Feed latency under 200ms. Scrolling must feel instant.
• High availability over strict consistency. A post appearing a few seconds late is invisible; an error page is not. This is a deliberate AP choice from the CAP lesson.
• Photos are immutable once uploaded. This single fact makes caching and CDN delivery trivial.

Phase 2: Estimate

Assume 500M daily active users, each posting once every ten days and viewing 20 photos a day.

Writes: 50M photos/day ÷ 86,400 ≈ 600 uploads per second.

Reads: 500M × 20 = 10B photo views/day ÷ 86,400 ≈ 115,000 per second.

Read:write ratio ≈ 200:1. This single number drives most of what follows.

Storage: at ~2MB per photo after processing, 50M × 2MB = 100 TB per day, roughly 36 PB a year.

Now state what the numbers mean, which is the part that earns credit:

"A 200:1 read ratio means this is a caching and delivery problem more than a database problem. 100TB a day rules out storing images in a database — they belong in object storage behind a CDN. And 600 writes per second is modest, so the upload path is not where the difficulty lies. The feed is."

Phase 3: API and data model

POST /v1/photos            multipart upload  -> { photoId, url }
GET  /v1/feed?cursor=...                     -> { photos[], nextCursor }
POST /v1/users/{id}/follow
POST /v1/photos/{id}/likes

The feed uses cursor pagination, not offset — the feed changes constantly while a user scrolls, and offset would duplicate and skip posts.

The data:

users     (id, username, email, created_at)
photos    (id, user_id, s3_key, caption, created_at)
follows   (follower_id, followee_id, created_at)   PK (follower_id, followee_id)
likes     (user_id, photo_id, created_at)          PK (user_id, photo_id)

Note what is not here: the image bytes. The database stores a key pointing into object storage.

Phase 4: High-level design

The read path is the one worth running: clients behind a load balancer, an API fleet, and a cache in front of the metadata store. (The upload path below — object storage, its CDN and the processing queue — is not part of this simulated slice.)

The upload path

The naive version routes 2MB through your API servers, which wastes bandwidth and ties up connections.

Instead, the client asks for a pre-signed URL and uploads directly to object storage. Your servers never touch the bytes. They only write a metadata row and publish an event.

Workers then consume that event and generate thumbnails and display sizes asynchronously — exactly the message queue pattern. The user's upload completes in the time it takes to store the original; resizing happens after.

The read path

Photos are immutable, so they are served from the CDN with a very long TTL. Your servers are never involved in delivering an image. Given 115,000 reads per second, this is not an optimisation — it is the only viable design.

The interesting problem: the feed

Everything above is standard. The feed is where the real decision is, and it is what the interviewer wants to discuss.

Approach A: fan-out on read (pull)

When a user opens the app, query their followees' recent photos and merge:

SELECT * FROM photos
WHERE user_id IN (SELECT followee_id FROM follows WHERE follower_id = ?)
ORDER BY created_at DESC LIMIT 20;

Good: writes are trivial — posting is one row. No wasted work for inactive users.

Bad: reads are expensive. A user following 500 accounts triggers a large query on every feed load, and we established reads outnumber writes 200:1. We would be optimising the rare operation and punishing the common one.

Approach B: fan-out on write (push)

When someone posts, immediately write that photo id into a precomputed feed list for each follower, held in Redis.

Reading the feed becomes a single list read — sub-millisecond.

Good: reads are extremely fast, which is exactly what a 200:1 ratio demands.

Bad: writes amplify. A user with 1,000 followers causes 1,000 list insertions. And for a celebrity with 50 million followers, one post means 50 million writes — a single action creating a massive burst. This is the hot-key problem from the partitioning lesson, in its most famous form.

The answer: hybrid

Use push for ordinary users and pull for celebrities.

• Posting by a normal account fans out to followers' feed lists.
• Posting by an account above a follower threshold writes nothing.
• On feed load, read the precomputed list and merge in recent posts from the handful of celebrities you follow.

This is the design Twitter and Instagram both converged on, and it is the answer interviewers are hoping for. The reasoning matters more than the conclusion: you pushed the cost onto whichever path could afford it, per user.

Bottlenecks and failure

Feed cache memory. Storing 500 photo ids per user for 500M users is large. Cap feed length — nobody scrolls past a few hundred posts — and only precompute feeds for users active recently. Dormant users fall back to pull.

The metadata database. 600 writes per second is fine for one leader, but 36PB of metadata over time is not. Shard photos by user_id, and replicate each shard.

Thundering herd on a viral post. Millions request the same image at once. The CDN absorbs this, provided request coalescing is on — the exact failure mode from the CDN lesson.

Queue backlog. If processing workers fall behind, uploads still succeed but thumbnails lag. Acceptable degradation, and worth saying so explicitly.

What if the feed service dies? Degrade to fan-out on read. Slower, but the product works.

Trade-offs you should name aloud

• AP over CP: a post may take seconds to appear in every feed. Acceptable here; not acceptable for a payment.
• Denormalised feeds duplicate data and must be rebuilt if corrupted, in exchange for fast reads.
• Async processing means a photo may briefly lack a thumbnail.
• A follower threshold is a tuning knob, not a law — set it from real distribution data.

Check your understanding

Common mistakes

Storing images in the database. Binary blobs in rows make every backup, replica and query slower for no benefit. Media goes to object storage; the database keeps the metadata and the key.

Serving media through the application. Every byte routed through your servers is bandwidth you pay for twice and latency you add for nothing. Media is served from the edge, and the application only ever hands out URLs.

Generating thumbnails on request. Resizing on the read path turns a cheap request into an expensive one and ties image processing to user-facing latency. Derivatives are produced once, asynchronously, on upload.

One size of image for everyone. A phone on a slow connection and a desktop on fibre should not receive the same file. Serving a single large original is the most common cause of a feed that feels slow despite a fast backend.

Forgetting the upload is the risky path. Uploads are large, slow, and frequently interrupted. A design that only describes the happy path has skipped resumability, partial writes and the orphaned objects an abandoned upload leaves behind.

Treating moderation as someone else's problem. User-uploaded media needs a scanning and takedown path, and where it sits changes the architecture — before publication is a different system from after.

Practice

Key takeaways

• Work the phases in order — the 200:1 read ratio is what makes this a caching and delivery problem, and you cannot know that without estimating.
• Never route media through your API. Pre-signed URLs upload straight to object storage; the CDN serves reads.
• Photos are immutable, which makes long CDN TTLs safe and removes invalidation entirely.
• Fan-out on read is cheap to write and expensive to read; fan-out on write is the reverse. The ratio picks the default.
• The hybrid — push for ordinary users, pull for celebrities — exists because follower counts are extremely skewed, and it is the expected answer.
• Name your degradations: stale feeds, late thumbnails, and a fallback to pull if the feed service fails.

Further reading

• Sanjay Ghemawat, Howard Gobioff and Shun-Tak Leung, "The Google File System" (SOSP 2003) — large immutable blobs.
• Doug Beaver et al., "Finding a Needle in Haystack: Facebook's Photo Storage" (OSDI 2010) — why general file systems fail for billions of photos.
• Nathan Bronson et al., "TAO: Facebook's Distributed Data Store for the Social Graph" (USENIX ATC 2013) — the read-heavy graph behind a feed.
• Rajesh Nishtala et al., "Scaling Memcache at Facebook" (NSDI 2013).
• Erik Nygren, Ramesh Sitaraman and Jennifer Sun, "The Akamai Network" (ACM SIGOPS, 2010) — serving the egress a photo product generates.