Introduction to System Design

You already know how to make a program work. System design asks a harder question:

Will it still work when a million people use it at the same time?

That one question is the whole subject. Everything else — load balancers, caches, queues, shards — is just the accumulated set of tricks engineers invented to answer it.

The moment everything breaks

System design only makes sense once you have felt the pain it solves, so let us be concrete.

You build a small app for your college. Students post notes, others download them. It runs on one rented server costing a few dollars a month. It works beautifully.

Then a popular account shares it. Traffic goes from 50 people a day to 50,000 an hour. Here is what actually happens, in order:

1. The server slows down. It has a fixed amount of CPU. Requests begin queuing behind each other, waiting their turn.
2. Response times climb. A page that took 100 milliseconds now takes 4 seconds. Users hit refresh, which creates more requests, which makes it worse.
3. The database runs out of connections. Every visitor needs one to read data. Your database allows perhaps 100 at once. Visitor 101 waits. Visitor 500 gives up.
4. Memory fills. The machine starts swapping to disk, which is thousands of times slower than RAM.
5. It dies. Not gracefully. It simply stops answering.

Notice that nothing here is a bug. Your code is correct. Every line does exactly what you told it to. The system failed because of scale, and scale is a design problem, not a coding problem.

So what is system design, really?

System design is deciding how to arrange parts so the whole survives reality.

Think of an airport. Every individual piece is simple: a check-in desk, a security lane, a gate, a baggage belt. Nobody finds a check-in desk intellectually difficult. The difficulty is:

• How many desks for 30,000 passengers a day?
• What happens when one security lane closes?
• Where do people queue so the queue does not block the entrance?
• If one flight is delayed, how do you stop that cascading into ten others?

An airport architect does not invent new kinds of desks. They decide how many, arranged how, with what fallbacks. That is exactly your job. You will rarely invent a new database. You will constantly decide how many, arranged how, with what fallbacks.

The two altitudes

The field splits into two levels of zoom. Both matter, and interviews test both.

High-Level Design: the map of the city

HLD is the view from above — boxes and arrows. This service talks to that database, requests arrive through this load balancer, images live in that storage bucket.

At this altitude you decide:

• Do we need a cache, and where does it sit?
• Should these two features be one service or two?
• SQL or NoSQL, and why?
• What happens when this component dies?

You are not writing functions. You are deciding the shape of the system.

Low-Level Design: the blueprint of one building

LLD zooms into a single component and asks how it is built inside. Which classes exist, what the tables look like, how the API behaves, which algorithm runs.

At this altitude you decide:

• What are the exact tables and columns, and which fields are indexed?
• How do these classes relate, and where do responsibilities sit?
• What does the API return when the input is invalid?
• Is this loop fast enough at ten thousand items?

The four forces you are always balancing

Every decision in this entire field trades between the same handful of pressures. Learn these four now and the rest of the course reads as variations on a theme.

The uncomfortable truth is that you cannot maximise all four. Pushing hard on one pulls another down. A bank chooses consistency and accepts being slower. A social feed chooses speed and accepts that a friend's new post may take a few seconds to appear.

This is why "it depends" is the honest answer to most system design questions, and why interviewers want to hear what it depends on.

A first look at a real shape

Here is roughly what the college notes app becomes after surviving its first viral moment. Every box gets its own lesson later — for now just notice that one machine became a set of specialists.

Read it as a story:

• The load balancer spreads visitors across several servers, so no single machine absorbs all the pressure.
• Three app servers instead of one, so losing a machine costs a third of capacity rather than everything.
• A cache sits in front of the database holding the most-requested notes in memory, so the database only sees traffic the cache could not answer.
• The database remains the single source of truth, but now handles far less load.

Nothing exotic happened. We took one overloaded thing and split its jobs across several specialised things.

What actually makes this hard

Beginners assume the difficulty is memorising technologies. It is not — there are perhaps fifteen building blocks in total, and you will know them all by the end of this course.

It is hard for three other reasons.

Machines fail constantly. At real scale something is always broken. A disk dies, a cable is unplugged, a deploy goes wrong. Good design treats failure as the normal state, not an emergency.

The network is unreliable. When Server A sends a message to Server B and hears nothing back, A cannot distinguish "B never received it" from "B is still working" from "B replied and the reply was lost". That ambiguity is the source of an enormous amount of later complexity.

There is no single correct answer. In an algorithms question your solution is right or wrong. Here your design is a bundle of trade-offs, and a good interviewer is judging whether you know which ones you made.

Check your understanding

How to study this course

Three habits make everything after this page easier.

Always ask "what breaks first?" Every component has a limit. When you look at a design, hunt for the piece that gives way soonest under load. That piece is the bottleneck, and finding it is most of the skill.

Always ask "what happens when this dies?" Point at any box and mentally delete it. If the whole system stops, you have found a single point of failure. Real designs have an answer for each one.

Prefer the boring option. Beginners reach for the most sophisticated tool. Experienced engineers reach for the simplest thing that meets the requirement, because every extra component is one more thing that can fail at 3am. If one database will do, use one database.

Common mistakes

Jumping to components before agreeing on the problem. Naming a queue and a cache in the first minute feels productive and commits you to a shape before you know the read/write ratio, the consistency the business actually needs, or how much data there is. Requirements first, every time.

Designing for a scale nobody asked for. A system that serves ten thousand people does not need the architecture of one that serves ten million, and proposing the second is not ambition — it is a failure to ask. Get the numbers, then pick the design the numbers justify.

Treating "scalable" as a property you can add. Scalability is not a component you bolt on; it is a consequence of where you put state. A design that keeps session state on the web server cannot be made scalable by adding machines, only by moving that state.

Naming technologies instead of describing behaviour. "We'll use Kafka" says nothing about what ordering you need, what happens to a message nobody consumes, or how long history is kept. Describe the behaviour you require and the technology choice becomes obvious — or becomes visibly arbitrary.

Going silent while thinking. In an interview the reasoning IS the answer. A silent candidate who arrives at a good design scores worse than one who narrates a merely adequate one, because only the second demonstrated how they got there.

Never saying what the design gives up. Every real design sacrifices something. A proposal presented as having no downside reads as one whose downsides have not been found yet.

Practice

Key takeaways

• System design is about behaviour under scale and failure, not code correctness. Working code can still be a broken system.
• HLD is the arrangement of services and data stores; LLD is the internal structure of one component. Interviews test both.
• Every decision trades between scalability, availability, consistency and latency. You cannot maximise all four, so name the trade you are making.
• Assume failure is routine. Machines die and networks drop messages as a matter of course.
• Two questions unlock most problems: what breaks first? and what happens when this dies?

Next we look at the most direct answer to "the server is overloaded" — adding more machines, and the surprisingly deep consequences of how you add them.

Further reading

• Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly, 2017) — the standard reference for the ideas this course builds on.
• Google SRE, Site Reliability Engineering (O'Reilly, 2016) — how large systems are actually operated.
• Peter Deutsch and James Gosling, "The Eight Fallacies of Distributed Computing" (Sun Microsystems, 1994–1997) — the assumptions that break every first distributed design.
• Fred Brooks, The Mythical Man-Month (Addison-Wesley, 1975) — essential versus accidental complexity, which is what the four forces are really about.
• Sam Newman, Building Microservices, second edition (O'Reilly, 2021) — when service boundaries help and when they do not.