AI Systems

Building with models: inference, retrieval and RAG, agents and their tools, fine-tuning, evaluation and guardrails, and running it all. 7 modules, about 10 hours of reading. Every lesson ends in something to do.

Foundations

Introduction to AI System DesignWhat changes when a model is the workload — non-determinism, GPU economics, and why quality becomes an SLO.Intermediate17 min
LLM Inference & ServingWhy generation is slow, what the KV cache is, and how batching and quantization change your cost per request.Advanced18 min
Reasoning Models and Test-Time ComputeModels that think before answering — thinking budgets as a design knob, where reasoning earns its cost, and where it is pure overhead.Intermediate18 min

Retrieval and Context

Embeddings & Vector SearchSearching by meaning instead of words, why exact nearest-neighbour does not scale, and what ANN gives up.Intermediate17 min
Retrieval-Augmented Generation (RAG)Giving a model facts it was never trained on — chunking, hybrid search, reranking, and why retrieval quality caps everything.Intermediate18 min
Advanced RAGQuery rewriting, multi-hop retrieval, fusion and compression — the techniques that fix what basic RAG cannot.Advanced17 min
Prompt & Semantic CachingThe dominant cost lever — exact caching, prefix reuse, and matching questions that mean the same thing.Intermediate17 min
Context EngineeringBudgeting the context window, why long context did not kill RAG, and how to compact a conversation without losing it.Intermediate17 min
Prompt and Instruction DesignWriting the instructions themselves — system prompt structure, why models drop rules, what examples really do, and treating prompts as versioned code.Intermediate17 min

Agents

AI Agents & OrchestrationWhen one model call is not enough — loops, planning, multi-agent patterns, and the control problems they create.Advanced17 min
Tool Use & Function CallingHow models call your code — schema design, validation, side effects, and why tool descriptions are the interface.Intermediate17 min
AI Harness EngineeringEverything around the model that turns it into an agent — tools, context, permissions, verification and feedback — and why it decides capability.Advanced18 min
Loop EngineeringDesigning the agentic loop — termination, stuck detection, budgets, compaction across iterations, and recovering from failure without spiralling.Advanced18 min
Agent Memory & StateWhat an agent should remember, where it lives, and why durable state is a database problem rather than a prompt one.Advanced17 min
Multi-Agent SystemsWhen more agents help and when they only cost more — context isolation, lossy handoffs, compounding error, and why verification must be independent.Advanced19 min
Human-in-the-Loop and Approval DesignDeciding where a person intervenes — the autonomy spectrum, approval fatigue, and why making actions reversible beats asking permission.Intermediate18 min
Computer-Use and Browser AgentsAgents that drive a UI instead of an API — grounding, observation cost, and why a logged-in browser is the most dangerous tool you can hand a model.Advanced18 min
Agent Protocols & InteroperabilityStandardising how agents reach tools and each other — the integration problem, and what protocols do not solve.Advanced16 min

Models and Outputs

Structured Outputs & Constrained DecodingGetting machine-readable data out of a text generator, and why schema enforcement beats parsing and retrying.Intermediate17 min
Fine-Tuning vs RAG: A Decision FrameworkThe ladder from prompting to training, and the two reasons people fine-tune that are almost always wrong.Intermediate17 min
Parameter-Efficient Fine-TuningTraining a small number of new weights instead of all of them — adapters, why full fine-tuning is rarely right, and serving many variants.Advanced16 min
Distillation & Small ModelsTeaching a small model to imitate a large one — the strongest cost lever available, and where it stops working.Advanced17 min

Evaluation and Safety

LLM EvaluationMeasuring quality when there is no correct answer — eval sets, model-as-judge, and gating deploys on results.Advanced17 min
Agent Evaluation and Trajectory TestingJudging a process rather than an answer — trajectory metrics, measuring reliability instead of accuracy, and catching agents that pass by cheating.Advanced18 min
Guardrails & Prompt InjectionWhy a prompt is untrusted input, where injection actually comes from, and building controls that instructions cannot override.Advanced16 min
Privacy & PII in AI PipelinesWhat leaves your network, where personal data accumulates, and honouring deletion when data is spread across derived stores.Intermediate17 min

Operations

LLM ObservabilityTracing non-deterministic multi-step systems, attributing cost, and detecting quality decay that throws no errors.Intermediate17 min
Agent Cost and Latency EngineeringWhy agents cost an order of magnitude more than chat — where the tokens actually go, per-step model routing, and caching that survives a loop.Intermediate17 min
ML Pipelines & Feature StoresThe classical ML lifecycle — training pipelines, the training-serving skew problem, and what a feature store actually solves.Intermediate17 min
Model Deployment & RolloutShipping a change you cannot fully test — shadow traffic, canaries, online experiments and fast rollback.Intermediate16 min
Deploying and Operating AgentsRunning agents in production — long jobs that outlive a request, sandboxing, resumability, and shipping a change to a system that is stateful mid-flight.Advanced18 min
AI Gateways & Model RoutingOne front door for every model — provider abstraction, tiered routing, budgets, and the cheapest request you never made.Intermediate17 min

Applied Systems

Multimodal & Real-Time Voice SystemsDesigning for audio, images and video — the latency budget that makes conversation feel natural, and where it is spent.Advanced17 min
Design an AI AssistantThe capstone — a full worked case study combining retrieval, agents, guardrails, cost control and evaluation.Advanced18 min

Foundations

Introduction to AI System DesignWhat changes when a model is the workload — non-determinism, GPU economics, and why quality becomes an SLO.Intermediate17 min

LLM Inference & ServingWhy generation is slow, what the KV cache is, and how batching and quantization change your cost per request.Advanced18 min

Reasoning Models and Test-Time ComputeModels that think before answering — thinking budgets as a design knob, where reasoning earns its cost, and where it is pure overhead.Intermediate18 min

Retrieval and Context

Embeddings & Vector SearchSearching by meaning instead of words, why exact nearest-neighbour does not scale, and what ANN gives up.Intermediate17 min

Retrieval-Augmented Generation (RAG)Giving a model facts it was never trained on — chunking, hybrid search, reranking, and why retrieval quality caps everything.Intermediate18 min

Advanced RAGQuery rewriting, multi-hop retrieval, fusion and compression — the techniques that fix what basic RAG cannot.Advanced17 min

Prompt & Semantic CachingThe dominant cost lever — exact caching, prefix reuse, and matching questions that mean the same thing.Intermediate17 min

Context EngineeringBudgeting the context window, why long context did not kill RAG, and how to compact a conversation without losing it.Intermediate17 min

Prompt and Instruction DesignWriting the instructions themselves — system prompt structure, why models drop rules, what examples really do, and treating prompts as versioned code.Intermediate17 min

Agents

AI Agents & OrchestrationWhen one model call is not enough — loops, planning, multi-agent patterns, and the control problems they create.Advanced17 min

Tool Use & Function CallingHow models call your code — schema design, validation, side effects, and why tool descriptions are the interface.Intermediate17 min

AI Harness EngineeringEverything around the model that turns it into an agent — tools, context, permissions, verification and feedback — and why it decides capability.Advanced18 min

Loop EngineeringDesigning the agentic loop — termination, stuck detection, budgets, compaction across iterations, and recovering from failure without spiralling.Advanced18 min

Agent Memory & StateWhat an agent should remember, where it lives, and why durable state is a database problem rather than a prompt one.Advanced17 min

Multi-Agent SystemsWhen more agents help and when they only cost more — context isolation, lossy handoffs, compounding error, and why verification must be independent.Advanced19 min

Human-in-the-Loop and Approval DesignDeciding where a person intervenes — the autonomy spectrum, approval fatigue, and why making actions reversible beats asking permission.Intermediate18 min

Computer-Use and Browser AgentsAgents that drive a UI instead of an API — grounding, observation cost, and why a logged-in browser is the most dangerous tool you can hand a model.Advanced18 min

Agent Protocols & InteroperabilityStandardising how agents reach tools and each other — the integration problem, and what protocols do not solve.Advanced16 min

Models and Outputs

Structured Outputs & Constrained DecodingGetting machine-readable data out of a text generator, and why schema enforcement beats parsing and retrying.Intermediate17 min

Fine-Tuning vs RAG: A Decision FrameworkThe ladder from prompting to training, and the two reasons people fine-tune that are almost always wrong.Intermediate17 min

Parameter-Efficient Fine-TuningTraining a small number of new weights instead of all of them — adapters, why full fine-tuning is rarely right, and serving many variants.Advanced16 min

Distillation & Small ModelsTeaching a small model to imitate a large one — the strongest cost lever available, and where it stops working.Advanced17 min

Evaluation and Safety

LLM EvaluationMeasuring quality when there is no correct answer — eval sets, model-as-judge, and gating deploys on results.Advanced17 min

Agent Evaluation and Trajectory TestingJudging a process rather than an answer — trajectory metrics, measuring reliability instead of accuracy, and catching agents that pass by cheating.Advanced18 min

Guardrails & Prompt InjectionWhy a prompt is untrusted input, where injection actually comes from, and building controls that instructions cannot override.Advanced16 min

Privacy & PII in AI PipelinesWhat leaves your network, where personal data accumulates, and honouring deletion when data is spread across derived stores.Intermediate17 min

Operations

LLM ObservabilityTracing non-deterministic multi-step systems, attributing cost, and detecting quality decay that throws no errors.Intermediate17 min

Agent Cost and Latency EngineeringWhy agents cost an order of magnitude more than chat — where the tokens actually go, per-step model routing, and caching that survives a loop.Intermediate17 min

ML Pipelines & Feature StoresThe classical ML lifecycle — training pipelines, the training-serving skew problem, and what a feature store actually solves.Intermediate17 min

Model Deployment & RolloutShipping a change you cannot fully test — shadow traffic, canaries, online experiments and fast rollback.Intermediate16 min

Deploying and Operating AgentsRunning agents in production — long jobs that outlive a request, sandboxing, resumability, and shipping a change to a system that is stateful mid-flight.Advanced18 min

AI Gateways & Model RoutingOne front door for every model — provider abstraction, tiered routing, budgets, and the cheapest request you never made.Intermediate17 min

Applied Systems

Multimodal & Real-Time Voice SystemsDesigning for audio, images and video — the latency budget that makes conversation feel natural, and where it is spent.Advanced17 min

Design an AI AssistantThe capstone — a full worked case study combining retrieval, agents, guardrails, cost control and evaluation.Advanced18 min

Foundations

Retrieval and Context

Agents

Models and Outputs

Evaluation and Safety

Operations

Applied Systems

Foundations

Retrieval and Context

Agents

Models and Outputs

Evaluation and Safety

Operations

Applied Systems

Your progress