Rate Limiting & Throttling

Your login endpoint works. It checks an email and password and returns a token.

Someone points a script at it that tries ten thousand passwords a second. Three things now happen at once: your database is saturated with lookups, real users cannot log in because every connection is busy, and given enough time the attacker finds a weak password.

You did not have a security bug. You had no limit on how often one caller could ask.

Rate limiting is the control that says: you may make N requests per unit of time; beyond that, I will refuse. It is the cheapest protection in this course and one of the most commonly missing.

What it actually protects

Abuse — brute-force logins, scraping, spam. A limit of five login attempts a minute makes password guessing useless without affecting a real person who mistypes twice.

Runaway costs — if each request triggers an AI model call costing a rupee, an unbounded loop in someone's client is a bill.

Fair sharing — one enthusiastic customer should not consume the capacity everyone else paid for.

Cascading failure — this is the one people underestimate. When a service is overwhelmed it slows, callers retry, retries add load, and the whole system spirals down. Shedding excess load early keeps the service healthy for the requests it can serve. Rejecting 10% of traffic quickly is far better than failing 100% of it slowly.

The four algorithms

Fixed window

Count requests in each clock-aligned window. Limit 100 per minute: everything in 10:00:00–10:00:59 shares one counter, which resets at 10:01:00.

Simple and cheap — one counter per user. But it has a real flaw:

A client sends 100 requests at 10:00:59 and another 100 at 10:01:00. Both windows are within limit, yet 200 requests arrived in one second. At a window boundary you can be hit with double your intended rate.

Sliding window

Instead of clock-aligned buckets, look at the trailing period from now. At 10:01:30, count requests since 10:00:30.

This removes the boundary burst entirely. The exact version stores a timestamp per request, which costs memory at scale; a common compromise weights the previous window's count by how much of it still overlaps, giving near-exact behaviour with two counters.

Token bucket

The most widely used, because it models the thing people actually want.

A bucket holds up to N tokens and refills at a steady rate. Each request removes one token. No tokens, no service.

class TokenBucket {
  constructor(capacity, refillPerSecond) {
    this.capacity = capacity;
    this.tokens = capacity;
    this.refillRate = refillPerSecond;
    this.lastRefill = Date.now();
  }

  tryConsume() {
    const now = Date.now();
    this.tokens = Math.min(
      this.capacity,
      this.tokens + ((now - this.lastRefill) / 1000) * this.refillRate
    );
    this.lastRefill = now;

    if (this.tokens >= 1) { this.tokens -= 1; return true; }
    return false;
  }
}

The valuable property is burst tolerance. A user idle for a minute has a full bucket and can fire twenty requests at once — which is exactly what a page loading twenty thumbnails does. Sustained abuse still gets throttled to the refill rate. It permits bursty legitimate behaviour while capping the long-run average.

Leaky bucket

Requests join a queue drained at a constant rate. Where the token bucket allows bursts through, the leaky bucket smooths them into an even stream.

Use it when the thing downstream cannot absorb spikes — a payment provider that permits exactly 10 requests per second, or a legacy system that falls over under bursts.

The mistake that makes it useless

You implement a beautiful token bucket in memory. You deploy behind a load balancer to three servers. Your limit is now silently three times what you configured.

Each server holds its own counters and sees only the traffic routed to it. A client spreading requests across all three gets the full quota on each.

The fix is shared state — put counters in Redis so all servers consult the same numbers:

// Atomic: increment and set expiry in one round trip
const key = `ratelimit:${userId}:${Math.floor(Date.now() / 60000)}`;
const count = await redis.incr(key);
if (count === 1) await redis.expire(key, 60);
if (count > LIMIT) throw new RateLimitError();

This is the single most common rate-limiting bug in production, and it is invisible in local testing where you run one instance.

Rejecting politely

When you refuse a request, say so in a way clients can act on. Return 429 Too Many Requests with headers describing the limit:

HTTP/1.1 429 Too Many Requests
Retry-After: 30
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1718020800

Retry-After matters most: without it, a well-meaning client retries immediately and makes things worse. With it, clients back off correctly and your limiter reduces load instead of merely relocating it.

Send X-RateLimit-Remaining on successful responses too, so good clients can slow down before hitting the wall rather than discovering it by being blocked.

What to key the limit on

The identifier you count against determines what you are actually protecting.

• API key or user ID — the right default for authenticated traffic; it follows the customer across devices.
• IP address — necessary for unauthenticated endpoints such as login and signup, but blunt. An entire office or university behind one NAT shares an address, so keep IP limits generous.
• Endpoint — limits should differ by cost. Login might allow 5 a minute; a cheap read might allow 1,000.
• Combination — in practice, per-user and per-IP and a global cap, since each catches a different abuse pattern.

Check your understanding

Common mistakes

Rate limiting only at the application layer. If the request already reached your code, you paid to parse and authenticate it. Limiting at the gateway or CDN rejects abuse before it costs you anything.

No limit on expensive endpoints. Teams limit login and forget the report generator that runs a thirty-second query.

Blocking your own internal services. Health checks, cron jobs and internal callers need exemptions or separate quotas, or you will page yourself.

Returning 500 instead of 429. Clients treat 500 as a transient fault and retry aggressively. 429 with Retry-After is instruction; 500 is an invitation to hammer.

Counting rejected requests toward the limit. Debatable, but usually wrong — it extends the block indefinitely for a client that keeps retrying, which is exactly what a badly written client does.

Practice

Key takeaways

• Rate limiting protects against abuse, runaway cost, unfair usage and cascading failure. Shedding 10% of load fast beats failing 100% slowly.
• Fixed window is cheapest but allows double rate at boundaries; sliding window fixes that at higher cost.
• Token bucket is the usual default because it tolerates legitimate bursts while capping the sustained rate; leaky bucket smooths traffic for fragile dependencies.
• In-memory counters break behind a load balancer — your limit multiplies by the number of instances. Use shared, atomically-updated state.
• Reject with 429 and Retry-After, never 500, and expose remaining quota on successful responses.
• Key limits on user, IP and endpoint together, with limits proportional to what each endpoint costs.

Further reading

• RFC 6585, Additional HTTP Status Codes — 429 Too Many Requests.
• RFC 9239 draft series, RateLimit header fields for HTTP — advertising limits to clients.
• Google SRE, Site Reliability Engineering (O'Reilly, 2016), chapter 21 — handling overload.
• OWASP, Authentication Cheat Sheet — rate limiting password reset and login endpoints.
• Jeff Kaufman, "Token Bucket vs Leaky Bucket" (2016) — the behavioural difference under bursty load.