Skip to content
All problems

Rate limits for a public address-lookup API

EasyAbout 30 minutes

Before you start:Rate Limiting,API Gateway

Your attempt

A complete grade of 70 or more solves it.

Your company sells an address-verification API. E-commerce and logistics companies call it from their checkout pages: a shopper types a PIN code and an address, and the API says whether the address exists and how to write it properly. About 3,000 partner integrations use it, each with its own API key.

Last month one partner shipped a bug. Their integration retried every failed call at once and without a limit, and it pushed the API to three times its normal traffic. The lookup service fell over, and every other partner's checkout pages broke with it. Your job is to make sure that cannot happen again, without refusing the traffic partners have paid for.

What the API does

  • GET /verify?pin=…&address=… with an API key returns the verified address.
  • Each partner has a contracted rate: most are sold 50 requests/s, the largest 500. Above its rate a partner gets HTTP 429 with a Retry-After header.
  • The platform as a whole is sold and provisioned for 2,400 requests/s. Traffic above that is refused at the edge, before it reaches the lookup service.

The numbers

  • About 60 million lookups a day, with an evening peak 2.3 times the daily average.
  • A few large partners send most of the traffic.
  • A lookup costs about 2 ms of CPU on a 2-vCPU instance. That cost belongs to the problem, so set read_cpu_ms on your lookup service to match it.
  • The surge you are designing for is 7,200 requests/s: three times the contracted rate.

What good looks like

Every answer, including a refusal, arrives within 100 ms at p99. Within the contracted rate at most 1 lookup in 1,000 fails. At three times the contract, at least half of all requests are refused at the gateway. A checkout page would much rather get a quick 429 it can handle than wait for a lookup that times out.

Where to start

Put a gateway in front of the lookup service and decide two numbers before you touch anything else: how much the service can take before it slows down, and how much you let through. Then run the surge and check which of them held.

Out of scope: how partners sign up and get their keys, billing for usage, and the address data itself.

Requirements

Functional

  • Look up and verify an address by PIN code for a partner that calls with its API key.
  • Each partner has a contracted rate. Above it, that partner is refused with HTTP 429 and a Retry-After header.
  • The platform refuses traffic above what it is provisioned for at the edge, before the traffic reaches the lookup service.

Non-functional

  • Checkout pages call the API while a shopper waits, so a fast refusal is better than a slow answer.
  • Most partners are contracted for 50 requests/s and the largest for 500.
  • Refusing a request must cost the platform almost nothing.

Estimation

Before you draw, estimate these. The workspace checks each answer against its band as you type.

  1. Lookups per second at the evening peak (requests/s)
  2. Service instances needed at the contracted rate (instances)
  3. Share of requests refused during the surge (ratio)

Tests

2 sample tests you can read and run

  • Traffic at three times the contract is mostly refused at the gateway (Functionality)
  • The full contracted rate is served within the SLO (Scalability)

2 hidden tests, graded when you submit

  • Reliability1
  • Scalability1

What the design must meet

Service levels

  • Every answer, refusals included, arrives within 100 ms at p99.
  • Within the contracted rate, at most 1 lookup in 1,000 fails.
  • When traffic reaches three times the contracted rate, at least half of all requests are refused at the gateway.

Load

  • The evening peak of an ordinary day runs at about 2.3 times the daily average.
  • The platform is sold and provisioned for 2,400 requests/s in total.
  • A partner's runaway retry loop has pushed traffic to 7,200 requests/s, three times the contracted rate.

How an attempt runs

  1. RequirementsWhat the service must do, and the numbers it must meet
  2. EstimateSize the load before you draw
  3. DesignBuild it on the canvas and run traffic through it
  4. TestRun the sample tests in your browser
  5. SubmitThe hidden tests grade the design