Queues, delivery guarantees, dead letters and backpressure are covered in high-level design. This topic is about the named structures built on top of them — the recurring arrangements of producers, consumers and messages that solve specific integration problems.
Each is small. Each has one consequence people discover late.
Competing consumers
Several consumers read from one queue; each message goes to exactly one of them.
Diagram, as text
- Producer leads to Queue
- Queue leads to Consumer 1
- Queue leads to Consumer 2
- Queue leads to Consumer 3
This is how you scale processing: add consumers, throughput rises, no code changes. It is the default arrangement for work queues and it is almost always what you want.
The consequence discovered late: you have given up ordering. Messages are delivered to whichever consumer is free, and one consumer may be slower or may retry. OrderUpdated arriving before OrderCreated at a downstream service is a real and common outcome.
Three ways to get ordering back, in increasing order of cost:
- Partition by key. All messages for one order go to one consumer, chosen by a hash of the order id. Ordering within a key, parallelism across keys — the same reasoning as choosing an actor boundary or a shard key, appearing for the third time.
- Make handlers order-independent. An update carrying full state can be applied whenever it arrives; an update carrying a delta cannot. This is often cheaper than enforcing order.
- Sequence numbers with buffering. The consumer holds out-of-order messages until the gap fills. Powerful, and it introduces a stall when a message is lost.
Partitioning is the usual answer, and its cost is that one slow key blocks its partition while others proceed.
Claim check
Large payloads do not belong in messages. Brokers have size limits, large messages slow every consumer that must deserialise them, and a message that is mostly binary is expensive to retain and replay.
Store the payload; send the reference.
const key = await objectStore.put(imageBytes);
await queue.publish({ type: 'image.uploaded', claimCheck: key, size: imageBytes.length });The consequences worth planning for:
- Two things now have lifetimes. If the object is deleted before the message is consumed — a retention policy, a cleanup job, a dead-letter replay months later — the consumer gets a reference to nothing. Object retention must exceed the maximum possible message age, including time in a dead-letter queue.
- Failure has two places to happen. Storing succeeded and publishing failed leaves an orphaned object; publishing succeeded and storing failed leaves a dangling reference. This is the dual-write problem again — store first, publish second, and sweep orphans periodically, because an orphaned object is cheap and a dangling reference is a failure.
- Access control follows the reference. Whoever holds the message can fetch the payload, so the reference should be authorised rather than a permanently valid public URL.
Scatter–gather
Send a request to several services, collect the responses, combine them. A search page querying products, reviews and inventory in parallel; a price comparison hitting five providers.
The design is entirely about the failure and timing rules, which have to be decided rather than defaulted:
- How long to wait. A fixed deadline for the whole gather, not per request. Otherwise the slowest possible response defines your latency, and the tail of five services is much worse than the tail of one.
- What a partial result means. If reviews time out, is the page still valid? Usually yes, degraded. If inventory times out, probably not. Mark which responders are required and which are optional, and let the gather return as soon as the required ones are in.
- Whether to cancel. Once the deadline passes, outstanding requests should be cancelled, or they consume capacity for a result nobody will read.
- What the response says. A partial result must be labelled as partial, or callers cache it and persist it as complete — the same argument as the degraded-response flag under Decorator.
Message routing and translation
Two small patterns that prevent a large mess.
A router sends a message to a destination based on its content — priority orders to a fast queue, or region-specific messages to a regional consumer. Keeping this decision in one router rather than in every producer means routing changes in one place.
A translator converts between message formats at a boundary, so an external partner's shape never enters your system. This is Adapter, applied to messages, and the same rule holds: the translator translates and does not decide.
Check your understanding
Common mistakes
Assuming ordering with competing consumers. Related messages are processed out of sequence and the bug is intermittent.
Partitioning by a key with poor distribution. One hot key becomes a single-threaded bottleneck while other partitions idle.
Large payloads in messages. Broker limits, slow deserialisation, and expensive retention and replay.
Claim-check retention shorter than maximum message age. Dead-letter replays find nothing.
Publishing the claim check before storing the payload. A dangling reference instead of a cheap orphan.
Per-request deadlines in a scatter–gather. Latency becomes the maximum of several tails.
Partial results not labelled as partial. Callers cache and persist them as complete.
Business decisions in a router. Untestable, invisible from the domain, unavailable elsewhere.
Practice
A video-processing pipeline publishes messages containing the video file's bytes. The broker rejects anything over 10 MB and consumers are slow. Redesign it, and enumerate the failure modes your design introduces.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Claim check is the obvious answer. The question is what it makes you responsible for.
Show a strong answer
The redesign is straightforward: upload to object storage, publish a reference. The message carries a storage key, the size, the content type and a checksum. Consumers fetch what they need. Messages become small, the broker limit stops mattering, and consumers no longer deserialise megabytes to decide whether they care about a message.
Order of operations: store first, then publish. This is the dual-write problem in miniature and the asymmetry is decisive — a stored object with no message is an orphan costing a few pence, while a message with no object is a hard failure at consume time. Store, then publish, then sweep orphans on a schedule.
Failure modes the design introduces, which is the substance of the question:
- Orphaned objects. Upload succeeds, publish fails. Handled by a sweeper that deletes objects with no corresponding message older than some threshold. Needs care not to delete objects whose message is legitimately still queued.
- Dangling references. Ordering prevents the common case, but retention can recreate it: a message dead-lettered and replayed six weeks later finds an object deleted by a 30-day policy. Object retention must exceed the maximum message age including dead-letter time, and I would set it well above rather than exactly at the limit.
- Fetch failures at consume time. Object storage can be slow or briefly unavailable, so what used to be a local deserialisation is now a network call inside the handler. It needs a timeout and a retry, and a permanent failure needs to dead-letter rather than spin.
- Duplicate processing costs more. At-least-once delivery now means downloading the video twice. Idempotency should be checked before the fetch, not after, or duplicates are expensive as well as incorrect.
- Access control. The reference must not be a permanently valid public URL. A signed, short-lived URL or an authorised fetch, or the message becomes a capability that leaks through logs.
- Partial uploads. A multipart upload that failed midway can leave an incomplete object that looks present. The checksum in the message is what catches this, and it is worth carrying for that reason alone.
On the slow consumers, which is the second half of their complaint and is not fixed by claim check: scale with competing consumers, and partition by video id if any per-video ordering matters — a "processing started" message must not overtake "upload complete". If processing stages differ wildly in cost, separate queues per stage let each scale independently rather than one pool doing everything.
Design a scatter–gather for a product page that needs pricing, inventory, reviews and recommendations from four services. Specify the timing, failure and caching rules.
Write your own answer first, on paper or in a note. Then compare.
Show a hint
Classify the four by what the page cannot be shown without.
Show a strong answer
Classify first, because everything else follows from it:
- Pricing: required. A product page with no price is not a product page, and showing a stale or missing price is a commercial problem.
- Inventory: required-ish. Without it the page cannot show availability or a working buy button. I would treat it as required but with a defined fallback — show the page with availability marked as unknown and the buy action deferring the check — rather than failing the whole page.
- Reviews: optional. Absent, the section is hidden. Nobody's purchase is blocked.
- Recommendations: optional. The most expendable of the four.
Timing:
- One deadline for the whole gather, set from the page's latency budget — say 300 ms — not four independent timeouts. With per-request deadlines the page waits for the slowest of four tails, which is far worse than any one service's tail.
- Return as soon as the required responses are in, without waiting for the optional ones. Optional results that arrive late are simply not used; if the client can accept them, load them separately after first paint, which is usually the better product answer anyway.
- Cancel outstanding requests at the deadline, or four services keep working on results nobody will read — capacity spent to make the next request slower.
Failure:
- Pricing fails: the page fails. An error is better than a page with a wrong or missing price, and this should be an explicit decision rather than an accident of code structure.
- Inventory fails: degrade explicitly. Availability unknown, purchase path re-checks. Do not default to "in stock" — an optimistic default here converts a service failure into oversold orders.
- Optional services fail: omit the section, count the failure, and let the circuit breaker do its job. Silent omission with no metric is how a permanently broken recommendations service goes unnoticed for months.
- Label the response as partial, so nothing downstream caches or persists a degraded page as a complete one.
Caching:
- Cache per component, not per page. Reviews change hourly, prices change often, inventory changes constantly. One page-level cache entry is forced to the shortest of those, which throws away almost all the benefit.
- Never cache a degraded response, or one outage is served for the cache's lifetime after the service recovers — a failure that outlives its cause and is confusing to diagnose.
- Cache optional components more aggressively. Stale recommendations are fine; stale prices are not.
What I would monitor: the gather's deadline-hit rate per service. A service that is regularly cancelled at the deadline is effectively absent from the page, and nobody will notice from success metrics alone.
Key takeaways
- Competing consumers scale processing and cost ordering; partition by key to get ordering where it matters and parallelism elsewhere.
- Cheaper than enforcing order: make handlers order-independent by carrying full state rather than deltas.
- Claim check keeps large payloads out of messages — store first, publish second, and keep object retention longer than the maximum message age including dead-letter time.
- Scatter–gather is a set of timing and failure decisions: one deadline for the whole gather, required versus optional responders, cancellation, and a partial-result flag.
- Cache scatter–gather components separately, and never cache a degraded response.
- Routers route and translators translate. Business decisions in either are untestable and invisible from the domain.
- Every pattern here runs on at-least-once delivery, so idempotency is a prerequisite rather than an enhancement.
Further reading
- Gregor Hohpe and Bobby Woolf, Enterprise Integration Patterns (Addison-Wesley, 2003) — the source for competing consumers, claim check, scatter–gather, routers and translators.
- Chris Richardson, Microservices Patterns (Manning, 2018) — API composition and messaging.
- Martin Kleppmann, Designing Data-Intensive Applications (O'Reilly, 2017) — partitioned logs and ordering guarantees.
- Jeffrey Dean and Luiz André Barroso, "The Tail at Scale" (Communications of the ACM, 2013) — why fan-out multiplies tail latency.
- Michael Nygard, Release It!, second edition (Pragmatic Bookshelf, 2018).