Backpressure

1The Problem: A Fast Producer, A Slow Consumer

Any time one part of a system produces work faster than another part can consume it, something has to give. Consider a producer pushing 10,000 messages/second into a queue that a consumer can only drain at 2,000 messages/second:

Producer: 10,000 msg/sec  ──▶  [ queue ]  ──▶  Consumer: 2,000 msg/sec

Every second, 8,000 messages that arrived have nowhere to go but pile up.

Without any mechanism to slow the producer down, the gap between production rate and consumption rate becomes a permanently growing backlog. Backpressure is the general term for a signal that propagates backward through a pipeline — from the slow consumer toward the producer — telling upstream stages "slow down, I can't keep up."

2What Happens Without Backpressure

This is not a theoretical concern — it is the direct, mechanical cause of one of the most common production incidents in distributed systems:

1. Consumer falls behind producer (even a small, temporary slowdown is enough)
2. The gap between produced and consumed work has nowhere to go but the queue
3. Queue depth grows, unbounded, second after second
4. If the queue lives in memory: the process's memory usage climbs continuously
5. Eventually: OOM (out of memory) kill, or GC pressure so severe latency
   collapses for EVERYONE using that process, not just the slow path
6. The process restarts, the backlog is gone (if it was in-memory) — quietly
   losing every message that had piled up, or the backlog was durable and now
   floods back in on restart, immediately repeating the same overload

An unbounded queue is not a safety margin — it's a deferred and amplified version of the same failure, because by the time it manifests (an OOM crash), far more work has been lost or needs to be replayed than if the system had simply refused new work earlier, while the problem was still small.

3Backpressure Mechanisms

4Backpressure in Node.js Streams (A Concrete, Familiar Example)

Node's stream module implements backpressure explicitly via the return value of .write():

function pipeWithBackpressure(readable, writable) {
  readable.on("data", (chunk) => {
    const canContinue = writable.write(chunk); // returns false if internal buffer is full
    if (!canContinue) {
      readable.pause(); // STOP pulling more data until writable catches up
    }
  });

  writable.on("drain", () => {
    readable.resume(); // writable's buffer has drained — safe to resume
  });
}

writable.write() returning false is exactly the "signal propagating backward" from Section 1 — the writable stream's internal buffer is a stand-in for the slow consumer, and readable.pause() is the producer responding to that signal instead of ignoring it. This is precisely why stream.pipeline() (which every Node.js codebase should prefer over manual .on("data") wiring) exists — it implements this pause/resume dance correctly, including error propagation, so application code doesn't have to hand-roll it.

5Backpressure Across a Distributed Pipeline

The signal has to propagate through every stage, not just one hop, or it stops being effective:

Client --HTTP--> API --publish--> Queue --pull--> Worker --query--> Database

If the Database is the slow stage, effective backpressure means:
  Database slow -> Worker's queries queue up -> Worker pulls fewer jobs from Queue
  -> Queue depth grows -> Queue signals (429/full) -> API stops accepting more from Client
  -> Client sees a 429/503 and applies ITS OWN backoff (see Retry topic)

A pipeline with backpressure at only one stage (say, the Queue rejects when full) but no propagation further upstream (the API doesn't relay that rejection to the Client, and instead retries the Queue internally in a tight loop) doesn't actually fix the overload — it just moves where the unbounded buffer lives.

6The Tradeoff: Backpressure vs Dropping Work

Backpressure isn't free — blocking or rejecting a producer has real consequences:

• Blocking the producer means its caller now waits longer too — backpressure can propagate all the way to a human waiting on a spinner, which is sometimes correct (a slow but working system) and sometimes not (a timeout further upstream fires first, and the user sees an error anyway).
• Rejecting work means genuine loss (if the caller doesn't retry) or shifted load (if it does) — a 429 response is only useful if something on the other end knows what to do with it (see Retry with Exponential Backoff).
• Buffering with limits is the usual middle ground: allow some slack (a bounded queue sized to smooth out brief spikes) but backpressure once that slack is exhausted, rather than either no limit or no slack at all.

Consumer's buffer:  [######    ] 60% full  -> still accepting
Consumer's buffer:  [##########] 100% full -> next write blocks or is rejected
                                                     |
                                                     v
                                         Producer's write() call now either:
                                           (a) blocks (synchronous backpressure), or
                                           (b) returns an error/false (explicit backpressure)
                                                     |
                                                     v
                                    Producer must respond: pause producing, or handle the
                                    rejection (retry later, shed load, apply its own backoff)

t=0s    queue depth: 0,      producer 10k/s, consumer 2k/s (net +8k/s)
t=1s    queue depth: 8,000   (memory: a few MB, still fine)
t=10s   queue depth: 80,000  (memory: tens of MB, still looks "fine" on a dashboard)
t=60s   queue depth: 480,000 (memory: hundreds of MB to low GB, GC pauses start lengthening)
t=90s   queue depth: 720,000+  process memory approaches the container/heap limit
        -> allocator can't satisfy a new allocation -> OOM killer (or heap OOM) terminates the process

⚠️ Common Pitfalls

1. Using an unbounded in-memory queue "because bounding it might drop messages" — an unbounded queue doesn't prevent loss, it just converts an early, controlled rejection into a later, uncontrolled crash that loses far more work at once.
2. Applying backpressure at one stage without propagating it further upstream — the growing buffer just relocates to whichever stage doesn't relay the signal, instead of actually being resolved.
3. A producer that retries a rejected write in a tight loop — turns a backpressure signal into CPU-burning busy-waiting instead of an actual slowdown; needs backoff (see the next topic), not a bare retry.
4. Monitoring only latency, not queue depth/buffer size — the OOM-trajectory example shows latency can look completely normal right up until the crash; queue depth is the leading indicator, latency is often the lagging one.
5. Sizing a bounded buffer arbitrarily instead of based on the downstream consumer's real throughput — too small causes constant rejection/blocking under normal jitter; too large just delays the same OOM problem instead of preventing it.

📋 Quick Reference

Backpressure = a signal propagating backward: slow consumer -> tells producer to slow down/stop

Mechanisms: bounded queue + block, bounded queue + reject, credit-based flow control,
            TCP receive window, pull-based consumption (consumer controls its own rate)

Without backpressure: queue grows unbounded -> memory climbs -> OOM kill -> crash takes
                       down ALL in-flight work, not just the slow path

Node.js streams: writable.write() returns false -> pause(); 'drain' event -> resume()
Golden rule: backpressure must propagate through EVERY stage, or the buffer just relocates

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: dist_backpressure_theory.md · generated by tools/build_lessons.py