# Backpressure

## 1. The Problem: A Fast Producer, A Slow Consumer

Any time one part of a system produces work faster than another part can consume it, something has to give. Consider a producer pushing 10,000 messages/second into a queue that a consumer can only drain at 2,000 messages/second:

```
Producer: 10,000 msg/sec  ──▶  [ queue ]  ──▶  Consumer: 2,000 msg/sec

Every second, 8,000 messages that arrived have nowhere to go but pile up.
```

Without any mechanism to slow the producer down, the gap between production rate and consumption rate becomes a permanently growing backlog. **Backpressure** is the general term for a signal that propagates *backward* through a pipeline — from the slow consumer toward the producer — telling upstream stages "slow down, I can't keep up."

## 2. What Happens Without Backpressure

This is not a theoretical concern — it is the direct, mechanical cause of one of the most common production incidents in distributed systems:

```
1. Consumer falls behind producer (even a small, temporary slowdown is enough)
2. The gap between produced and consumed work has nowhere to go but the queue
3. Queue depth grows, unbounded, second after second
4. If the queue lives in memory: the process's memory usage climbs continuously
5. Eventually: OOM (out of memory) kill, or GC pressure so severe latency
   collapses for EVERYONE using that process, not just the slow path
6. The process restarts, the backlog is gone (if it was in-memory) — quietly
   losing every message that had piled up, or the backlog was durable and now
   floods back in on restart, immediately repeating the same overload
```

An unbounded queue is not a safety margin — it's a deferred and *amplified* version of the same failure, because by the time it manifests (an OOM crash), far more work has been lost or needs to be replayed than if the system had simply refused new work earlier, while the problem was still small.

![Without backpressure the queue grows unbounded toward OOM; with backpressure a signal propagates back to slow or reject producers before that happens](diagrams/01-backpressure-signal.svg)

## 3. Backpressure Mechanisms

| Mechanism | How it works |
|---|---|
| **Bounded queue + blocking** | The queue has a max size; once full, `enqueue()` blocks (or the producer's socket write blocks) until space frees up. The producer is naturally slowed to the consumer's pace. |
| **Bounded queue + rejection** | Once full, new items are rejected immediately (e.g. HTTP 429/503) instead of blocking — the producer finds out *now* that it needs to slow down or retry later. |
| **Credit-based flow control** | The consumer explicitly grants the producer "credits" (permission to send N more items); the producer must wait for more credit before sending further. This is how HTTP/2 and gRPC streaming flow control work. |
| **TCP-level backpressure** | TCP itself has a receive window — if the receiving side's buffer fills up, the OS stops accepting more bytes and the sender's writes eventually block. This is backpressure baked into the transport layer, often invisible to application code. |
| **Pull-based consumption** | Instead of the producer pushing, the consumer pulls at its own rate (`dequeue()` / poll). Kafka consumers work this way — a consumer that's behind simply doesn't ask for more, and the broker holds the backlog in its durable log instead of the consumer's memory. |

## 4. Backpressure in Node.js Streams (A Concrete, Familiar Example)

Node's `stream` module implements backpressure explicitly via the return value of `.write()`:

```js
function pipeWithBackpressure(readable, writable) {
  readable.on("data", (chunk) => {
    const canContinue = writable.write(chunk); // returns false if internal buffer is full
    if (!canContinue) {
      readable.pause(); // STOP pulling more data until writable catches up
    }
  });

  writable.on("drain", () => {
    readable.resume(); // writable's buffer has drained — safe to resume
  });
}
```

`writable.write()` returning `false` is exactly the "signal propagating backward" from Section 1 — the writable stream's internal buffer is a stand-in for the slow consumer, and `readable.pause()` is the producer responding to that signal instead of ignoring it. This is precisely why `stream.pipeline()` (which every Node.js codebase should prefer over manual `.on("data")` wiring) exists — it implements this pause/resume dance correctly, including error propagation, so application code doesn't have to hand-roll it.

## 5. Backpressure Across a Distributed Pipeline

The signal has to propagate through every stage, not just one hop, or it stops being effective:

```
Client --HTTP--> API --publish--> Queue --pull--> Worker --query--> Database

If the Database is the slow stage, effective backpressure means:
  Database slow -> Worker's queries queue up -> Worker pulls fewer jobs from Queue
  -> Queue depth grows -> Queue signals (429/full) -> API stops accepting more from Client
  -> Client sees a 429/503 and applies ITS OWN backoff (see Retry topic)
```

A pipeline with backpressure at only one stage (say, the Queue rejects when full) but no propagation further upstream (the API doesn't relay that rejection to the Client, and instead retries the Queue internally in a tight loop) doesn't actually fix the overload — it just moves where the unbounded buffer lives.

## 6. The Tradeoff: Backpressure vs Dropping Work

Backpressure isn't free — blocking or rejecting a producer has real consequences:

- **Blocking** the producer means *its* caller now waits longer too — backpressure can propagate all the way to a human waiting on a spinner, which is sometimes correct (a slow but working system) and sometimes not (a timeout further upstream fires first, and the user sees an error anyway).
- **Rejecting** work means genuine loss (if the caller doesn't retry) or shifted load (if it does) — a 429 response is only useful if something on the other end knows what to do with it (see Retry with Exponential Backoff).
- **Buffering with limits** is the usual middle ground: allow some slack (a bounded queue sized to smooth out brief spikes) but backpressure once that slack is exhausted, rather than either no limit or no slack at all.

## Under the Hood: Backpressure as a Propagating Signal, and the Failure Mode Without It

The precise mechanism worth internalizing: backpressure is not a single component's behavior, it's a **propagating signal** that must survive being relayed through every intermediate stage for the system to actually self-regulate.

### The signal, traced through one concrete hop

```
Consumer's buffer:  [######    ] 60% full  -> still accepting
Consumer's buffer:  [##########] 100% full -> next write blocks or is rejected
                                                     |
                                                     v
                                         Producer's write() call now either:
                                           (a) blocks (synchronous backpressure), or
                                           (b) returns an error/false (explicit backpressure)
                                                     |
                                                     v
                                    Producer must respond: pause producing, or handle the
                                    rejection (retry later, shed load, apply its own backoff)
```

The mechanism only works end-to-end if the producer's response to "blocked or rejected" is to actually slow down (or fail loudly) — a producer that catches the rejection and immediately retries in a tight loop has turned a backpressure signal into a busy-wait that pins CPU while still not making progress, which is often worse than doing nothing.

### Exactly what happens without it: from healthy to OOM, step by step

```
t=0s    queue depth: 0,      producer 10k/s, consumer 2k/s (net +8k/s)
t=1s    queue depth: 8,000   (memory: a few MB, still fine)
t=10s   queue depth: 80,000  (memory: tens of MB, still looks "fine" on a dashboard)
t=60s   queue depth: 480,000 (memory: hundreds of MB to low GB, GC pauses start lengthening)
t=90s   queue depth: 720,000+  process memory approaches the container/heap limit
        -> allocator can't satisfy a new allocation -> OOM killer (or heap OOM) terminates the process
```

The insidious part: for the first 10-60 seconds this looks completely fine from a request-latency dashboard — the *producer* is still fast, requests still return quickly, because "accepting into the queue" and "actually processing" are decoupled. The failure is invisible until the memory limit is hit, at which point it's not a gradual degradation but a hard crash that takes down in-flight work for every request being served by that process, not just the slow-consumer path. This is exactly why backpressure has to be designed in proactively (bounded queues, credit-based flow control) rather than detected reactively (queue-depth alerting) — by the time an alert fires, the fix (restart, scale up) is happening under active memory pressure, which is the worst possible time to do it.

## Common Pitfalls

1. **Using an unbounded in-memory queue "because bounding it might drop messages"** — an unbounded queue doesn't prevent loss, it just converts an early, controlled rejection into a later, uncontrolled crash that loses far more work at once.
2. **Applying backpressure at one stage without propagating it further upstream** — the growing buffer just relocates to whichever stage doesn't relay the signal, instead of actually being resolved.
3. **A producer that retries a rejected write in a tight loop** — turns a backpressure signal into CPU-burning busy-waiting instead of an actual slowdown; needs backoff (see the next topic), not a bare retry.
4. **Monitoring only latency, not queue depth/buffer size** — the OOM-trajectory example shows latency can look completely normal right up until the crash; queue depth is the leading indicator, latency is often the lagging one.
5. **Sizing a bounded buffer arbitrarily instead of based on the downstream consumer's real throughput** — too small causes constant rejection/blocking under normal jitter; too large just delays the same OOM problem instead of preventing it.

## Quick Reference

```
Backpressure = a signal propagating backward: slow consumer -> tells producer to slow down/stop

Mechanisms: bounded queue + block, bounded queue + reject, credit-based flow control,
            TCP receive window, pull-based consumption (consumer controls its own rate)

Without backpressure: queue grows unbounded -> memory climbs -> OOM kill -> crash takes
                       down ALL in-flight work, not just the slow path

Node.js streams: writable.write() returns false -> pause(); 'drain' event -> resume()
Golden rule: backpressure must propagate through EVERY stage, or the buffer just relocates
```