# RabbitMQ — Part 2: Prefetch, Durability, Clustering, and Internals

## 1. Prefetch (QoS): Controlling How Much a Consumer Can Hold

By default, RabbitMQ pushes messages to a connected consumer as fast as it can, without waiting for previous messages to be acked. If one consumer is slow and others are idle, the slow one can end up holding a huge backlog of unacked messages while fast consumers starve.

**Prefetch count** (set via `basic.qos`) caps how many unacknowledged messages the broker will have "in flight" to a given consumer at once. `prefetch = 1` means "don't send me a new message until I've acked the one I'm holding" — the simplest way to get even load distribution across a pool of consumers.

```
prefetch = 10:  broker can push up to 10 unacked messages to this consumer
                 before it must wait for an ack before sending an 11th

prefetch = 1:   strict "one at a time" — best load balancing, but more
                network round-trips (lower throughput per consumer)
```

Tuning prefetch is a direct throughput-vs-fairness trade-off: higher prefetch means fewer round trips and higher throughput per consumer, but a slow consumer can hoard a large batch while others sit idle.

## 2. Message and Queue Durability

Two independent settings determine whether a message survives a broker restart:

1. **The queue must be declared `durable`** — its existence/metadata is written to disk.
2. **The message must be published with the `persistent` delivery mode** — its body is written to disk, not just held in memory.

Both are required. A persistent message published to a non-durable queue is lost on restart because the queue itself disappears; a non-persistent message in a durable queue is also lost because the message body was never written to disk.

Durability has a real performance cost — writing to disk on every publish is slower than an in-memory hand-off — so RabbitMQ is usually configured to persist only what actually needs to survive a crash (e.g. order events), while allowing lower-value, high-volume traffic (e.g. live metrics) to skip persistence.

## 3. Dead Lettering (Setup Mechanics)

A queue can be configured with a **dead-letter exchange** (DLX) — when a message in that queue is rejected/nacked without requeue, or expires (TTL), or the queue overflows its length limit, RabbitMQ automatically republishes it to the DLX instead of discarding it. The full mechanics and retry-then-redirect pattern are covered in the dedicated Dead Letter Queues topic; the piece worth knowing here is that this is configured as **queue arguments** at declaration time:

```
queue.declare("orders", {
  arguments: {
    "x-dead-letter-exchange": "orders.dlx",
    "x-dead-letter-routing-key": "orders.failed"
  }
})
```

## 4. Clustering and Quorum Queues

A single RabbitMQ node is a single point of failure. Production deployments run a **cluster** of nodes. Two queue types handle replication differently:

| Queue type | Replication model |
|---|---|
| **Classic queue** (legacy default) | Lives on one node by default; optional mirroring to others was the old HA mechanism, now deprecated. |
| **Quorum queue** (current recommended default) | Uses the **Raft consensus algorithm** to replicate the queue's log across multiple nodes. A write is only acknowledged once a majority (quorum) of replicas have persisted it, giving strong consistency and automatic failover if the leader node dies. |

Quorum queues are the modern recommendation for anything where message durability across a node failure matters — they trade a bit of latency (waiting for a quorum of acks) for real fault tolerance, rather than the "hope the mirror caught up" behavior of legacy mirrored classic queues.

## Under the Hood: The Real AMQP Exchange-and-Binding Model, Precisely

It's worth being exact about what actually happens inside RabbitMQ when a message is published, because it's easy to blur "exchange" and "queue" into one mental blob.

**An exchange is not a buffer.** It holds no messages and has no memory beyond its bindings table. When a producer publishes, the broker looks up every binding registered against that exchange, evaluates the routing key (or headers) against each binding's pattern, and for every match, **copies a reference to the message into that queue's own storage.** If a message matches three queues, three independent copies (or, internally, reference-counted deliveries) exist — acking in one queue has zero effect on the message's presence in the others. This is why the topic-exchange walkthrough in Part 1 could deliver the same message into `eu-orders`, `all-created`, and `audit-log` simultaneously: the exchange isn't choosing one destination, it's evaluating every binding independently on every publish.

**A queue is a genuine, per-queue, mostly-FIFO log with per-message state.** Internally (in the modern Raft-backed quorum queue implementation) each queue keeps its messages in an append-only log, tracks each message's delivery state (unacked, acked, dead-lettered), and only physically reclaims disk space once a message is acked. This looks superficially like Kafka's log — but the crucial difference (spelled out fully in the RabbitMQ vs Kafka topic) is that **RabbitMQ's log is a private implementation detail of one queue that gets consumed and discarded**, not a durable, independently-addressable stream multiple consumer groups replay at their own offsets. Once every consumer has acked a message, RabbitMQ has no reason to keep it, and doesn't.

**Routing decisions happen once, at publish time, not at consume time.** This is the opposite of Kafka, where a message's fate (which partition, and later which offset each consumer group reads) is largely decided by consumers pulling at their own pace. In RabbitMQ, the moment a message is published, the exchange has already fully decided every queue it will land in — a consumer simply drains whatever queue it's attached to. This "smart broker" design is what makes RabbitMQ's routing so flexible (topic patterns, multiple simultaneous destinations) but also means routing logic changes require touching exchange/binding configuration, not consumer code.

## Common Pitfalls

1. **Publishing directly assuming a queue name works like a destination.** There is no way to "publish to a queue" in AMQP — you always publish to an exchange (even the unnamed default exchange, which does direct routing by queue name as a convenience, is still technically an exchange).
2. **Declaring a durable queue but forgetting `persistent: true` on the message** — the queue survives a restart, empty, because the messages inside were never written to disk.
3. **Using prefetch = unlimited (the old default) in a multi-consumer pool** — one consumer can end up starving the others by hoarding every in-flight message.
4. **Assuming a mirrored classic queue gives the same guarantees as a quorum queue** — legacy mirroring can lose unsynced messages on failover; quorum queues use actual majority consensus.
5. **Forgetting that a fanout/topic exchange delivering to multiple queues means multiple independent copies** — acking in one consumer's queue does not remove the message from a sibling queue that also matched.

## Quick Reference

```
Producer -> Exchange -> (binding match) -> Queue -> Consumer -> ack -> deleted

Exchange types: direct (exact key), fanout (all bound queues), topic (* / # patterns), headers
Durability: queue must be durable AND message must be persistent
Prefetch:   caps unacked messages in flight per consumer (1 = strict round robin)
Quorum queue: Raft-replicated, majority-ack durability, modern HA default
Dead letter: queue arg "x-dead-letter-exchange" reroutes rejected/expired messages
```