# Retry with Exponential Backoff

## 1. Why "Just Retry" Isn't Enough

A transient failure — a dropped connection, a momentary 503, a timeout under brief load — often succeeds on a second attempt. Retrying is the right instinct. But retrying **immediately**, in a tight loop, has a specific failure mode:

```
1,000 clients get a 503 from an overloaded service at the same instant
1,000 clients immediately retry, at the same instant
The service, already overloaded, now receives another simultaneous
  spike of 1,000 requests — often WORSE than the original load
The retry attempt fails too -> immediate re-retry -> repeat
```

This is a **retry storm**: a herd of clients synchronized by the same failure event, hammering a struggling service in lockstep, at the exact moment it can least afford it. A naive retry loop doesn't just fail to help — it actively prevents the service from recovering, because every brief improvement in capacity is immediately consumed by the next synchronized wave.

## 2. Exponential Backoff: Growing the Delay Between Attempts

Instead of retrying immediately, wait — and make each subsequent wait **longer than the last**, growing exponentially with the attempt number:

```
delay(attempt) = baseDelayMs * 2^attempt

attempt 0 (first retry):  100ms  *  1  =   100ms
attempt 1:                 100ms  *  2  =   200ms
attempt 2:                 100ms  *  4  =   400ms
attempt 3:                 100ms  *  8  =   800ms
attempt 4:                 100ms  * 16  =  1600ms
```

```ts
function backoffDelayMs(attempt: number, baseDelayMs: number): number {
  return baseDelayMs * Math.pow(2, attempt);
}
```

The exponential growth means a client that's failing repeatedly backs off increasingly aggressively — a few fast retries for a truly transient blip, but a rapidly widening gap if the failure persists, so a genuinely struggling service isn't kept under sustained retry pressure from any single client.

## 3. A Maximum Delay Cap — Exponential Growth Is Unbounded Otherwise

Left alone, `2^attempt` grows without limit — after enough failed attempts the "delay" becomes hours, which is rarely the intent. Real implementations cap the delay:

```ts
function cappedBackoffDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  return Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
}
```

Alongside a delay cap, retry logic almost always also caps the **number of attempts** — at some point, continuing to retry (even slowly) stops being useful and the caller needs to fail loudly instead of hanging indefinitely.

## 4. The Problem Exponential Backoff Alone Doesn't Solve: Synchronized Retries

Exponential backoff spreads out *one client's* retries over time — but it does nothing to desynchronize **many clients that all failed at the same instant**. If 1,000 clients all compute `backoffDelayMs(0, 100) = 100ms`, they all retry again at exactly the same moment, just 100ms later than the first storm. The storm didn't go away, it just got rescheduled.

![Without jitter, 1000 clients that failed together compute the identical delay and retry together again; jitter spreads each client's retry to a random point inside the window, smoothing the retry rate over time](diagrams/01-jitter-desync.svg)

## 5. Jitter: Randomizing the Delay to Desynchronize Clients

**Jitter** adds randomness to the computed delay so that clients which failed simultaneously don't retry simultaneously:

```ts
// Full jitter: pick a RANDOM delay anywhere between 0 and the computed cap
function fullJitterDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  const cap = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
  return Math.random() * cap;
}

// Equal jitter: always wait AT LEAST half the computed delay, randomize the rest
function equalJitterDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  const cap = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
  return cap / 2 + Math.random() * (cap / 2);
}
```

- **Full jitter** spreads retries across the widest possible window (`[0, cap]`), which minimizes total load on the recovering service most effectively, at the cost of some clients retrying almost immediately (by chance) and some waiting nearly the full cap.
- **Equal jitter** guarantees a minimum wait (`cap / 2`) so no client retries *too* soon, while still desynchronizing the herd — a middle ground between "no jitter" (fully synchronized, worst case) and "full jitter" (best desynchronization, more variance in individual wait times).

Both strictly dominate no-jitter exponential backoff for the retry-storm problem; the choice between them is a tuning decision, not a correctness one.

## 6. Retries Must Be Idempotent

Backoff and jitter solve *when* to retry — they say nothing about whether retrying is *safe*. Retrying a request that has a side effect (charge a card, send an email, create a resource) can duplicate that side effect if the original request actually succeeded server-side but the response was lost before the client saw it. Safe retrying requires the operation to be **idempotent** — repeating it produces the same end state as doing it once (see `backend/api-design/06-idempotency` for the full mechanics, e.g. idempotency keys for POST requests that aren't naturally idempotent). Retry logic that assumes safety without checking this is a correctness bug waiting for a network blip to trigger it.

## 7. What Should and Shouldn't Be Retried

Not every failure deserves a retry:

| Failure | Retry? | Why |
|---|---|---|
| Network timeout / connection reset | Yes | Classic transient failure |
| HTTP 503 Service Unavailable | Yes | Usually explicitly transient/overload |
| HTTP 429 Too Many Requests | Yes, honoring `Retry-After` if present | The server is telling you exactly when to come back |
| HTTP 400 Bad Request | No | The request itself is wrong — retrying sends the same wrong request again |
| HTTP 401/403 | No | Retrying without fixing auth just repeats the failure |
| HTTP 404 | No | The resource isn't there; retrying doesn't change that |

Blindly retrying every failure treats permanent errors as if they were transient, wasting attempts (and time before the caller finds out it's hopeless) on failures no amount of waiting will fix.

## Common Pitfalls

1. **Retrying immediately with no delay** — turns a transient blip into a self-inflicted retry storm at the worst possible moment for the struggling service.
2. **Exponential backoff with no jitter** — solves one client's retry pacing but leaves synchronized clients retrying in lockstep; the storm just gets rescheduled to a later, still-synchronized instant.
3. **No maximum delay or attempt cap** — unbounded exponential growth eventually means "retry in a few hours," which is rarely the intended behavior; callers need a defined point where retrying stops and failure is reported.
4. **Retrying non-idempotent operations without an idempotency key** — a lost response after a successful write, followed by a blind retry, can duplicate the side effect.
5. **Retrying errors that will never succeed** (4xx client errors) — wastes time and attempts on failures no delay will fix; only retry failures that are plausibly transient.

## Quick Reference

```
delay(attempt) = min(baseDelayMs * 2^attempt, maxDelayMs)     -- exponential backoff
fullJitter      = random(0, delay)                             -- widest desync, most variance
equalJitter      = delay/2 + random(0, delay/2)                 -- guaranteed floor + desync

Retry storm  = many clients failing together, retrying together, worsening the overload
Jitter fixes = desynchronizes WHEN clients retry, not just how long any one client waits

Retry only transient failures (timeouts, 503, 429) -- never blind-retry 4xx client errors
Retries must be idempotent -- see backend/api-design/06-idempotency
Always cap both the delay AND the number of attempts
```