Retry with Exponential Backoff

1Why "Just Retry" Isn't Enough

A transient failure — a dropped connection, a momentary 503, a timeout under brief load — often succeeds on a second attempt. Retrying is the right instinct. But retrying immediately, in a tight loop, has a specific failure mode:

1,000 clients get a 503 from an overloaded service at the same instant
1,000 clients immediately retry, at the same instant
The service, already overloaded, now receives another simultaneous
  spike of 1,000 requests — often WORSE than the original load
The retry attempt fails too -> immediate re-retry -> repeat

This is a retry storm: a herd of clients synchronized by the same failure event, hammering a struggling service in lockstep, at the exact moment it can least afford it. A naive retry loop doesn't just fail to help — it actively prevents the service from recovering, because every brief improvement in capacity is immediately consumed by the next synchronized wave.

2Exponential Backoff: Growing the Delay Between Attempts

Instead of retrying immediately, wait — and make each subsequent wait longer than the last, growing exponentially with the attempt number:

delay(attempt) = baseDelayMs * 2^attempt

attempt 0 (first retry):  100ms  *  1  =   100ms
attempt 1:                 100ms  *  2  =   200ms
attempt 2:                 100ms  *  4  =   400ms
attempt 3:                 100ms  *  8  =   800ms
attempt 4:                 100ms  * 16  =  1600ms

function backoffDelayMs(attempt: number, baseDelayMs: number): number {
  return baseDelayMs * Math.pow(2, attempt);
}

The exponential growth means a client that's failing repeatedly backs off increasingly aggressively — a few fast retries for a truly transient blip, but a rapidly widening gap if the failure persists, so a genuinely struggling service isn't kept under sustained retry pressure from any single client.

3A Maximum Delay Cap — Exponential Growth Is Unbounded Otherwise

Left alone, 2^attempt grows without limit — after enough failed attempts the "delay" becomes hours, which is rarely the intent. Real implementations cap the delay:

function cappedBackoffDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  return Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
}

Alongside a delay cap, retry logic almost always also caps the number of attempts — at some point, continuing to retry (even slowly) stops being useful and the caller needs to fail loudly instead of hanging indefinitely.

4The Problem Exponential Backoff Alone Doesn't Solve: Synchronized Retries

Exponential backoff spreads out one client's retries over time — but it does nothing to desynchronize many clients that all failed at the same instant. If 1,000 clients all compute backoffDelayMs(0, 100) = 100ms, they all retry again at exactly the same moment, just 100ms later than the first storm. The storm didn't go away, it just got rescheduled.

5Jitter: Randomizing the Delay to Desynchronize Clients

Jitter adds randomness to the computed delay so that clients which failed simultaneously don't retry simultaneously:

// Full jitter: pick a RANDOM delay anywhere between 0 and the computed cap
function fullJitterDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  const cap = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
  return Math.random() * cap;
}

// Equal jitter: always wait AT LEAST half the computed delay, randomize the rest
function equalJitterDelayMs(attempt: number, baseDelayMs: number, maxDelayMs: number): number {
  const cap = Math.min(baseDelayMs * Math.pow(2, attempt), maxDelayMs);
  return cap / 2 + Math.random() * (cap / 2);
}

• Full jitter spreads retries across the widest possible window ([0, cap]), which minimizes total load on the recovering service most effectively, at the cost of some clients retrying almost immediately (by chance) and some waiting nearly the full cap.
• Equal jitter guarantees a minimum wait (cap / 2) so no client retries too soon, while still desynchronizing the herd — a middle ground between "no jitter" (fully synchronized, worst case) and "full jitter" (best desynchronization, more variance in individual wait times).

Both strictly dominate no-jitter exponential backoff for the retry-storm problem; the choice between them is a tuning decision, not a correctness one.

6Retries Must Be Idempotent

Backoff and jitter solve when to retry — they say nothing about whether retrying is safe. Retrying a request that has a side effect (charge a card, send an email, create a resource) can duplicate that side effect if the original request actually succeeded server-side but the response was lost before the client saw it. Safe retrying requires the operation to be idempotent — repeating it produces the same end state as doing it once (see backend/api-design/06-idempotency for the full mechanics, e.g. idempotency keys for POST requests that aren't naturally idempotent). Retry logic that assumes safety without checking this is a correctness bug waiting for a network blip to trigger it.

7What Should and Shouldn't Be Retried

Not every failure deserves a retry:

Blindly retrying every failure treats permanent errors as if they were transient, wasting attempts (and time before the caller finds out it's hopeless) on failures no amount of waiting will fix.

const policy = {
  retryOn: ["ECONNRESET", "ETIMEDOUT", 502, 503, 504, 429],
  never:   [400, 401, 403, 404, 409, 422],
  maxAttempts: 4,
  baseMs: 100, maxDelayMs: 2_000,
  jitter: "full",
  honorRetryAfter: true,           // 429/503 may tell you exactly when
  idempotencyKey: () => `order-${orderId}`,
};

⚠️ Common Pitfalls

1. Retrying immediately with no delay — turns a transient blip into a self-inflicted retry storm at the worst possible moment for the struggling service.
2. Exponential backoff with no jitter — solves one client's retry pacing but leaves synchronized clients retrying in lockstep; the storm just gets rescheduled to a later, still-synchronized instant.
3. No maximum delay or attempt cap — unbounded exponential growth eventually means "retry in a few hours," which is rarely the intended behavior; callers need a defined point where retrying stops and failure is reported.
4. Retrying non-idempotent operations without an idempotency key — a lost response after a successful write, followed by a blind retry, can duplicate the side effect.
5. Retrying errors that will never succeed (4xx client errors) — wastes time and attempts on failures no delay will fix; only retry failures that are plausibly transient.

📋 Quick Reference

delay(attempt) = min(baseDelayMs * 2^attempt, maxDelayMs)     -- exponential backoff
fullJitter      = random(0, delay)                             -- widest desync, most variance
equalJitter      = delay/2 + random(0, delay/2)                 -- guaranteed floor + desync

Retry storm  = many clients failing together, retrying together, worsening the overload
Jitter fixes = desynchronizes WHEN clients retry, not just how long any one client waits

Retry only transient failures (timeouts, 503, 429) -- never blind-retry 4xx client errors
Retries must be idempotent -- see backend/api-design/06-idempotency
Always cap both the delay AND the number of attempts

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: dist_retry_with_exponential_backoff_theory.md · generated by tools/build_lessons.py