# Idempotency

## 1. What Idempotency Actually Means

An operation is **idempotent** if performing it multiple times has the same effect as performing it once. This is a statement about *effect on state*, not about the response body being identical byte-for-byte.

```
DELETE /orders/42   -> first call: order deleted, 204
                    -> second call: order already gone, 404 (or 204 again)
                    -> EITHER response is fine — the STATE is identical after 1 or N calls
```

Idempotency matters because networks are unreliable: a client sends a request, the server processes it successfully, but the response is lost (timeout, connection reset, proxy hiccup) before the client sees it. The client, unable to distinguish "my request never arrived" from "it arrived and succeeded but the reply got lost," has exactly one safe move: **retry**. Idempotency is what makes that retry harmless instead of catastrophic.

## 2. Which HTTP Methods Are Idempotent by Specification

| Method | Idempotent? | Why |
|---|---|---|
| `GET` | Yes | Read-only, no state change |
| `PUT` | Yes | Replaces a resource with a given representation — replacing it again with the same representation is a no-op |
| `DELETE` | Yes | Deleting an already-deleted resource leaves the same end state |
| `HEAD`, `OPTIONS` | Yes | Read-only |
| `PATCH` | **Not guaranteed** | Depends entirely on the patch semantics — see section 3 |
| `POST` | **Not idempotent** | Conventionally *creates* a new resource each time — this is the entire problem this topic addresses |

This table is a **specification promise**, not an automatic property of your code — see Under the Hood for exactly how a naive `PUT` handler can quietly violate it.

## 3. Why `PATCH` Is the Dangerous Middle Case

```json
// Idempotent PATCH — absolute assignment
PATCH /accounts/1  { "status": "active" }
// running this 1 time or 100 times leaves status="active" either way

// NON-idempotent PATCH — relative operation disguised as a patch
PATCH /accounts/1  { "balance_delta": +50 }
// running this once: balance += 50
// running this twice (a retried request): balance += 100 — DOUBLE CHARGED
```

The syntax looks the same (a `PATCH` with a JSON body); the semantics are opposite. This is precisely the case idempotency keys (section 4) exist to fix — you cannot make `balance_delta` idempotent by construction without changing the API, but you *can* make retrying the same `balance_delta` request safe.

## 4. Idempotency Keys

For `POST` (and any operation with delta/relative semantics), the client generates a unique key — typically a UUID — and sends it with the request:

```http
POST /payments
Idempotency-Key: 7b9e6e2c-3f1a-4c9a-9e2b-1a2b3c4d5e6f
Content-Type: application/json

{ "amount_cents": 4999, "account_id": "acct_88" }
```

Server-side contract:

```
1. Look up the key in an idempotency store.
2. If found AND the stored request matches this one -> return the STORED response. Do not re-run the handler.
3. If found AND the stored request does NOT match (same key, different body) -> 422/409, this is a client bug (key reuse).
4. If not found -> run the handler, store {key, request_fingerprint, response, status} atomically with the effect, THEN return the response.
```

```ts
async function handleWithIdempotency(key: string, body: unknown, handler: () => Promise<Response>) {
  const existing = await store.get(key);
  if (existing) {
    if (existing.fingerprint !== fingerprint(body)) {
      throw new ConflictError("idempotency key reused with a different request body");
    }
    return existing.response; // replay — handler never runs again
  }
  return await runInTransaction(async (tx) => {
    const response = await handler();          // the actual side effect: charge card, create order, etc.
    await store.put(tx, key, fingerprint(body), response); // SAME transaction — see Under the Hood
    return response;
  });
}
```

The retried request never reaches the "real" logic a second time — it's intercepted and answered from the store. This is the mechanism that turns a non-idempotent `POST /payments` into something safe to retry blindly.

![A retried POST with the same idempotency key is intercepted before reaching the handler and answered from the stored response instead of re-executing the charge](diagrams/01-idempotency-key-flow.svg)

## 5. Where the Key Comes From and Its Scope

- **Client-generated**, not server-generated — the whole point is the client can retry with the *same* key without a round trip to obtain one first.
- **Scoped**, usually per API credential/account — the same UUID from two different customers must not collide.
- **Given an expiry** — stores typically retain keys for 24 hours to a few days, not forever; a legitimate new payment with an accidentally-reused key from a week ago should succeed, not be replayed as ancient stale state.

```
Idempotency-Key: <uuid>              -- client generates, sends verbatim
Store key = hash(account_id + ":" + idempotency_key)
Store TTL = 24h (typical; document your own)
```

## 6. Natural Idempotency vs. Enforced Idempotency

Two different techniques solve two different shapes of the problem:

**Natural idempotency** — design the operation itself so repeating it is a no-op, no key needed:

```
PUT /users/42/roles/admin     -- "ensure user 42 has role admin" — repeat freely
DELETE /users/42/roles/admin  -- "ensure user 42 lacks role admin" — repeat freely
```

**Enforced idempotency** (idempotency keys) — the operation is inherently a delta or a creation event (`POST /payments`, `POST /orders`) and cannot be redesigned as a "set" operation without losing meaning (you cannot "PUT the state of having charged $50" — the charge *is* the event).

Prefer natural idempotency wherever the domain allows it — it needs no store, no key management, no TTL policy, and works for every client automatically instead of only clients disciplined enough to send keys.

## 7. Client-Side Retry Strategy

Idempotency keys only pay off paired with a sane client retry policy:

```ts
async function postWithRetry(body: unknown, maxAttempts = 3): Promise<Response> {
  const key = crypto.randomUUID(); // ONE key for the whole retry sequence
  for (let attempt = 1; attempt <= maxAttempts; attempt++) {
    try {
      return await fetch("/payments", {
        method: "POST",
        headers: { "Idempotency-Key": key },
        body: JSON.stringify(body),
      });
    } catch (e) {
      if (attempt === maxAttempts) throw e;
      await sleep(backoff(attempt)); // exponential backoff, not immediate retry
    }
  }
  throw new Error("unreachable");
}
```

The key is generated **once outside the loop**, not per attempt — generating a fresh key on each retry defeats the entire mechanism, because the server sees each attempt as a brand-new, never-seen request.

## 8. Under the Hood

### The idempotency record must be written in the SAME transaction as the effect

The single most common bug in a real idempotency implementation is a race between "perform the effect" and "record that the key was used" as two separate operations:

```
BROKEN (two separate steps):
  1. charge the card                     <-- succeeds
  2. crash / process killed / network blip
  3. idempotency record never written
  4. client, having gotten no response, retries with the SAME key
  5. lookup finds nothing (step 3 never happened) -> handler runs AGAIN -> double charge
```

The fix is to make the effect and the idempotency-store write **one atomic unit** — either both commit or neither does:

```sql
BEGIN;
  INSERT INTO charges (id, account_id, amount_cents) VALUES ($1, $2, $3);
  INSERT INTO idempotency_keys (key, account_id, fingerprint, response_body, status_code)
    VALUES ($4, $2, $5, $6, $7);
COMMIT;
```

If the process crashes between the `charge` insert and the `idempotency_keys` insert, the transaction rolls back **both** — there is never a state where the charge exists but the key record doesn't (or vice versa). A retry after a crash at any point before `COMMIT` sees no record and safely re-runs the whole thing exactly once; a retry after `COMMIT` sees the record and replays. This is the entire reason idempotency keys must live in the same datastore/transaction boundary as the business effect — a separate cache (Redis, an in-memory map) that isn't part of the same transaction reintroduces exactly the race it was meant to close, because "write the charge" and "write the key" become two independent points of failure again.

### Distributed/two-phase variant: reserve-then-commit

When the effect and the key store genuinely can't share one transaction (e.g. the charge happens via a third-party payment processor's API call, which cannot be transactional with your own database), the pattern splits into two phases with the key row as the source of truth for *which phase you're in*:

```
1. INSERT idempotency_keys(key, status='processing') -- reserve the key FIRST, before the side effect
2. call the external payment processor                -- the actual non-transactional effect
3. UPDATE idempotency_keys SET status='completed', response=... WHERE key=$1
```

A retry that finds `status='processing'` (not yet `'completed'`) means a previous attempt is still in flight or crashed mid-flight — the safe response is typically to poll/wait briefly rather than either replaying a response that doesn't exist yet or re-calling the external processor and risking a real double charge.

### Fingerprinting the request, not just the key

Storing only the key (not a fingerprint/hash of the request body) opens a subtle hole: a client bug that reuses a key across two *different* logical requests (e.g. a UUID generator seeded incorrectly, or the key hardcoded during testing) causes the second request to silently receive the first request's response instead of an error. Hashing the meaningful part of the request body and comparing it on every lookup converts that class of client bug from silent data corruption into a loud, immediate `409/422` — exactly the guard shown in section 4's step 3.

### Idempotency vs. HTTP caching — a common confusion

Idempotency and caching solve different problems and are sometimes conflated because both involve "the same request returning the same thing." A `GET` is idempotent by specification regardless of caching — caching decides whether the server is even *asked* again; idempotency decides what happens to state if it *is* asked again. An idempotency-key store on a `POST` is not a cache in the HTTP sense (no `Cache-Control`, no shared/public semantics) — it's a private, per-credential replay guard.

## 9. Common Pitfalls

1. **Generating a new idempotency key on every retry attempt.** As shown in section 7, this defeats the mechanism completely — each "retry" looks like a fresh, unique request to the server.
2. **Writing the idempotency record in a separate transaction (or a different datastore) from the effect.** This is the exact race condition detailed in Under the Hood — the two can get out of sync under a crash.
3. **Treating `PATCH` as automatically idempotent.** As section 3 shows, a delta-style `PATCH` body is exactly as dangerous to retry blindly as `POST` — it needs the same key-based protection.
4. **No fingerprint check on key reuse.** Silently replaying a stored response for a *different* request body (because only the key was checked) hides real client bugs as successful-looking responses.
5. **No TTL on stored keys — or too short a TTL.** Forever-retained keys grow the store unboundedly; too-short a TTL lets a legitimately delayed retry (a mobile client reconnecting after being offline for an hour) fall through and double-execute.
6. **Assuming natural idempotency where the domain doesn't have it.** `POST /orders` cannot be made naturally idempotent by wishing — creating "the same order" twice by definition creates two orders unless a key (or a natural dedup field, like a client-supplied order UUID used as the primary key) makes the second attempt a no-op.
7. **Only documenting idempotency keys for payment endpoints.** Any `POST` with a real-world side effect that's expensive or irreversible to duplicate (sending an email, provisioning a resource, charging a card) benefits from the same pattern — payments are just the case everyone remembers.

## 10. Quick Reference

```
Idempotent by spec        GET, PUT, DELETE, HEAD, OPTIONS
Not idempotent by spec    POST         -- and PATCH, conditionally (see below)
PATCH is idempotent       IFF it's an absolute assignment ({"status":"active"})
PATCH is NOT idempotent   IFF it's a delta/relative op ({"balance_delta": +50})

Idempotency-Key: <client-generated UUID>, ONE key per logical retry sequence
Server contract:
  found + fingerprint matches    -> replay stored response, don't re-run handler
  found + fingerprint mismatches -> 409/422, key reuse is a client bug
  not found                      -> run handler; write {key, fingerprint, response}
                                     in the SAME transaction as the effect

Natural idempotency   PUT/DELETE on a well-designed resource — no store needed, prefer this
Enforced idempotency  idempotency-key store — needed for POST / delta PATCH

Distributed effect    reserve key (status=processing) -> call external system -> mark completed
Client retry          generate key ONCE outside the retry loop; exponential backoff
```