# Rate Limiting — Part 2: Distribution, Scope, and the Client Contract

Part 1 covered the counting algorithms themselves, all implicitly assuming one process holding the counter in memory. Part 2 covers what breaks once there's more than one server, how to choose what a limit applies *to*, and how a limited client is supposed to find out and react.

## 1. In-Memory Counters Break the Moment There's More Than One Server

Every algorithm in Part 1 assumed a single process's memory as the counter's home. Behind a load balancer with N server instances, each instance tracking its own in-memory counter means the *effective* limit is `N * limit` — a client bounced across instances (or simply lucky with load-balancer routing) can exceed the intended cap entirely, with no single instance ever seeing more than its own local slice of the traffic.

## 2. The Fix: a Shared, Centralized Counter Store

The standard fix is moving the counter out of any one process's memory into a shared store every instance reads and writes — commonly **Redis** (see `databases/caching/01-redis`), chosen for its atomic increment-with-expiry primitives and speed:

```ts
// Every server instance calls this SAME Redis instance -- one true counter, not N local ones
async function allowFixedWindowRedis(redis: RedisClient, key: string, limit: number, windowSeconds: number): Promise<boolean> {
  const count = await redis.incr(key);       // atomic: increment and return the NEW value in one round trip
  if (count === 1) {
    await redis.expire(key, windowSeconds);  // first request in this window sets the TTL
  }
  return count <= limit;
}
```

The atomicity of `INCR` matters specifically because it closes a race two servers could otherwise hit: both reading count=`limit - 1` concurrently, both deciding "still under the limit," both incrementing — allowing `limit + 1` through. A single atomic increment-and-compare, done by Redis itself rather than read-then-write from the application, removes that race entirely.

## 3. Choosing the Limiting Scope

The same algorithm can be applied at different granularities — and the choice changes what kind of abuse it actually prevents:

| Scope | Key shape | Prevents | Doesn't prevent |
|---|---|---|---|
| **Per-user** (authenticated) | `rate-limit:{userId}` | One account hammering the API | An attacker rotating across many accounts/IPs |
| **Per-IP** | `rate-limit:{ip}` | One source hammering the API regardless of auth | Distributed abuse (many IPs); false positives behind shared NAT/corporate proxy |
| **Per-API-key** | `rate-limit:{apiKey}` | One integration exceeding its allotted tier | Nothing beyond that specific credential's scope |
| **Global** | `rate-limit:global` | Total load exceeding what the backend can survive, regardless of source | Any single caller from hogging a *disproportionate* share within that global budget |

Real systems commonly layer **multiple scopes simultaneously** — a global cap protecting total backend capacity, plus a per-user/per-key cap protecting fairness between callers — rather than picking exactly one.

## 4. The Client-Facing Contract: `429` and Response Headers

A rate-limited request must communicate clearly enough that a well-behaved client can back off correctly without guessing:

```http
HTTP/1.1 429 Too Many Requests
Retry-After: 42
X-RateLimit-Limit: 100
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1717000000

{ "error": { "code": "RATE_LIMITED", "message": "Too many requests. Retry after 42 seconds." } }
```

- **`Retry-After`** (a standard HTTP header, not rate-limit-specific) tells the client exactly how long to wait before retrying — this is what lets a client's retry logic (`distributed-systems/fundamentals/06-retry_with_exponential_backoff`) back off correctly instead of guessing or hammering immediately again.
- **`X-RateLimit-Limit` / `-Remaining` / `-Reset`** (a de facto convention, not a single official standard — exact header names vary slightly by API) let a well-behaved client *proactively* throttle itself before ever hitting `429`, by watching `Remaining` approach zero.
- The response body follows the same consistent error shape as every other error (`backend/api-design/09-error_response_design`) — rate limiting isn't a special case that gets its own bespoke error format.

Returning these headers on **every** response (not just `429`s) is what makes proactive client-side throttling possible — a client that only learns its limit status after already being rejected has already wasted the request that triggered the rejection.

## Common Pitfalls

1. **Rate limiting with in-memory state behind a multi-instance deployment.** Section 1's `N * limit` gap is the single most common real-world rate-limiting bug — it works perfectly in local testing (one process) and silently fails in production (many instances).
2. **Read-then-write instead of an atomic increment for a shared counter.** Reintroduces the exact race section 2 describes, even with a centralized store, if the increment isn't atomic.
3. **Only rate limiting by IP.** Misses per-account abuse entirely when many abusive requests share one legitimate corporate NAT IP (false positives), or when a single attacker cycles through many IPs (false negatives) — pair IP-based limiting with per-user/per-key limiting rather than relying on it alone.
4. **Sending `429` with no `Retry-After`.** Forces the client to guess a backoff duration, defeating the entire purpose of a machine-readable retry signal.
5. **Only sending rate-limit headers on the `429` response itself.** Prevents proactive client-side throttling — see section 4's closing point.

## Quick Reference

```
Single-process counter  -- breaks at N instances: effective limit becomes N * limit
Fix                      -- shared store (Redis), ATOMIC increment-and-check (INCR), not read-then-write

Scopes: per-user, per-IP, per-API-key, global -- often layered, not exclusive

429 Too Many Requests
  Retry-After: <seconds>              -- machine-readable backoff signal
  X-RateLimit-Limit/-Remaining/-Reset -- send on EVERY response, enables proactive throttling
  body: same consistent {error:{...}} shape as every other error
```