Databases · Caching

Memcached

Memcached is a deliberately simple, multi-threaded, in-memory key-value cache. The servers don't know about each other: clients hash keys to nodes. See what happens when a node dies.

🗄️ Databases● Beginner⏱ 8 min read▶ 2 interactive🛠️ 2 exercises🎤 5 interview Q&As

🎯 What you'll learn

What Memcached is and isn't
The protocol and client usage
The full command surface (get/set/cas/incr)
Client-side distribution and what a node failure means

📚 Before you start

Redis

1What Memcached Is

Memcached is a distributed, in-memory key-value cache — deliberately narrower in scope than Redis. There is exactly one data type: a byte string, up to 1MB by default per value. There are no hashes, lists, sorted sets, or streams; if you need structure, you serialize it (JSON, MessagePack) into a string yourself before storing it, and deserialize on read. This narrowness is the whole design philosophy: Memcached does one job — a fast, simple, volatile key-value cache — and does not try to be a general-purpose data store.

2Basic Protocol and Client Usage

bash

# via the memcached text protocol directly (telnet-style, illustrative)
set user:1:name 0 3600 3
Ana
STORED

get user:1:name
VALUE user:1:name 0 3
Ana
END

delete user:1:name
DELETED

The 0 is client-defined flags (opaque to the server), 3600 is the TTL in seconds, 3 is the byte length of the value that follows.

From Node, a typical client (memjs, illustrated here in ioredis-adjacent style for consistency with the Redis topic):

import memjs from "memjs";
const mc = memjs.Client.create("localhost:11211");

await mc.set("user:1:name", "Ana", { expires: 3600 });
const { value } = await mc.get("user:1:name");        // Buffer("Ana")
await mc.delete("user:1:name");

// atomic increment — requires the value to already be a plain integer string
await mc.set("pageviews", "0");
await mc.increment("pageviews", 1);

3The Command Surface, In Full

Unlike Redis's dozens of type-specific command families, Memcached's entire vocabulary is small:

Command	Purpose
`set`	store (create or overwrite)
`add`	store only if key does NOT already exist
`replace`	store only if key DOES already exist
`get` / `gets`	fetch (with `gets`, also returns a CAS token)
`cas`	compare-and-swap using a `gets` CAS token — the concurrency-safety primitive
`delete`	remove
`incr` / `decr`	atomic integer increment/decrement on a numeric value
`flush_all`	wipe everything (or everything after a delay)
`stats`	server introspection

bash

# optimistic concurrency control via CAS (compare-and-swap)
gets counter
VALUE counter 0 1 42815     # <- the trailing number is the CAS token
0

cas counter 0 3600 1 42815
1
STORED                       # succeeds only if nobody else modified `counter` since our `gets`

This CAS pattern is Memcached's answer to "how do I do a safe read-modify-write without a distributed lock" — conceptually the same idea as Redis's WATCH/MULTI/EXEC optimistic-locking pattern (01-redis/cache_redis_theory.md section 5), just exposed as a single explicit token instead of a transaction block.

4Distribution Is a Client-Side Concern

A Memcached deployment is typically a pool of independent server processes that do not talk to each other or coordinate at all — each instance is just a standalone in-memory hash table. Distributing keys across the pool, and routing each request to the right server, is entirely the client library's job, almost always via consistent hashing over the server list (see 08-distributed_caching/cache_distributed_caching_theory.md for exactly how that hashing works). This is a sharp contrast with Redis Cluster, where the server nodes themselves participate in hash-slot ownership and can redirect a misrouted client.

text

client picks a server for key "user:42" using consistent hashing over [server1, server2, server3]
      -> routes GET/SET directly to that one server
      -> that server has zero awareness of the other two servers' existence

Under the HoodHow it works internally▾

Multi-threaded, not single-threaded

Memcached's architecture is essentially the mirror image of Redis's. Where Redis executes commands on a single thread and relies on that serialization for atomicity (01-redis/cache_redis_theory.md section 7), Memcached runs a fixed pool of worker threads, each with its own event loop (libevent-based), sharing the same in-memory hash table behind fine-grained locking (historically a global lock per bucket range, refined over versions to reduce contention). Each thread accepts and services a subset of client connections independently, in parallel, on separate CPU cores.

This means Memcached scales its raw request throughput on a single box more directly with CPU core count — a workload of purely independent get/set calls on unrelated keys can be serviced by multiple threads truly simultaneously, whereas Redis serves all such requests through the one event-loop thread. The cost is that Memcached needs actual synchronization primitives (locks) around its shared data structures, whereas Redis's single-thread model gets atomicity for free by construction. Neither approach is strictly "faster" — they optimize for different shapes of workload, which is the central subject of 03-redis_vs_memcached/.

Fixed worker-thread pool sharing a locked hash table, backed by fixed-size slab chunk classes

Slab allocation — how it avoids memory fragmentation

Memcached does not call malloc/free per key on every set/delete. Doing so under high churn (millions of small objects constantly created and destroyed) causes severe heap fragmentation over time. Instead, Memcached pre-divides its allocated memory into slabs, and each slab is cut into fixed-size chunks:

text

slab class 1:  chunks of  96 bytes
slab class 2:  chunks of 120 bytes
slab class 3:  chunks of 152 bytes
...            (each class ~1.25x the previous, configurable growth factor)
slab class N:  chunks of  1MB  (the max item size)

When a value comes in, Memcached picks the smallest chunk size class that fits it and stores it in a free chunk from that class. When the item is deleted or evicted, the chunk goes back to that class's free list — it is not returned to a general heap and does not get coalesced with neighboring free space, because chunk sizes within a class are fixed. This trades some memory efficiency (a 100-byte value stored in a 120-byte chunk wastes 20 bytes — "slab overhead") for the elimination of fragmentation and for allocation/deallocation that is a simple free-list pop/push rather than a general allocator call.

Pure LRU eviction — and why it's simpler than Redis's eviction policies

Once a slab class's chunks are all in use and a new item needs to go into that class, Memcached evicts using strict LRU (least-recently-used) within that slab class specifically — it does not evict across classes, and (in the default configuration) it does not consult TTL at all when deciding what to evict under pressure; an item can be evicted before its TTL expires if the slab class is full, and TTL only guarantees it won't be served after expiry, not that it will survive until then. This is a deliberate simplification compared to Redis's selectable maxmemory-policy (allkeys-lru, allkeys-lfu, volatile-ttl, and others) — Memcached offers exactly one eviction algorithm, scoped per slab class, with no configuration knob to change the algorithm itself (only overall memory limits and slab growth factor are tunable).

No persistence, no replication, by design

Memcached has no RDB-equivalent, no AOF-equivalent, and no built-in replication. A restart empties the cache completely — every key is gone, with no recovery mechanism at all. This isn't an oversight; it's consistent with the "pure cache" philosophy: Memcached assumes the source of truth lives elsewhere (a database), and that anything in Memcached is, by definition, reconstructible by re-querying that source of truth on a miss. Building in persistence would add complexity in service of a durability guarantee the tool is explicitly not meant to provide — contrast with Redis, which is sometimes used as a lightweight primary data store precisely because it offers RDB/AOF.

⚠️ Common Pitfalls

Storing values over 1MB — the default max item size will reject them outright; either raise the limit server-side (with real memory cost) or chunk the value client-side.
Assuming TTL guarantees survival until expiry — under memory pressure, LRU eviction inside a full slab class can remove an item before its TTL, since eviction and expiry are two independent mechanisms.
Doing read-modify-write without cas — a plain get then set race-loses updates under concurrent writers; use gets/cas for anything that isn't a blind overwrite or an incr/decr.
Expecting Memcached servers to coordinate or rebalance data — they don't; if you add/remove a server from the pool without consistent hashing on the client, most keys reroute to a different server and effectively become instant cache misses (see 08-distributed_caching/).
Choosing Memcached when you need richer operations — atomic per-field hash updates, sorted-set leaderboards, pub/sub, and durability all require Redis (or an application-side workaround); Memcached's single-string-type model has no equivalent primitive.
Treating a Memcached restart as recoverable — there is no snapshot to reload from; a restart is a full cold cache, with the corresponding thundering-herd risk on downstream systems (see 05-cache_invalidation/).

📋 Quick Reference

text

set k v [ttl]     add k v [ttl]     replace k v [ttl]
get k              gets k (returns CAS token)     cas k v ttl CASTOKEN
delete k           incr k n         decr k n
flush_all          stats

Architecture: fixed thread pool, shared hash table + locks -> true parallel request handling
Memory: slab allocation (fixed size classes) -> no fragmentation, some internal waste
Eviction: strict LRU per slab class only, no policy choice, can evict before TTL
Durability: none — restart = empty cache, no persistence, no replication
Values: single type, opaque byte strings, up to 1MB default

🧠 Key takeaways

Memcached = simple multi-threaded key → blob cache with LRU eviction, no persistence, no replication.
Commands: get/set/add/replace/delete, incr/decr, cas for optimistic updates, TTLs.
Distribution is client-side (consistent hashing); servers are independent.
A node failure loses its keys, so plan database capacity for the resulting misses.

🎤 Interview Q&A

Questions interviewers ask about this topic. Answer out loud first, then open a card to compare.

Q1What is Memcached and what's it best at?core▾

A simple, multi-threaded, in-memory key-value cache with string values, LRU eviction and no persistence. It's excellent for straightforward caching of serialized objects at high throughput.

Follow-ups interviewers ask

What doesn't it have? Rich data types, persistence, replication or pub/sub.

Q2How does Memcached distribute data across servers?core▾

The client does it: it hashes keys (usually with consistent hashing) to pick a server. Servers don't talk to each other; adding a server remaps only part of the keyspace with consistent hashing.

Follow-ups interviewers ask

What happens when a server dies? Its keys are lost (cache misses) and get remapped to other servers.

Q3How does Memcached manage memory?senior▾

With a slab allocator: memory is divided into slab classes of fixed chunk sizes; items go into the smallest fitting chunk. This avoids fragmentation but wastes space when item sizes don't match chunk sizes, and eviction is per slab class.

Follow-ups interviewers ask

What's slab calcification? Memory stuck in slab classes that no longer match the workload.

Q4What's the max item size and how do you handle large values?core▾

1 MB by default (configurable). Avoid caching huge values; compress them, split them, or cache smaller pieces.

Follow-ups interviewers ask

Why avoid huge values? They increase latency and evict many small items.

Q5When would you choose Memcached over Redis?core▾

For pure, simple caching with very high multi-threaded throughput where you don't need data structures, persistence or replication. In practice, most new projects choose Redis for its versatility.

Follow-ups interviewers ask