Buffers

1What a Buffer Actually Is

A Buffer is a fixed-length chunk of raw binary data — a sequence of bytes (integers 0-255), with no concept of encoding until you explicitly ask for one. JavaScript strings are UTF-16 text; a Buffer is neither text nor any particular format — it's the same kind of raw memory you'd manipulate in C, exposed to JS.

const buf = Buffer.from("hi");
console.log(buf);          // <Buffer 68 69>  — the raw bytes of "hi" in UTF-8
console.log(buf.length);   // 2 — byte length, NOT necessarily character count
console.log(buf[0]);       // 104 — indexing gives you the raw byte value

Buffer predates JavaScript's own TypedArray/ArrayBuffer support; today Buffer is implemented as a subclass of Uint8Array, so every TypedArray method (.slice(), .map() via array-like iteration, etc.) works on it too, plus Node adds buffer-specific conveniences (.toString(), .write(), numeric read/write helpers).

2Creating Buffers

Buffer.alloc(4);              // <Buffer 00 00 00 00> — zero-filled, SAFE
Buffer.alloc(4, 1);           // <Buffer 01 01 01 01> — zero-filled then filled with 1
Buffer.allocUnsafe(4);        // <Buffer ?? ?? ?? ??> — UNINITIALIZED memory, FAST but may contain old data
Buffer.from("hi", "utf8");    // from a string, with an explicit (or default) encoding
Buffer.from([1, 2, 3]);       // from an array of byte values
Buffer.from(existingBuffer);  // COPIES the bytes into a new buffer

3Reading and Writing Numeric Data

Buffers store raw bytes, but binary protocols (network packets, file formats) usually encode multi-byte integers in a specific byte order (endianness). Node's numeric read/write methods handle that explicitly:

const buf = Buffer.alloc(4);

buf.writeUInt32BE(1, 0);   // big-endian: most significant byte first
console.log(buf);          // <Buffer 00 00 00 01>

buf.writeUInt32LE(1, 0);   // little-endian: least significant byte first
console.log(buf);          // <Buffer 01 00 00 00>

console.log(buf.readUInt32LE(0)); // 1 — read it back out

BE/LE suffixes mean big-endian/little-endian; picking the wrong one against a spec (a file format, a network protocol) silently produces a different number, not an error.

4Strings and Encodings

const buf = Buffer.from("café", "utf8");
console.log(buf.length);              // 5 — "é" takes 2 bytes in UTF-8, not 1 character = 1 byte
console.log(buf.toString("utf8"));    // "café"
console.log(buf.toString("hex"));     // "636166c3a9"
console.log(buf.toString("base64"));  // "Y2Fmw6k="

Buffer.byteLength("café", "utf8");    // 5 — byte length WITHOUT allocating a buffer

"café".length (a JS string) is 4 (character count); buf.length (a Buffer of that same string) is 5 (byte count) — this mismatch between "characters" and "bytes" is the source of a whole category of buffer bugs with multi-byte characters (see Common Pitfalls).

5Slicing: Views, Not Copies

const original = Buffer.from("hello world");
const view = original.subarray(0, 5); // "hello" — but NOT a copy

view[0] = 72; // 'H'
console.log(original.toString()); // "Hello world" — the ORIGINAL changed too

.subarray() (and the deprecated but still-common .slice(), which behaves the same way on Buffers) returns a view into the same underlying memory, not a new copy. Writing through the view mutates the original buffer's bytes directly. This is a deliberate performance choice — avoiding a copy — but it means slices are not safe to hand out if the recipient shouldn't be able to affect the source.

const copy = Buffer.alloc(5);
original.copy(copy, 0, 0, 5); // an ACTUAL copy, into a separate allocation

6Concatenation and Comparison

const a = Buffer.from("foo");
const b = Buffer.from("bar");

Buffer.concat([a, b]);              // <Buffer 66 6f 6f 62 61 72> — new allocation, copies both in
Buffer.concat([a, b], 5);           // explicit total length — truncates/allocates exactly 5 bytes

a.equals(b);                        // false — byte-for-byte comparison
Buffer.compare(a, b);               // -1, 0, or 1 — for sorting buffers

Buffer.concat always allocates a brand-new buffer and copies every input into it — unlike .subarray(), this one IS a real copy, which matters when building up data incrementally from many small chunks (e.g. reassembling stream chunks) since each concat is O(total size).

console.log(Buffer.poolSize); // 8192 — 8KB, the default pool size

const a = Buffer.allocUnsafe(10); // carved from the current pool slab
const b = Buffer.allocUnsafe(10); // carved from right after `a` in the SAME slab
// a and b's underlying memory is adjacent, from the same 8KB block

⚠️ Common Pitfalls

1. Using Buffer.allocUnsafe() and forgetting to overwrite every byte — leftover data from a previous, unrelated buffer's slab region can leak into your buffer; only use it when you're about to fill the entire thing yourself.
2. Slicing a sensitive buffer and handing the slice to untrusted code — .subarray()/.slice() shares memory with the original; mutations through the slice affect the source, and reads through it can expose bytes beyond what you intended if the slice bounds are wrong.
3. Confusing string .length (characters/UTF-16 code units) with Buffer.length (bytes) — multi-byte UTF-8 characters (emoji, accented letters, CJK text) make these numbers diverge; always use Buffer.byteLength(str) when you need a byte count from a string.
4. Picking the wrong endianness (BE vs LE) when reading/writing multi-byte integers against a spec that mandates one — produces a different (wrong) number silently, no error thrown.
5. Building up a large buffer via repeated Buffer.concat in a loop — each call copies everything into a fresh allocation; for many chunks this is O(n²) total copying. Collect chunks in an array and concat once, or use a stream.

📋 Quick Reference

Buffer.alloc(n)          → zero-filled, safe, slightly slower
Buffer.allocUnsafe(n)     → uninitialized, fast, may contain stale data
Buffer.from(str, enc)      → copy bytes from a string/array/another buffer
buf.length                 → byte count (NOT character count for multi-byte text)
buf.toString(enc)           → decode bytes to a string
Buffer.byteLength(str, enc)  → byte count of a string WITHOUT allocating
buf.subarray(start, end)      → VIEW into same memory (mutations affect original)
buf.copy(target, ...)           → actual copy into a separate allocation
Buffer.concat([...])              → new allocation, copies all inputs in
Buffer.poolSize                    → 8192 (8KB) — shared slab size for small allocUnsafe buffers
readUInt32BE/LE, writeUInt32BE/LE     → explicit-endianness numeric access

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: node_buffers_theory.md · generated by tools/build_lessons.py