# Streams — Part 1: Types, Reading, and Piping

## 1. Why Streams Exist

Loading an entire file (or HTTP response body) into memory before processing it works fine for small data, but falls apart for large or unbounded data — a multi-gigabyte file, or a never-ending data feed. Streams process data **incrementally**, in chunks, keeping memory usage bounded regardless of the total size.

```js
// CommonJS
const fs = require("fs");

// Bad for large files: holds the ENTIRE file in memory at once
fs.readFile("huge.log", (err, data) => { /* data is one giant Buffer */ });

// Good: processes the file in small chunks, memory stays flat
fs.createReadStream("huge.log").on("data", (chunk) => { /* chunk is small */ });
```

## 2. The Four Stream Types

| Type | Direction | Example |
|---|---|---|
| **Readable** | source of data, you read from it | `fs.createReadStream`, an incoming HTTP request body |
| **Writable** | destination for data, you write to it | `fs.createWriteStream`, an outgoing HTTP response |
| **Duplex** | both readable AND writable, independently | a TCP socket (`net.Socket`) |
| **Transform** | a Duplex where output is derived from input | `zlib.createGzip()`, a CSV-to-JSON converter |

All four extend Node's `EventEmitter` (full mechanics in the Events/EventEmitter topic) — streams communicate progress and completion via events (`'data'`, `'end'`, `'error'`, `'finish'`, `'drain'`).

## 3. Reading From a Readable Stream

```js
const fs = require("fs");
const readable = fs.createReadStream("data.txt", { encoding: "utf8" });

readable.on("data", (chunk) => {
  console.log("got chunk:", chunk.length, "bytes");
});
readable.on("end", () => {
  console.log("done reading");
});
readable.on("error", (err) => {
  console.error("stream error:", err);
});
```

Attaching a `'data'` listener switches the stream into **flowing mode** — more on paused vs flowing mode in Part 2.

## 4. Writing To a Writable Stream

```js
const fs = require("fs");
const writable = fs.createWriteStream("out.txt");

writable.write("first line\n");
writable.write("second line\n");
writable.end("last line\n"); // signals no more data will be written
writable.on("finish", () => console.log("all data flushed to disk"));
```

`write()` returns a `boolean` — this return value is the entire foundation of backpressure, covered in Part 2. `end()` both writes a final chunk (optional) and signals the stream is done.

## 5. `.pipe()` — Connecting a Readable to a Writable

```js
const fs = require("fs");

fs.createReadStream("input.txt")
  .pipe(fs.createWriteStream("output.txt"));
```

`.pipe()` handles reading from the source, writing to the destination, AND backpressure management automatically (mechanics in Part 2). It's the idiomatic way to move data between streams without manually wiring `'data'`/`write()` yourself.

```js
// piping through a Transform — e.g. gzip a file while copying it
const fs = require("fs");
const zlib = require("zlib");

fs.createReadStream("input.txt")
  .pipe(zlib.createGzip())
  .pipe(fs.createWriteStream("input.txt.gz"));
```

Chaining `.pipe()` calls builds a pipeline where each stage processes data as it flows through, without any stage needing the whole payload in memory at once.

## 6. Transform Streams

```js
const { Transform } = require("stream");

const upperCaseTransform = new Transform({
  transform(chunk, encoding, callback) {
    // `this.push()` sends data downstream; `callback()` signals this chunk is done
    this.push(chunk.toString().toUpperCase());
    callback();
  },
});

process.stdin.pipe(upperCaseTransform).pipe(process.stdout);
```

A Transform stream's `transform()` method is called once per incoming chunk; it can push zero, one, or multiple output chunks per input chunk, making it the natural building block for parsers, compressors, and encoders in a pipeline.

## 7. Object Mode

```js
const { Readable } = require("stream");

const objectStream = new Readable({
  objectMode: true, // chunks are arbitrary JS values, not just Buffers/strings
  read() {
    this.push({ id: 1, name: "Ana" });
    this.push({ id: 2, name: "Bo" });
    this.push(null); // signals end of stream
  },
});

objectStream.on("data", (obj) => console.log(obj)); // { id: 1, name: 'Ana' }, ...
```

By default streams operate on Buffers/strings. `objectMode: true` lets a stream carry arbitrary JavaScript objects instead — common in data-processing pipelines (e.g. streaming rows out of a database query one object at a time).

Continue to `node_streams_theory_2.md` for backpressure internals, paused vs flowing mode, and building custom streams.