# Comprehensions

## 1. List Comprehensions

```python
squares = [x ** 2 for x in range(10)]
# [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

evens = [x for x in range(20) if x % 2 == 0]
# [0, 2, 4, ..., 18]

pairs = [(x, y) for x in range(3) for y in range(2)]
# [(0,0), (0,1), (1,0), (1,1), (2,0), (2,1)] — left-to-right nesting, same as nested for loops
```

A comprehension is a concise expression for building a new list from an iterable, optionally filtering (`if`) and optionally nesting multiple `for` clauses (equivalent to nested loops, outermost first).

## 2. Set and Dict Comprehensions

```python
unique_lengths = {len(word) for word in ["a", "bb", "cc", "ddd"]}
# {1, 2, 3} — duplicates collapse, like any set

word_lengths = {word: len(word) for word in ["a", "bb", "ccc"]}
# {"a": 1, "bb": 2, "ccc": 3}
```

Same syntax pattern, different brackets/delimiter: `{...}` with `:` is a dict comprehension; `{...}` without `:` is a set comprehension; `[...]` is a list comprehension.

## 3. Generator Expressions — Lazy, Not Eager

```python
gen = (x ** 2 for x in range(1_000_000))  # parentheses, not brackets
print(type(gen))  # <class 'generator'>

total = sum(x ** 2 for x in range(1_000_000))  # parens can be dropped when it's the sole argument
```

A generator expression looks like a list comprehension but **does not build the whole collection in memory**. It produces values one at a time, on demand, as something iterates over it. For a one-time pass over large or unbounded data (summing, searching for the first match, feeding into another iterator), this is far more memory-efficient than building an intermediate list just to throw it away.

## 4. Conditional Expression Inside a Comprehension (Not a Filter)

```python
labels = ["even" if x % 2 == 0 else "odd" for x in range(5)]
# ["even", "odd", "even", "odd", "even"]
```

This is different from the filtering `if` in section 1 — a trailing `if...else` **before** the `for` is a ternary expression that transforms every element; a trailing `if` **after** the `for` (with no `else`) filters which elements are included at all.

```python
[x if x % 2 == 0 else -1 for x in range(5)]  # transform: [0, -1, 2, -1, 4]  (len == 5)
[x for x in range(5) if x % 2 == 0]           # filter:    [0, 2, 4]         (len == 3)
```

## 5. Nested Comprehensions

```python
matrix = [[1, 2, 3], [4, 5, 6], [7, 8, 9]]
flattened = [n for row in matrix for n in row]
# [1, 2, 3, 4, 5, 6, 7, 8, 9]

transposed = [[row[i] for row in matrix] for i in range(3)]
# [[1, 4, 7], [2, 5, 8], [3, 6, 9]]
```

Read multi-`for` comprehensions left to right, exactly like the equivalent nested `for` loops — the leftmost `for` is the outermost loop.

## 6. When *Not* to Use a Comprehension

```python
# hard to read — nested logic, side effects, multiple conditions
result = [transform(x) for x in data if validate(x) if x.status == "active" for y in x.children if y.enabled]

# clearer as a regular loop
result = []
for x in data:
    if not validate(x) or x.status != "active":
        continue
    for y in x.children:
        if y.enabled:
            result.append(transform(x))
```

A comprehension should read like a sentence. Once it needs more than one `if`/`for` clause or the logic inside stops being a simple expression, a plain loop is more maintainable — "can I write it as a comprehension" and "should I" are different questions.

## 7. Under the Hood: Comprehensions Compile to Their Own Function

### A comprehension gets its own code object and its own scope

Since Python 3, every list/set/dict comprehension and generator expression is compiled as an **implicit nested function** with its own code object — not inlined into the enclosing function's bytecode. This is exactly why the comprehension's loop variable doesn't leak into the enclosing scope (Variables and Scope topic, section 6):

```python
def demo():
    x = "outer"
    result = [x for x in range(3)]
    return x  # still "outer" — the comprehension's x lived in its own frame

print(demo())  # "outer"
```

At the bytecode level, `dis.dis` on a function containing a list comprehension shows a `MAKE_FUNCTION` for the comprehension's own code object, followed by a `CALL` that immediately invokes it with the outer iterable as its argument — the comprehension body runs as a genuine (if invisible) function call, complete with its own local variable slots.

![A list comprehension compiles to its own hidden function, called immediately with the source iterable as its argument](diagrams/01-comprehension-as-function.svg)

### List/set/dict comprehensions still build the whole thing eagerly

Despite having their own scope like a generator expression, `[...]`, `{...}`, and `{k: v}` comprehensions still **eagerly** iterate the entire source and build the complete result object before returning it — the "own scope" optimization is unrelated to laziness. Only the parenthesized generator expression form is lazy.

### Generator expressions compile to an actual generator function

```python
gen = (x * 2 for x in range(3))
print(type(gen))          # generator
print(gen.__class__.__name__)  # 'generator'
```

A generator expression's hidden function body doesn't `return` a completed collection — it contains `yield`, making the hidden function a generator function. Calling it doesn't run any of the loop body immediately; it returns a suspended generator object (see the Generators and Iterators topic for exactly how frame suspension works), and each element is only computed when something calls `next()` on it. This is the real mechanical reason `sum(x**2 for x in huge_range)` never materializes a million-element list: nothing eagerly builds anything, the generator produces exactly one value per `next()` call, consumed and discarded by `sum` as it goes.

## 8. Common Pitfalls

1. **Overusing nested comprehensions until they're unreadable** — if you need more than one `for` or more than one `if`, strongly consider a regular loop instead (section 6).
2. **Confusing the filtering `if` with the ternary `if/else`** — `[x if cond else y for x in xs]` keeps every element (transformed); `[x for x in xs if cond]` drops elements. Mixing them up silently changes the result length.
3. **Building a full list comprehension just to sum/any/all/max it once** — use a generator expression instead (`sum(x for x in xs)`, not `sum([x for x in xs])`) to avoid the wasted intermediate list.
4. **Re-using a generator expression** — once exhausted, a generator cannot be restarted; `gen = (x for x in range(3)); list(gen); list(gen)` gives `[0,1,2]` then `[]`. A list comprehension doesn't have this problem since it's a real, re-readable list.

## 9. Quick Reference

```python
[expr for x in iterable]                 # list comprehension — eager
{expr for x in iterable}                 # set comprehension — eager, dedups
{k: v for x in iterable}                 # dict comprehension — eager
(expr for x in iterable)                 # generator expression — LAZY
[expr for x in iterable if cond]          # filter
[a if cond else b for x in iterable]       # transform (ternary), no filtering
[y for row in matrix for y in row]          # flatten nested — left-to-right = outer-to-inner
```