# LINQ (Language Integrated Query)

## 1. What LINQ Actually Is

LINQ is a set of extension methods (defined mostly on `IEnumerable<T>` in `System.Linq`, and `IQueryable<T>` for providers like EF Core) that let you filter, transform, and aggregate sequences with a consistent, composable API, plus two equivalent syntaxes to write those calls: **method syntax** and **query syntax**.

```csharp
var names = new[] { "Alice", "Bob", "Charlie", "Dave", "Eve" };

// Method syntax
var longNamesMethod = names
    .Where(n => n.Length > 3)
    .OrderBy(n => n)
    .Select(n => n.ToUpper());

// Query syntax — same result
var longNamesQuery =
    from n in names
    where n.Length > 3
    orderby n
    select n.ToUpper();
```

Both produce identical results because query syntax is compiled directly into the equivalent method-syntax chain (see "Under the Hood"). Method syntax can express things query syntax cannot (e.g. `Sum`, `Count`, `Any`, most aggregates), so most C# code uses method syntax by default and reaches for query syntax mainly when a `join`/`group by`-heavy query reads more clearly that way.

## 2. Filtering, Projecting, Ordering

```csharp
var people = new List<(string Name, int Age)>
{
    ("Alice", 30), ("Bob", 25), ("Charlie", 35), ("Dave", 25)
};

var adults = people.Where(p => p.Age >= 18);                 // filter
var names = people.Select(p => p.Name);                       // project (transform each element)
var sorted = people.OrderBy(p => p.Age).ThenBy(p => p.Name);    // multi-key sort, ascending
var descending = people.OrderByDescending(p => p.Age);            // sort descending
```

`Select` is a 1-to-1 transform; `SelectMany` flattens a sequence of sequences into one:

```csharp
var groups = new List<List<int>> { new() { 1, 2 }, new() { 3, 4, 5 } };
var flat = groups.SelectMany(g => g);  // 1, 2, 3, 4, 5
```

## 3. Aggregation

```csharp
int[] nums = { 4, 8, 15, 16, 23, 42 };

int sum = nums.Sum();
double avg = nums.Average();
int max = nums.Max();
int count = nums.Count(n => n > 10);
bool anyBig = nums.Any(n => n > 100);
bool allPositive = nums.All(n => n > 0);
int product = nums.Aggregate(1, (acc, n) => acc * n);  // general-purpose fold/reduce
```

`Aggregate` is LINQ's `reduce` — everything else in this section could theoretically be written in terms of it, but the specialized methods (`Sum`, `Max`, ...) exist because they can be implemented more efficiently for their specific case and read far more clearly.

## 4. Grouping and Joining

```csharp
var byAge = people.GroupBy(p => p.Age);
foreach (var group in byAge)
    Console.WriteLine($"Age {group.Key}: {string.Join(", ", group.Select(p => p.Name))}");

var orders = new[] { (CustomerId: 1, Item: "Book") };
var customers = new[] { (Id: 1, Name: "Alice") };

var joined = orders.Join(
    customers,
    order => order.CustomerId,
    customer => customer.Id,
    (order, customer) => $"{customer.Name} bought {order.Item}");
```

`GroupBy` returns a sequence of `IGrouping<TKey, TElement>` — each group is itself an `IEnumerable<TElement>` with an extra `Key` property. `Join` is an inner join (unmatched elements from either side are dropped); `GroupJoin` performs the LINQ equivalent of a left outer join grouped by key.

## 5. Deferred Execution vs Immediate Execution

```csharp
var query = nums.Where(n => n > 10);   // NOTHING has executed yet — just builds a description
nums[0] = 999;                           // mutate the source AFTER building the query...
foreach (var n in query)                  // ...the mutation IS visible here, because Where() only
    Console.WriteLine(n);                  // runs when you actually enumerate
```

Most LINQ operators (`Where`, `Select`, `OrderBy`, `GroupBy`, ...) are **deferred** — calling them doesn't iterate the source at all; it returns an object that knows *how* to produce results once something actually asks for them (a `foreach`, `.ToList()`, another LINQ call, etc.). `Count()`, `Sum()`, `ToList()`, `ToArray()`, `First()`, and similar are **immediate** — they force full or partial enumeration right away and return a concrete value/collection.

## 6. Materializing Results

```csharp
List<int> list = nums.Where(n => n > 10).ToList();   // executes NOW, snapshots into a List<int>
int[] array = nums.Where(n => n > 10).ToArray();
Dictionary<int, string> dict = people.ToDictionary(p => p.Age, p => p.Name);  // throws on duplicate keys!
```

Call `.ToList()`/`.ToArray()` when you need to: (a) enumerate the results more than once without re-running the whole query pipeline, (b) freeze the results against later mutation of the source, or (c) hand the data to an API expecting a concrete collection type.

## Under the Hood: Query Syntax Desugaring and Deferred Execution's Iterator Machinery

### Query syntax is pure sugar for method chains

The C# compiler translates `from`/`where`/`select`/`orderby`/`group by`/`join` clauses into calls to `Where`, `Select`, `OrderBy`, `GroupBy`, `Join`, etc. — this happens entirely at compile time, before any type checking of LINQ-to-Objects vs LINQ-to-SQL/EF even occurs. This translation is mechanical and can be reproduced by hand:

```csharp
from n in names where n.Length > 3 orderby n select n.ToUpper();
//  desugars to:
names.Where(n => n.Length > 3).OrderBy(n => n).Select(n => n.ToUpper());
```

Because this is a textual, compile-time rewrite, `orderby ... select` query-syntax expressions ALWAYS have a method-syntax equivalent, but the reverse isn't true — many method-only operators (`Sum`, `Aggregate`, `Any`, `SelectMany` with certain overloads) have no query-syntax keyword at all, which is exactly why mixed method+query syntax in the same expression is common and completely normal.

### Extension methods are how this attaches to `IEnumerable<T>` at all

`Where`, `Select`, and friends are `static` methods on the static class `System.Linq.Enumerable`, written as C# extension methods (`this IEnumerable<TSource> source` as the first parameter). `names.Where(...)` compiles to `Enumerable.Where(names, ...)` — there's no special language feature giving `IEnumerable<T>` these methods; any type implementing that one interface gets the entire LINQ surface for free, purely through extension-method resolution, as long as `using System.Linq;` is in scope.

![Where and Select are nested iterator state machines; MoveNext calls pull right-to-left, one element at a time, only when foreach asks](diagrams/01-deferred-execution.svg)

### Deferred execution is implemented with C# iterator methods (`yield return`)

`Where` isn't implemented by eagerly looping over the source and building a result list. It's implemented (conceptually) as an iterator method:

```csharp
public static IEnumerable<T> Where<T>(this IEnumerable<T> source, Func<T, bool> predicate)
{
    foreach (var item in source)
        if (predicate(item))
            yield return item;
}
```

When the C# compiler sees `yield return` inside a method, it doesn't compile that method's body to run when called — it generates an entire compiler-synthesized **state machine class** implementing `IEnumerable<T>`/`IEnumerator<T>`, whose `MoveNext()` method contains the loop body, resuming from wherever it last yielded each time `MoveNext()` is called. Calling `Where(...)` just constructs and returns one of these state machine objects immediately — none of the actual filtering logic runs until something calls `MoveNext()` on it (which `foreach`, `.ToList()`, etc. all do internally). This is the entire mechanism behind deferred execution: a LINQ query is a *chain of nested iterator state machines*, each one pulling from the one before it, lazily, one element at a time, only when the outermost consumer asks for the next element.

```
foreach over query
  → outer iterator.MoveNext()
    → pulls from Select's iterator.MoveNext()
      → pulls from Where's iterator.MoveNext()
        → pulls from the actual source's enumerator.MoveNext()
```

This pull-based chaining is why `.Where(...).Select(...).First()` on a huge sequence can be dramatically cheaper than eagerly filtering and projecting everything: `First()` only pulls elements one at a time until it finds one, potentially processing a tiny fraction of the source before stopping, rather than running the whole pipeline to completion up front.

### Why `IQueryable<T>` is a completely different execution model

`IEnumerable<T>` LINQ (LINQ-to-Objects) runs your lambda delegates directly, in-process, against in-memory data. `IQueryable<T>` (used by EF Core, LINQ-to-SQL) instead builds an **expression tree** — a data structure literally representing the *syntax* of your lambda (not a compiled delegate) — and hands that tree to a query provider, which translates it into SQL (or another target query language) and executes it remotely. This is why the exact same LINQ method-syntax code can mean "run this C# in a loop" or "generate and run this SQL query" depending purely on whether the source is `IEnumerable<T>` or `IQueryable<T>` — the compiler picks between `Enumerable.Where` and `Queryable.Where` based on the static type, and `Queryable.Where` takes an `Expression<Func<T,bool>>` instead of a plain `Func<T,bool>`, forcing the compiler to build an expression tree instead of a delegate at the call site.

## Common Pitfalls

1. **Multiple enumeration of a deferred query**, e.g. calling `.Count()` then `.ToList()` on the same un-materialized query — each enumeration re-runs the entire pipeline from scratch (re-querying a database twice, for example). Materialize once with `.ToList()`/`.ToArray()` if you need the results more than once.
2. **Assuming a query executes at the point it's declared** — `var query = source.Where(...)` does not touch `source` yet; if `source` mutates before the query is enumerated, the mutation is visible, which can be surprising.
3. **`.ToDictionary()` throwing `ArgumentException` on duplicate keys** — unlike a plain loop with `dict[key] = value` (which silently overwrites), `ToDictionary` throws if the key selector produces a duplicate; use `GroupBy` or `ToLookup` when duplicates are expected.
4. **Using `.Count() > 0` instead of `.Any()`** — `Count()` on a plain `IEnumerable<T>` may have to enumerate the entire sequence to count it, while `Any()` stops at the first matching (or first, if no predicate) element; on `ICollection<T>`-backed sequences `Count()` is O(1) via a cached count, but that's not guaranteed for a general `IEnumerable<T>` from a deferred query.
5. **Capturing a mutable loop variable in a LINQ lambda inside a loop** and being surprised by closure-over-variable behavior — modern C# (post C# 5 `foreach`) allocates a fresh loop variable per iteration for `foreach`, but a manually indexed `for` loop with a shared variable can still bite you if captured across deferred queries evaluated later.
6. **Silently changing `IEnumerable` to `IQueryable` performance characteristics by calling `.ToList()` too early** — materializing before applying further `Where`/`Select` operators forces the ENTIRE table/source to be pulled into memory first, defeating server-side filtering that would otherwise translate to a `WHERE` clause in SQL.

## Quick Reference

```
Where(predicate)          → filter
Select(selector)           → project / transform
SelectMany(selector)        → flatten nested sequences
OrderBy / OrderByDescending  → sort (ThenBy/ThenByDescending for secondary keys)
GroupBy(keySelector)          → bucket into IGrouping<TKey, TElement>
Join / GroupJoin                → inner join / left-outer-style join
Sum/Average/Min/Max/Count        → aggregate, IMMEDIATE execution
Any/All                            → short-circuiting boolean checks
Aggregate(seed, func)                → general fold/reduce
First/FirstOrDefault/Single/SingleOrDefault → pick one element, IMMEDIATE
ToList/ToArray/ToDictionary/ToLookup    → materialize, IMMEDIATE execution

Deferred (build a pipeline, run on enumeration): Where, Select, OrderBy, GroupBy, Join
Immediate (run right now, return a value):        ToList, Count, Sum, First, Any

IEnumerable<T> → runs delegates in-process (LINQ-to-Objects)
IQueryable<T>  → builds an expression tree, translated by a provider (e.g. to SQL)
```