# Distributed Transactions — Part 1: Why One ACID Transaction Can't Span Services, and Two-Phase Commit This topic assumes familiarity with ACID transactions in a single database (`databases/fundamentals/03-transactions_and_acid`). Part 1 covers why that single-database guarantee breaks down the moment a business operation spans multiple services/databases, and the classic (largely legacy) answer, two-phase commit. Part 2 covers the pattern that replaced it in most modern distributed systems: the Saga. ## 1. The Problem: One Business Operation, Multiple Databases A single ACID transaction gives you atomicity — every statement inside `BEGIN`/`COMMIT` either all takes effect or none does — but that guarantee is scoped to **one database connection talking to one database**. The moment a business operation needs to update state in two *separate* databases (or two separate services, each owning its own database — the normal shape of a microservice architecture), there is no single `BEGIN`/`COMMIT` that can wrap both: ``` Order Service: BEGIN; INSERT INTO orders (...); COMMIT; -- its own DB Payment Service: BEGIN; UPDATE accounts SET balance = ...; COMMIT; -- a DIFFERENT DB Placing an order needs BOTH to succeed together. But they are two independent transactions against two independent databases -- nothing forces them to succeed or fail as one atomic unit. If the order insert commits and the payment charge then fails, you have an order with no payment. If it's the other way around, you've charged a customer for an order that doesn't exist. ``` This is the core problem distributed transactions address: how do you get an atomicity-like guarantee ("all of these operations across multiple services happen, or none of them do") when there's no single database transaction that can enforce it. ## 2. Two-Phase Commit (2PC): The Classic Answer Two-phase commit introduces a **coordinator** that orchestrates all the participating databases/services (**participants**) through two distinct phases, so that either every participant commits or every participant aborts — never a mix. ``` Phase 1 (PREPARE / VOTE): Coordinator -> every participant: "can you commit this?" Each participant does all the work needed to GUARANTEE it CAN commit if told to (locks acquired, changes written to a durable log, but NOT yet visible/committed) and replies "yes" (vote-commit) or "no" (vote-abort). Phase 2 (COMMIT / ABORT): IF every participant voted yes: Coordinator -> every participant: "COMMIT" -> each makes its prepared changes permanent and visible. IF any participant voted no (or didn't respond): Coordinator -> every participant: "ABORT" -> each rolls back. ``` ``` COORDINATOR PARTICIPANT A PARTICIPANT B │── PREPARE ────────────────▶│ │ │── PREPARE ─────────────────┼───────────────────────▶│ │◀── vote: yes ───────────────│ │ │◀── vote: yes ────────────────────────────────────────│ │ (all voted yes -- safe to commit) │── COMMIT ──────────────────▶│ │ │── COMMIT ───────────────────┼───────────────────────▶│ ``` The critical property Phase 1 buys: once a participant votes "yes," it has made an unconditional promise — it must be able to commit later no matter what, even across its own crash and restart, because the coordinator's final decision might come after a delay. This is why Phase 1 requires participants to durably persist their prepared state before voting yes, not just check that committing *would* be possible. ## 3. 2PC's Fatal Weakness: The Blocking Coordinator Two-phase commit's atomicity guarantee has one very costly dependency: **the coordinator must stay reachable and eventually decide.** Consider what happens if the coordinator crashes after Phase 1 but before sending its Phase 2 decision: ``` Coordinator: sent PREPARE to A and B, both voted yes... Coordinator: CRASHES before sending COMMIT or ABORT to anyone Participant A: has voted yes, has WRITTEN AND LOCKED its prepared changes, and now has NO IDEA whether the final decision was commit or abort -- it cannot safely commit (the coordinator might have decided abort after hearing from B) and cannot safely abort either (the coordinator might have decided commit). A is stuck HOLDING ITS LOCKS, indefinitely, until the coordinator recovers and tells it what happened. ``` This is 2PC's defining weakness: a participant that has voted yes but hasn't yet heard the final decision is **blocked** — it must hold its locks (meaning any other transaction touching that same data is also blocked) until the coordinator recovers, which could be a long time, or forever if the coordinator's crash is unrecoverable. A protocol meant to coordinate distributed work ends up making every participant's availability depend on one coordinator's availability — exactly the kind of single point of failure a distributed system is usually trying to avoid. ![In 2PC, once a participant votes yes it must hold its locks until the coordinator's final decision arrives; if the coordinator crashes before sending that decision, every prepared participant is blocked, unable to safely commit or abort, until the coordinator recovers](diagrams/01-2pc-blocking-coordinator.svg) ## 4. Why 2PC Is Rarely Used Across Microservices Today Beyond the blocking problem, 2PC has practical costs that make it a poor fit for the normal microservice shape (independently-owned databases, services that may be built with different technologies, higher latency between services than within one datacenter): - It requires every participant to support the 2PC protocol (locking + durable prepared state) — most application databases used by microservices (and virtually all external third-party APIs) don't expose this. - Holding locks across the full round-trip of coordinator <-> every participant <-> coordinator, over a network, for the duration of the entire operation, is far slower and more contention-prone than holding a lock for the duration of one local transaction. - It fundamentally trades availability for consistency during a coordinator failure — which is often the opposite of what a microservice architecture is built to prioritize (see the upcoming CAP theorem topic). 2PC still shows up within a single, tightly-coupled system (some message-broker/database combinations use it internally, for instance), but it is rarely the answer for coordinating independently-owned services. Part 2 covers the pattern that took its place. ## Common Pitfalls 1. **Assuming a "distributed transaction" gives the same guarantees as a local one** — 2PC provides atomicity of outcome (all commit or all abort), but at the cost of availability during coordinator failure, which a single-database transaction never has to worry about. 2. **Treating the prepare vote as a "check," not a promise** — once a participant votes yes, it MUST be able to commit later, even after its own crash/restart; this is why prepared state has to be durably persisted, not just verified in memory. 3. **Underestimating the blocking window** — participants holding locks from PREPARE until they hear the final decision isn't a rare edge case, it's the expected steady-state cost of every 2PC transaction, and it directly limits throughput. 4. **Reaching for 2PC as the default answer for "operation spans two services"** — its blocking/availability cost and requirement that every participant implement the protocol make it a poor default fit for typical independently-owned microservice databases; Part 2's Saga pattern is the far more common real-world answer. ## Quick Reference ``` Problem: one business operation spans multiple independently-committed databases/services -- no single BEGIN/COMMIT can wrap all of them atomically 2PC Phase 1 (PREPARE): coordinator asks every participant "can you commit?" -- each participant durably persists its prepared state and votes yes/no 2PC Phase 2 (COMMIT/ABORT): if ALL voted yes -> commit everywhere; if ANY voted no -> abort everywhere Fatal weakness: a participant that voted yes MUST hold its locks until it hears the final decision -- if the coordinator crashes in between, that participant is BLOCKED, indefinitely, unable to safely commit or abort on its own ``` Continue to `dist_distributed_transactions_theory_2.md` for the Saga pattern (choreography vs orchestration), compensating transactions, and the eventual-consistency tradeoff that replaces 2PC's atomicity guarantee.