# Exercise 2: Diagnose a Stuck 2PC Coordinator Incident

## Scenario

`LedgerCo` runs an internal funds-transfer system between two account databases (`AccountsA` and `AccountsB`), coordinated using classic two-phase commit, built years ago before the team knew better. At 2:14am, an on-call engineer gets paged: several transfers between accounts have been "stuck" for over an hour. Investigation so far:

- The 2PC coordinator process crashed at 2:09am (confirmed via its crash logs) and was auto-restarted by the process supervisor at 2:11am.
- `AccountsA`'s transaction log shows it received a `PREPARE` for transfer `TXN-88213` at 2:08am, voted yes, and has been holding a row-level lock on the source account ever since — it never received a `COMMIT` or `ABORT` for that transaction.
- `AccountsB`'s transaction log shows it received the SAME `PREPARE` for `TXN-88213` at 2:08am and ALSO voted yes, and is similarly still holding its lock.
- Every other transaction attempting to touch either of those two specific accounts since 2:08am has been blocked waiting on those locks — including unrelated transfers that don't involve `TXN-88213` at all, but happen to touch the same source account.
- The coordinator, now restarted, has no record of `TXN-88213`'s final decision — its own log shows it sent both `PREPARE`s and received both `yes` votes, but crashed before its next log entry (whatever that would have been).

## Deliverables

Produce all of the following, in this file, below this line:

1. **Root-cause explanation** — explain, referencing theory Part 1 Sections 2-3, exactly why both `AccountsA` and `AccountsB` are still holding their locks over an hour later, and why the coordinator's restart at 2:11am didn't fix it. Be precise about what state a participant is in immediately after voting yes and why it cannot safely decide on its own.
2. **Immediate mitigation options, with tradeoffs** — propose at least two concrete options the on-call engineer could take RIGHT NOW to unblock the two accounts, and for each, state what could go wrong (e.g. what happens if you manually force-commit or force-abort a transaction whose true outcome you don't actually know). Reference theory Part 1 Section 3's description of the blocking window.
3. **Why "just add more replicas of the coordinator" doesn't fully fix this class of incident** — explain, in your own words grounded in the theory, why coordinator redundancy reduces the *probability* of this scenario but doesn't eliminate the structural dependency every participant has on eventually learning the final decision from *some* coordinator.
4. **A migration recommendation** — recommend whether LedgerCo should migrate this funds-transfer flow to a Saga (theory Part 2), and if so, sketch briefly (2-4 sentences) what the compensating transaction for "debit AccountsA" would need to look like, given that a financial debit can't simply be "rolled back" once other transactions may have already read the post-debit balance.

## Constraints

- Deliverable 1 must explicitly name the exact moment (which log line / which phase) after which each participant became stuck, not just "2PC has this problem in general."
- Any mitigation you propose in Deliverable 2 must acknowledge the real risk of guessing wrong about the transaction's true outcome (double-crediting or double-debiting an account) — a mitigation that ignores this risk is incomplete.
- Deliverable 4's compensating transaction must be a real accounting operation (e.g. a reversing credit entry), not a claim that the original debit can be "undone" in place.