Event-Driven Architecture

The previous topics covered the plumbing — brokers, queues, topics, delivery guarantees. Event-driven architecture (EDA) is the design style built on top: services communicate primarily by publishing facts about what happened, and other services react to those facts, instead of calling each other directly.

1Request-Driven vs Event-Driven

request-driven (orchestrated by the caller):
  orders ──HTTP──► payments
         ──HTTP──► inventory
         ──HTTP──► email
         ──HTTP──► loyalty            orders must know (and wait for) every downstream service

event-driven (choreographed by reactions):
  orders ──publish OrderPlaced──► broker ──► payments   (charge)
                                         ──► inventory  (reserve)
                                         ──► email      (confirm)
                                         ──► loyalty    (add points)
  orders knows nothing about who listens

2Three Kinds of Events

Most systems use event-carried state transfer for integration events: consumers stay autonomous even when the publisher is down.

3Designing Good Events

• Past tense, business language: OrderPlaced, PaymentFailed — not CreateOrder (that's a command, 05-queues_and_topics/ §2) and not OrderTableRowUpdated (that leaks the schema).
• Envelope + payload:
  jsonCopy

  {
    "eventId": "01HZX6K8…",            // unique, for dedup (09-delivery_guarantees/)
    "eventType": "OrderPlaced",
    "schemaVersion": 2,
    "occurredAt": "2025-06-12T10:15:03Z",
    "source": "orders-service",
    "correlationId": "req-81f…",        // ties to the original request / trace
    "data": { "orderId": "o-17", "customerId": "c-4", "total": 4599, "currency": "EUR" }
  }

  (CloudEvents is a standard for this envelope.)

• Key by the aggregate id (orderId) so all events of one order stay ordered (08-…/).
• Schema evolution: only make backward-compatible changes (add optional fields; never rename/remove/change types) within a version; use a schema registry (Avro/Protobuf/JSON Schema) to enforce it; publish a new event type or version for breaking changes.

4Choreography vs Orchestration

Multi-step business processes (place order → pay → reserve → ship) can be coordinated two ways:

• Choreography: each service reacts to events and emits its own (OrderPlaced → payments emits PaymentSucceeded → inventory emits StockReserved → shipping…). No central brain. Great for simple flows; for long flows the process is implicit and hard to see or change.
• Orchestration: a process manager / saga orchestrator sends commands and listens for replies (distributed-systems/fundamentals/08-distributed_transactions). The flow is explicit in one place; that service is a coupling point.

Rule of thumb: choreograph notifications and loosely related reactions; orchestrate a business transaction with compensations.

5Eventual Consistency, Made Visible

After OrderPlaced, the loyalty service's points balance is updated "soon" — not in the same transaction. Design for it:

• UIs show pending states ("Points will appear shortly").
• Read-your-writes where it matters: the service that accepted the write serves the immediate follow-up read.
• Consumers must be idempotent and tolerant of reordering (08-…/, 09-…/).

⚠️ Common Pitfalls (Part 1)

1. Commands disguised as events (SendEmailRequested published to "whoever") — hidden coupling.
2. Leaking internal schemas (orders_table_updated) — every DB refactor breaks consumers.
3. Notification events that trigger N callbacks to the publisher — a self-inflicted DDoS on every event.
4. Breaking schema changes without versioning.
5. Publishing outside the DB transaction — lost events (use the outbox, Part 2).

📋 Quick Reference (Part 1)

event = past-tense fact, envelope {eventId, type, version, occurredAt, correlationId} + data
key by aggregate id • event-carried state transfer for integration • backward-compatible evolution
choreography for reactions • orchestration for multi-step transactions with compensation

Part 1 defined events as the integration contract. Part 2 covers the three patterns that make event-driven systems reliable and that use events as more than notifications.

1The Transactional Outbox

A service must update its database and publish an event. Doing both naively has the dual-write problem:

await db.orders.insert(order);                 // ✅ committed
await broker.publish("OrderPlaced", order);    // 💥 crash here → event lost forever

Outbox pattern: write the event to an outbox table in the same transaction as the business change; a separate relay publishes outbox rows to the broker and marks them sent.

BEGIN;
  INSERT INTO orders (id, customer_id, total) VALUES ('o-17', 'c-4', 4599);
  INSERT INTO outbox (event_id, aggregate_id, event_type, payload)
       VALUES ('01HZX…', 'o-17', 'OrderPlaced', '{"orderId":"o-17", ...}');
COMMIT;

Relay options: polling (SELECT … FROM outbox WHERE sent_at IS NULL ORDER BY id LIMIT 100 FOR UPDATE SKIP LOCKED) or CDC (Debezium tails the WAL and publishes outbox inserts). Either way the guarantee is at-least-once publishing — consumers dedup by eventId.

The mirror image on the consumer side is the inbox: record processed eventIds in the consumer's DB in the same transaction as the side effect.

2Event Sourcing

Instead of storing current state and also emitting events, store only the events; current state is derived by replaying them.

stream "account-42":
  1 AccountOpened   { owner: "ana" }
  2 MoneyDeposited  { amount: 100 }
  3 MoneyWithdrawn  { amount: 30 }
  4 MoneyDeposited  { amount: 50 }
  ──────────────────────────────
  fold(events) → balance = 120

• Commands are validated against the current state (rebuilt from the stream), then produce new events appended to the stream.
• Optimistic concurrency: append with "expected version = 4"; if someone else appended first, the append fails and the command retries.
• Snapshots every N events avoid replaying long streams.

Gains: complete audit history; temporal queries ("balance on March 1st"); rebuild or add new read models by replaying; debugging by replay. Costs: events are forever — schema evolution needs upcasters; GDPR erasure needs crypto-shredding or careful design; querying current state needs projections (CQRS); a steeper learning curve.

Use it where history is the domain (ledgers, carts, workflows, compliance), not everywhere.

3CQRS — Command Query Responsibility Segregation

Separate the write model (validates commands, enforces invariants) from one or more read models (shaped for specific queries), kept in sync by events:

command ──► write model ──events──► projector ──► read model A (order list per customer, PostgreSQL)
                                    projector ──► read model B (search, Elasticsearch)
                                    projector ──► read model C (dashboard counters, Redis)
query ◄───────────────────────────────────────── read models

• Read models are eventually consistent with the write side (lag = projector delay).
• A read model can be dropped and rebuilt by replaying events from the beginning (Kafka retention/compaction or the event store).
• CQRS doesn't require event sourcing (you can project from outbox events), and event sourcing almost always needs CQRS.

4Projections Must Be Idempotent and Ordered per Aggregate

A projector processes events at-least-once, so: track the last applied position/version per aggregate, skip anything older or equal, and make each update idempotent. That's what makes replay and redelivery safe.

Under the HoodAppending to an Event Store Safely▾

An event store needs exactly two guarantees: append-only streams and atomic conditional append. In PostgreSQL:

sqlCopy

CREATE TABLE events (
  stream_id   TEXT    NOT NULL,
  version     INTEGER NOT NULL,          -- 1, 2, 3 … per stream
  event_type  TEXT    NOT NULL,
  data        JSONB   NOT NULL,
  global_pos  BIGSERIAL,                 -- total order for projectors
  PRIMARY KEY (stream_id, version)       -- ← the optimistic concurrency check
);

-- command handler loaded the stream at version 4, appends version 5:
INSERT INTO events (stream_id, version, event_type, data) VALUES ('account-42', 5, 'MoneyWithdrawn', '{"amount":80}');
-- a concurrent writer that also read version 4 tries version 5 → unique violation → reload and retry

The primary key turns "expected version" into a database-enforced check, so two concurrent commands can never both act on the same stale state (e.g. both withdraw the last €80). Projectors read by global_pos; note that with concurrent transactions a BIGSERIAL can commit out of order, so robust projectors either read only up to a safe position (no in-flight lower positions) or rely on a store that assigns positions at commit (EventStoreDB, Kafka offsets).

⚠️ Common Pitfalls (Part 2)

1. Dual writes instead of an outbox — lost or phantom events.
2. Event sourcing everything — CRUD screens with no history needs don't benefit.
3. Mutating stored events — breaks replay; version and upcast instead.
4. Non-idempotent projections — rebuilds and redeliveries corrupt read models.
5. Reading your own write from a lagging read model — route that read to the write side or wait for the projection position.

📋 Quick Reference (Part 2)

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: broker_event_driven_architecture_theory_1.md, broker_event_driven_architecture_theory_2.md · generated by tools/build_lessons.py