DynamoDB

1What DynamoDB Is

Amazon DynamoDB is a fully managed, serverless key-value and document database with single-digit-millisecond latency at virtually any scale. No servers, no connection limits (HTTP API), automatic replication across 3 AZs, optional global tables across regions. The trade-off: you must design the data model around your access patterns up front — there are no joins and no ad-hoc queries over arbitrary attributes.

2Core Concepts

3Reading: GetItem, Query, Scan

import { DynamoDBDocumentClient, GetCommand, QueryCommand } from "@aws-sdk/lib-dynamodb";

// GetItem: exact primary key → 1 item
await ddb.send(new GetCommand({ TableName: "app", Key: { PK: "USER#42", SK: "PROFILE" } }));

// Query: one partition key, optional sort-key condition → items in SK order (paginated, 1 MB per page)
await ddb.send(new QueryCommand({
  TableName: "app",
  KeyConditionExpression: "PK = :pk AND begins_with(SK, :prefix)",
  ExpressionAttributeValues: { ":pk": "USER#42", ":prefix": "ORDER#2025-" },
  ScanIndexForward: false,      // newest first
  Limit: 20,
}));

• Query is the workhorse: efficient because it reads one partition, ordered by SK.
• Scan reads the whole table (every partition) — expensive and slow; fine for small tables, backfills or exports, never for request paths.
• FilterExpression is applied after reading — you still pay for everything read. Design keys so filters are rarely needed.
• Reads are eventually consistent by default (half the cost); ConsistentRead: true for strongly consistent reads (not available on GSIs).

4Writing

await ddb.send(new PutCommand({
  TableName: "app",
  Item: { PK: "USER#42", SK: "ORDER#2025-06-12#o-9", total: 4599, status: "PLACED" },
  ConditionExpression: "attribute_not_exists(PK)",           // create only (idempotent creates, no overwrite)
}));

await ddb.send(new UpdateCommand({
  TableName: "app",
  Key: { PK: "PRODUCT#7", SK: "STOCK" },
  UpdateExpression: "SET qty = qty - :n",
  ConditionExpression: "qty >= :n",                          // atomic, no oversell (advanced-engineering/…/02-race_conditions)
  ExpressionAttributeValues: { ":n": 2 },
}));

• Conditional writes give optimistic concurrency (version attributes) and invariants without locks.
• TransactWriteItems: up to 100 items across tables, all-or-nothing (2× cost).
• BatchWriteItem / BatchGetItem: up to 25 / 100 items per call, not atomic, retry UnprocessedItems.
• TTL attribute: items expire automatically (deletion within ~48 h, free) — sessions, idempotency keys, temporary data.

5Capacity Modes

1 WCU = one write/s of ≤ 1 KB; 1 RCU = one strongly consistent read/s of ≤ 4 KB (or two eventually consistent). Item size drives cost — keep items small, project only needed attributes into GSIs.

Under the HoodPartitions and Hot Keys▾

DynamoDB hashes the partition key to place items on partitions — storage nodes each holding up to ~10 GB and serving up to about 3,000 RCU and 1,000 WCU per second. Each partition is replicated to three nodes in three AZs; writes are acknowledged after a quorum (two of three) persists them, with a leader per partition handling writes and strongly consistent reads. As data or throughput grows, DynamoDB splits partitions automatically, and adaptive capacity shifts throughput toward busy partitions. What it can't fix is a single hot key: all requests for PK = "GLOBAL_COUNTER" land on one partition, capping it at ~1,000 writes/s regardless of table capacity → ProvisionedThroughputExceededException / throttling. Spread load with high-cardinality partition keys or write sharding (COUNTER#<0..9> and sum on read) — the same idea as sharded counters in databases/caching/08-distributed_caching.

📋 Quick Reference (Part 1)

PK (+SK) · item ≤ 400 KB · GetItem (exact) · Query (one PK, SK range, sorted) · Scan (avoid in request paths)
filters cost what they read · eventually consistent default · ConditionExpression for invariants · transactions ≤ 100 items
on-demand vs provisioned · TTL for expiry · hot keys cap at ~1k WCU/3k RCU per partition → shard keys

1Start From Access Patterns

Relational design starts from entities and normalizes; DynamoDB design starts from the list of queries the application will run, then shapes keys so each query is a GetItem or Query.

Example — an e-commerce app:

2Single-Table Design

Store different entity types in one table with generic key names (PK, SK, GSI1PK, …) and a type attribute. Items that are read together share a partition key, so one Query returns a whole "aggregate" — the DynamoDB equivalent of a join, precomputed at write time:

PK               SK                         type       attributes
CUSTOMER#42      PROFILE                    customer   name, email, GSI2PK=EMAIL#ana@x.io
CUSTOMER#42      ORDER#2025-06-12#o-9       orderref   total, status
ORDER#o-9        META                       order      customerId, status, GSI1PK=STATUS#PLACED, GSI1SK=2025-06-12
ORDER#o-9        ITEM#1                     item       sku, qty, price
ORDER#o-9        ITEM#2                     item       ...

Pros: fewest requests per page, one table to provision/monitor/back up. Cons: harder to read and evolve, requires knowing access patterns, analytics need export (S3/Athena) or streams. Many teams use a pragmatic middle ground: one table per bounded context/service, single-table inside it.

PK            SK                          GSI1PK          attributes
CUSTOMER#42   PROFILE                     EMAIL#a@x.io    name, plan
CUSTOMER#42   ORDER#2026-10-01#o9         STATUS#PLACED   total
ORDER#o9      ITEM#1                      -               sku, qty

1. profile          → GetItem PK=CUSTOMER#42 SK=PROFILE
2. recent orders    → Query PK=CUSTOMER#42 SK begins_with ORDER# (desc)
3. order + items    → Query PK=ORDER#o9
4. by email         → Query GSI1 PK=EMAIL#a@x.io
5. placed orders    → Query GSI1 PK=STATUS#PLACED

3Secondary Indexes

• GSI: any attribute as partition/sort key; sparse — only items that have the GSI key attributes appear (great for "orders awaiting shipment": only those items carry GSI1PK).
• Projections: KEYS_ONLY, INCLUDE (chosen attributes) or ALL — every projected byte is written again (cost).
• GSIs are updated asynchronously (eventually consistent) and can throttle the base table if under-provisioned.
• Up to 20 GSIs per table; adding one later triggers a backfill.

4Patterns

• Adjacency list for many-to-many (users ↔ groups): items USER#1 / GROUP#7 and the inverse via a GSI.
• Composite sort keys for hierarchical queries: SK = COUNTRY#DE#CITY#Berlin#STORE#12 → begins_with at any level.
• Version history: SK = v0 (latest) + v1, v2… written in a transaction.
• Idempotency / dedup items with TTL.
• Write sharding for hot keys (Part 1 Under the Hood).

5DynamoDB Streams

A stream records every item-level change (new/old images) for 24 hours, ordered per partition key. Consumers (Lambda event source mapping, Kinesis adapter) build:

• materialized views / read models in another table or OpenSearch (CQRS — distributed-systems/message-brokers/11-event_driven_architecture),
• event publishing to EventBridge (a native outbox: write once, the stream guarantees the event),
• aggregations (counters, leaderboards),
• replication/audit trails.

Global tables use streams internally for multi-region, multi-active replication with last-writer-wins conflict resolution (databases/scaling/04-regional_partitioning).

6When DynamoDB Is the Wrong Choice

• Ad-hoc querying, reporting, complex joins → relational DB or analytics (export to S3 + Athena).
• Unknown or rapidly changing access patterns early in a product.
• Large items (> 400 KB) → store in S3, keep a pointer.
• Strong multi-item consistency across many entities beyond transactions' limits.

Under the HoodHow Global Secondary Indexes Are Maintained▾

A GSI is physically another partitioned table. When you write an item, DynamoDB's request router first commits the write to the base table's partition (quorum across AZs), then asynchronously propagates the change to each GSI whose key attributes are present — inserting the new projection and deleting the old one if the GSI key changed. Propagation is usually milliseconds but not transactional, which is why GSI reads are always eventually consistent and why a read-after-write via a GSI can miss a just-written item. If a GSI's partition is throttled (hot GSI key, insufficient provisioned WCU), DynamoDB applies back-pressure to the base table writes so the index can't fall arbitrarily behind — a classic surprise where "the table" throttles because of one badly designed index.

⚠️ Common Pitfalls (Part 2)

1. Designing tables like relational schemas — one table per entity and "joins" in code.
2. Scans + filters in request paths.
3. Low-cardinality partition keys (status, country) — hot partitions.
4. Projecting ALL into every GSI — doubled write costs.
5. Assuming GSI reads are consistent right after writes.
6. No plan for analytics — use exports/streams, not table scans.

📋 Quick Reference (Part 2)

list access patterns → design PK/SK/GSIs so each is Get/Query
single-table: generic keys, type attr, co-locate data read together
GSI: sparse, projections, eventually consistent, can throttle base table
streams: 24 h change log → Lambda → read models / EventBridge / aggregates · global tables = multi-region

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: aws_dynamodb_theory_1.md, aws_dynamodb_theory_2.md · generated by tools/build_lessons.py