S3

1What S3 Is

S3 (Simple Storage Service) stores arbitrary blobs of data — objects — inside buckets, addressed by a key (a string, not a filesystem path). S3 is not a filesystem: there is no real directory tree, no partial-file edits, no file locking — every write replaces an object wholesale under its key.

aws s3api create-bucket --bucket my-app-uploads-prod --region us-east-1
aws s3api put-object --bucket my-app-uploads-prod --key uploads/2024/report.csv --body ./report.csv

2Buckets Are Globally Unique

A bucket name is unique across all of AWS, not just your account — my-app-uploads-prod is claimed the moment anyone, anywhere, creates it. This is a direct consequence of S3's addressing scheme: https://my-app-uploads-prod.s3.amazonaws.com/... needs the bucket name to resolve via DNS globally.

3Objects, Keys, and the "Folder" Illusion

A key like uploads/2024/report.csv LOOKS like a path, but S3 stores it as one flat string. The Console/CLI render /-delimited prefixes as folders purely as a UI convenience (a "list objects with delimiter /" API call) — there is no actual uploads/ directory object unless you explicitly create one (some tools upload a zero-byte object with a trailing slash to fake it).

aws s3api list-objects-v2 --bucket my-app-uploads-prod --prefix uploads/2024/ --delimiter /
# groups results as if uploads/2024/ were a folder, but it's a prefix filter, not a real directory

4Storage Classes and Lifecycle Rules

A lifecycle rule automates transitions/expiration without application code touching every object:

{
  "Rules": [{
    "ID": "archive-old-logs",
    "Filter": { "Prefix": "logs/" },
    "Status": "Enabled",
    "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" }, { "Days": 90, "StorageClass": "GLACIER" }],
    "Expiration": { "Days": 365 }
  }]
}

{ "Rules": [{
  "ID": "photos-tiering",
  "Filter": { "Prefix": "photos/" },
  "Status": "Enabled",
  "Transitions": [
    { "Days": 30,  "StorageClass": "STANDARD_IA" },
    { "Days": 180, "StorageClass": "GLACIER_IR" }
  ],
  "NoncurrentVersionExpiration": { "NoncurrentDays": 90 },
  "AbortIncompleteMultipartUpload": { "DaysAfterInitiation": 7 }
}]}

5Access Control: Bucket Policy vs. IAM Policy

Same identity-based / resource-based split from 01-fundamentals_and_iam section 5: an IAM policy attached to a user/role says what that identity can do; a bucket policy attached to the bucket itself says who can act on it — and is the only way to grant a principal in a different AWS account access without them assuming a role in yours.

{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::123456789012:role/AppRole" },
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "arn:aws:s3:::my-app-uploads-prod/*"
  }]
}

Legacy per-object ACLs still exist but are discouraged in favor of bucket policies + IAM — modern buckets have ACLs disabled by default.

6Public vs. Private: Block Public Access

Every bucket has an account-level and bucket-level Block Public Access setting, which — when enabled (the default for new buckets) — overrides any bucket policy or ACL that would otherwise make objects public, as a deliberate last line of defense. A bucket only becomes genuinely public when Block Public Access is explicitly disabled and a policy grants Principal: "*".

7Versioning

Enabling versioning means a PUT to an existing key creates a new version instead of overwriting, and a DELETE inserts a delete marker (the object appears gone, but every prior version is still retrievable) rather than destroying data. This is what makes accidental-overwrite and accidental-delete recoverable.

8Presigned URLs

A presigned URL grants temporary access to a private object without making the bucket public and without the requester needing any AWS credentials at all — the URL itself embeds a signature computed from the issuer's credentials.

aws s3 presign s3://my-app-uploads-prod/uploads/2024/report.csv --expires-in 300
# https://my-app-uploads-prod.s3.amazonaws.com/uploads/2024/report.csv
#   ?X-Amz-Signature=...&X-Amz-Expires=300&X-Amz-Date=...

Under the HoodHow a Presigned URL Is Verified▾

The signature is computed once, client-side (by whoever calls presign), from that caller's secret key + the exact request (bucket, key, HTTP method, expiry timestamp) using AWS's Signature Version 4 algorithm. When the URL is later fetched, S3 recomputes the same signature independently from the query-string parameters and its own copy of that IAM identity's secret key, and compares. No call back to IAM happens at access time — everything needed to verify (or reject, if X-Amz-Expires has elapsed) is self-contained in the URL. This is exactly why a presigned URL cannot be revoked early: there's no session to invalidate, only a signature that stays valid until its baked-in expiry passes.

⚠️ Common Pitfalls

1. Disabling Block Public Access "just to test something quickly." This is the single most common cause of accidental public data exposure — it's a deliberate multi-layer guard for exactly this reason.
2. Treating /-delimited keys as real directories — renaming a "folder" actually means copying every object under that prefix to new keys and deleting the old ones; there's no atomic rename.
3. Assuming deleted objects are gone in a versioned bucket. A delete marker hides the object from normal GETs but every version remains billable and recoverable until explicitly purged.
4. Setting a presigned URL's --expires-in too long, effectively creating a long-lived, unrevocable credential baked into a shareable link.
5. Forgetting that a bucket policy Deny overrides an otherwise-fine IAM Allow, exactly like the identity/resource policy interaction covered in 01-fundamentals_and_iam.

📋 Quick Reference

aws s3api put-bucket-versioning --bucket my-app-uploads-prod --versioning-configuration Status=Enabled
aws s3api put-public-access-block --bucket my-app-uploads-prod --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: aws_s3_theory.md · generated by tools/build_lessons.py