# S3

## 1. What S3 Is

**S3 (Simple Storage Service)** stores arbitrary blobs of data — **objects** — inside **buckets**, addressed by a **key** (a string, not a filesystem path). S3 is not a filesystem: there is no real directory tree, no partial-file edits, no file locking — every write replaces an object wholesale under its key.

```bash
aws s3api create-bucket --bucket my-app-uploads-prod --region us-east-1
aws s3api put-object --bucket my-app-uploads-prod --key uploads/2024/report.csv --body ./report.csv
```

## 2. Buckets Are Globally Unique

A bucket name is unique across **all of AWS**, not just your account — `my-app-uploads-prod` is claimed the moment anyone, anywhere, creates it. This is a direct consequence of S3's addressing scheme: `https://my-app-uploads-prod.s3.amazonaws.com/...` needs the bucket name to resolve via DNS globally.

## 3. Objects, Keys, and the "Folder" Illusion

A key like `uploads/2024/report.csv` LOOKS like a path, but S3 stores it as one flat string. The Console/CLI render `/`-delimited prefixes as folders purely as a UI convenience (a "list objects with delimiter `/`" API call) — there is no actual `uploads/` directory object unless you explicitly create one (some tools upload a zero-byte object with a trailing slash to fake it).

```bash
aws s3api list-objects-v2 --bucket my-app-uploads-prod --prefix uploads/2024/ --delimiter /
# groups results as if uploads/2024/ were a folder, but it's a prefix filter, not a real directory
```

## 4. Storage Classes and Lifecycle Rules

| Class | Use case |
|---|---|
| Standard | frequently accessed data |
| Standard-IA | infrequent access, still millisecond retrieval, cheaper storage / pricier retrieval |
| Glacier / Glacier Deep Archive | archival, retrieval takes minutes-to-hours, cheapest storage |
| Intelligent-Tiering | AWS auto-moves objects between tiers based on observed access patterns |

A **lifecycle rule** automates transitions/expiration without application code touching every object:

```json
{
  "Rules": [{
    "ID": "archive-old-logs",
    "Filter": { "Prefix": "logs/" },
    "Status": "Enabled",
    "Transitions": [{ "Days": 30, "StorageClass": "STANDARD_IA" }, { "Days": 90, "StorageClass": "GLACIER" }],
    "Expiration": { "Days": 365 }
  }]
}
```

## 5. Access Control: Bucket Policy vs. IAM Policy

Same identity-based / resource-based split from `01-fundamentals_and_iam` section 5: an **IAM policy** attached to a user/role says what that identity can do; a **bucket policy** attached to the bucket itself says who can act on it — and is the only way to grant a principal in a *different* AWS account access without them assuming a role in yours.

```json
{
  "Version": "2012-10-17",
  "Statement": [{
    "Effect": "Allow",
    "Principal": { "AWS": "arn:aws:iam::123456789012:role/AppRole" },
    "Action": ["s3:GetObject", "s3:PutObject"],
    "Resource": "arn:aws:s3:::my-app-uploads-prod/*"
  }]
}
```

Legacy per-object **ACLs** still exist but are discouraged in favor of bucket policies + IAM — modern buckets have ACLs disabled by default.

## 6. Public vs. Private: Block Public Access

Every bucket has an account-level and bucket-level **Block Public Access** setting, which — when enabled (the default for new buckets) — overrides *any* bucket policy or ACL that would otherwise make objects public, as a deliberate last line of defense. A bucket only becomes genuinely public when Block Public Access is explicitly disabled *and* a policy grants `Principal: "*"`.

## 7. Versioning

Enabling **versioning** means a `PUT` to an existing key creates a new version instead of overwriting, and a `DELETE` inserts a **delete marker** (the object appears gone, but every prior version is still retrievable) rather than destroying data. This is what makes accidental-overwrite and accidental-delete recoverable.

## 8. Presigned URLs

A **presigned URL** grants temporary access to a private object without making the bucket public and without the requester needing any AWS credentials at all — the URL itself embeds a signature computed from the *issuer's* credentials.

```bash
aws s3 presign s3://my-app-uploads-prod/uploads/2024/report.csv --expires-in 300
# https://my-app-uploads-prod.s3.amazonaws.com/uploads/2024/report.csv
#   ?X-Amz-Signature=...&X-Amz-Expires=300&X-Amz-Date=...
```

## 9. Under the Hood: How a Presigned URL Is Verified

The signature is computed once, client-side (by whoever calls `presign`), from that caller's secret key + the exact request (bucket, key, HTTP method, expiry timestamp) using AWS's Signature Version 4 algorithm. When the URL is later fetched, S3 **recomputes the same signature independently** from the query-string parameters and its own copy of that IAM identity's secret key, and compares. No call back to IAM happens at access time — everything needed to verify (or reject, if `X-Amz-Expires` has elapsed) is self-contained in the URL. This is exactly why a presigned URL cannot be revoked early: there's no session to invalidate, only a signature that stays valid until its baked-in expiry passes.

![S3 buckets hold objects addressed by keys, are private by default with IAM and bucket policies and Block Public Access, use lifecycle and versioning, and share content via presigned URLs or CloudFront with origin access control](diagrams/01-s3-access-model.svg)

## 10. Common Pitfalls

1. **Disabling Block Public Access "just to test something quickly."** This is the single most common cause of accidental public data exposure — it's a deliberate multi-layer guard for exactly this reason.
2. **Treating `/`-delimited keys as real directories** — renaming a "folder" actually means copying every object under that prefix to new keys and deleting the old ones; there's no atomic rename.
3. **Assuming deleted objects are gone in a versioned bucket.** A delete marker hides the object from normal `GET`s but every version remains billable and recoverable until explicitly purged.
4. **Setting a presigned URL's `--expires-in` too long**, effectively creating a long-lived, unrevocable credential baked into a shareable link.
5. **Forgetting that a bucket policy Deny overrides an otherwise-fine IAM Allow**, exactly like the identity/resource policy interaction covered in `01-fundamentals_and_iam`.

## 11. Quick Reference

| Concept | Key fact |
|---|---|
| Bucket | globally-unique namespace; objects addressed by flat key |
| Key | a string, NOT a real path — "folders" are a prefix-filter UI illusion |
| Storage class | Standard / IA / Glacier — cost vs. retrieval-speed tradeoff |
| Lifecycle rule | automated transition/expiration by prefix + age |
| Bucket policy | resource-based; only way to grant cross-account access |
| Block Public Access | account/bucket-level guard that overrides policies/ACLs |
| Versioning | overwrite/delete become new versions / delete markers, not destruction |
| Presigned URL | self-contained signed access, verified without a live IAM call |

```bash
aws s3api put-bucket-versioning --bucket my-app-uploads-prod --versioning-configuration Status=Enabled
aws s3api put-public-access-block --bucket my-app-uploads-prod --public-access-block-configuration BlockPublicAcls=true,IgnorePublicAcls=true,BlockPublicPolicy=true,RestrictPublicBuckets=true
```