# Exercise 2: Design an Automated Canary Rollout and Its Analysis

## Scenario

`orders-api` runs on Kubernetes with a service mesh (weighted routing available). Metrics in Prometheus:

```
http_requests_total{service="orders-api", version="v1|v2", code="2xx|4xx|5xx"}
http_request_duration_seconds_bucket{service="orders-api", version="…", le="…"}
orders_placed_total{version="…"}
```

Traffic: ~600 req/s; baseline 5xx ratio 0.2%; baseline p99 280 ms; ~50 orders/minute.

## Deliverables

1. **Rollout plan**: steps (weights and pause durations) from 0% to 100%. Justify the first step's size and duration using the traffic numbers (how many requests and orders does v2 see in that window?).
2. **Analysis queries**: write the PromQL for (a) v2's 5xx ratio over 5 minutes, (b) v2's p99 latency, (c) v2's orders per minute normalized by its traffic share.
3. **Promotion/abort criteria**: thresholds relative to v1 (e.g. "v2 5xx ratio > 2× v1 and > 0.5% absolute"), how many consecutive failed checks abort, and what "inconclusive" means.
4. **Stickiness**: how users are assigned to v2 consistently, and why it matters for this service.
5. **Rollback mechanics**: what the controller does on abort, how long until v2 receives 0% traffic, and what happens to in-flight requests.
6. **Write it as configuration**: an Argo Rollouts `Rollout` + `AnalysisTemplate` (or Flagger `Canary`) in YAML or close pseudo-YAML.
7. **The trap**: v2 includes a migration adding a NOT NULL column without a default. Explain what happens during the canary and how to fix the release plan.

## Requirements Checklist

- [ ] First step small (1–5%) but long enough to collect meaningful samples (e.g. 5% × 600 req/s = 30 req/s → 9,000 requests in 5 minutes; ~2.5 orders/minute → orders need longer windows).
- [ ] PromQL uses `rate()` over windows and `histogram_quantile(0.99, sum by (le) (rate(..._bucket{version="v2"}[5m])))`.
- [ ] Criteria are relative to the baseline with absolute floors; require several consecutive passes; inconclusive (too little data) pauses instead of promoting.
- [ ] Sticky routing by user/session (header or cookie hash).
- [ ] Abort sets weight to 0 immediately; graceful draining handles in-flight requests.
- [ ] The NOT NULL migration breaks v1 inserts once applied → must use expand/contract (nullable/defaulted column first, enforce NOT NULL in a later release).