# Autoscaling in Kubernetes

## 1. Three Layers of Autoscaling

| Layer | What scales | Tools |
|---|---|---|
| **Pods horizontally** | number of replicas | **HorizontalPodAutoscaler** (HPA), **KEDA** |
| **Pods vertically** | requests/limits of each Pod | **VerticalPodAutoscaler** (VPA), in-place resize |
| **Nodes** | number/size of machines | **Cluster Autoscaler**, **Karpenter** |

They work together: HPA adds Pods → some are Pending for lack of capacity → the node autoscaler adds nodes → Pods schedule. When load drops, HPA removes Pods → nodes become underused → the node autoscaler consolidates and removes them.

## 2. HorizontalPodAutoscaler

```yaml
apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 65 }   # % of *requests*
    - type: Pods
      pods:
        metric: { name: http_requests_per_second }               # via Prometheus Adapter
        target: { type: AverageValue, averageValue: "50" }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies: [{ type: Percent, value: 100, periodSeconds: 30 }]   # at most double every 30 s
    scaleDown:
      stabilizationWindowSeconds: 300                                # wait 5 min before shrinking
      policies: [{ type: Pods, value: 2, periodSeconds: 60 }]
```

- Formula: `desired = ceil(current × currentMetric / target)`; with multiple metrics, the highest wins.
- Needs **metrics-server** for CPU/memory; custom/external metrics via **Prometheus Adapter** or KEDA.
- CPU utilization is relative to **requests** — wrong requests mean wrong scaling.
- Memory is a poor HPA signal for most runtimes (memory rarely drops after load).
- Don't set `replicas` in the Deployment manifest when an HPA manages it (GitOps will fight the HPA).

## 3. KEDA: Event-Driven Autoscaling

**KEDA** scales on external event sources — queue length (SQS, RabbitMQ, Kafka lag), cron schedules, Prometheus queries — and can scale **to zero**:

```yaml
apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: { name: thumbnail-worker }
spec:
  scaleTargetRef: { name: thumbnail-worker }
  minReplicaCount: 0
  maxReplicaCount: 50
  cooldownPeriod: 120
  triggers:
    - type: aws-sqs-queue
      metadata: { queueURL: "https://sqs.eu-west-1.amazonaws.com/111111111111/thumbnails", queueLength: "20", awsRegion: eu-west-1 }
      authenticationRef: { name: keda-aws-pod-identity }
```

Queue-based scaling targets the *cause* (backlog) rather than the *symptom* (CPU) — ideal for workers.

## 4. VerticalPodAutoscaler

VPA observes actual usage and recommends (or applies) requests. Modes: `Off` (recommendations only — great for right-sizing), `Initial` (set at Pod creation), `Recreate`/`InPlaceOrRecreate` (update running Pods). Don't let VPA and HPA both act on CPU for the same workload.

## 5. Node Autoscaling

- **Cluster Autoscaler**: scales predefined node groups (ASGs) up when Pods are Pending and down when nodes are underutilized and their Pods can move elsewhere.
- **Karpenter**: provisions right-sized nodes directly from Pending Pods' requirements (instance types, arch, Spot/on-demand, zones) in about a minute, and **consolidates** continuously by replacing/removing underused nodes.

```yaml
apiVersion: karpenter.sh/v1
kind: NodePool
metadata: { name: default }
spec:
  template:
    spec:
      requirements:
        - { key: karpenter.sh/capacity-type, operator: In, values: [spot, on-demand] }
        - { key: kubernetes.io/arch, operator: In, values: [arm64, amd64] }
      nodeClassRef: { group: karpenter.k8s.aws, kind: EC2NodeClass, name: default }
  limits: { cpu: "400" }
  disruption: { consolidationPolicy: WhenEmptyOrUnderutilized, consolidateAfter: 1m }
```

Protect workloads during scale-down with **PodDisruptionBudgets** and graceful shutdown; mark Pods that must not move with `karpenter.sh/do-not-disrupt`.

## Under the Hood: The HPA Control Loop

Every 15 seconds the HPA controller (in kube-controller-manager) reads metrics for the target's Pods from the **metrics APIs** (`metrics.k8s.io` from metrics-server, `custom.metrics.k8s.io`/`external.metrics.k8s.io` from adapters such as Prometheus Adapter or KEDA's metrics server). For resource metrics it averages usage over Pods as a percentage of their requests, ignoring Pods that are not yet ready or are missing metrics (conservatively assuming 0% on scale-up / 100% on scale-down for those), and skips scaling if the ratio is within a 10% **tolerance**. It computes a recommendation per metric, takes the maximum, applies `behavior` policies and the stabilization window (scale-down uses the highest recommendation over the last 5 minutes to prevent flapping), and updates the `scale` subresource of the Deployment. New Pods then need to be scheduled — if no node fits, they're Pending until Karpenter or Cluster Autoscaler adds capacity, which is why fast-booting nodes, pre-pulled images and some headroom (overprovisioning with low-priority placeholder Pods) matter for spiky traffic.

![Rising load raises Pod CPU metrics; the HPA reads them from metrics-server or KEDA and increases replicas; Pods that don't fit stay Pending until Karpenter or Cluster Autoscaler launches nodes; VPA right-sizes requests, and on low load consolidation removes nodes](diagrams/01-autoscaling-layers.svg)

## Common Pitfalls

1. **No or wrong requests** — HPA percentages and bin-packing are meaningless.
2. **HPA on memory** for runtimes that never release memory.
3. **`replicas` in Git fighting the HPA**.
4. **Scale-down too aggressive** — flapping; no PDBs during consolidation.
5. **Slow startup** (big images, long JVM warm-up) making autoscaling too late; use startup probes, pre-scaling or KEDA cron triggers.
6. **No `maxReplicas`/node limits** — a bug or attack scales your bill.

## Quick Reference

```
HPA autoscaling/v2: CPU % of requests · custom/external metrics · behavior (stabilization, policies) · max of metrics
KEDA: event sources (SQS, Kafka lag, cron, Prometheus) · scale to zero
VPA: Off (recommend) / Initial / Recreate · not with HPA on same resource
nodes: Cluster Autoscaler (node groups) · Karpenter (NodePools, Spot, consolidation) · PDBs