Autoscaling in Kubernetes

1Three Layers of Autoscaling

They work together: HPA adds Pods → some are Pending for lack of capacity → the node autoscaler adds nodes → Pods schedule. When load drops, HPA removes Pods → nodes become underused → the node autoscaler consolidates and removes them.

2HorizontalPodAutoscaler

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata: { name: api }
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: api }
  minReplicas: 3
  maxReplicas: 30
  metrics:
    - type: Resource
      resource:
        name: cpu
        target: { type: Utilization, averageUtilization: 65 }   # % of *requests*
    - type: Pods
      pods:
        metric: { name: http_requests_per_second }               # via Prometheus Adapter
        target: { type: AverageValue, averageValue: "50" }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 0
      policies: [{ type: Percent, value: 100, periodSeconds: 30 }]   # at most double every 30 s
    scaleDown:
      stabilizationWindowSeconds: 300                                # wait 5 min before shrinking
      policies: [{ type: Pods, value: 2, periodSeconds: 60 }]

• Formula: desired = ceil(current × currentMetric / target); with multiple metrics, the highest wins.
• Needs metrics-server for CPU/memory; custom/external metrics via Prometheus Adapter or KEDA.
• CPU utilization is relative to requests — wrong requests mean wrong scaling.
• Memory is a poor HPA signal for most runtimes (memory rarely drops after load).
• Don't set replicas in the Deployment manifest when an HPA manages it (GitOps will fight the HPA).

3KEDA: Event-Driven Autoscaling

KEDA scales on external event sources — queue length (SQS, RabbitMQ, Kafka lag), cron schedules, Prometheus queries — and can scale to zero:

apiVersion: keda.sh/v1alpha1
kind: ScaledObject
metadata: { name: thumbnail-worker }
spec:
  scaleTargetRef: { name: thumbnail-worker }
  minReplicaCount: 0
  maxReplicaCount: 50
  cooldownPeriod: 120
  triggers:
    - type: aws-sqs-queue
      metadata: { queueURL: "https://sqs.eu-west-1.amazonaws.com/111111111111/thumbnails", queueLength: "20", awsRegion: eu-west-1 }
      authenticationRef: { name: keda-aws-pod-identity }

Queue-based scaling targets the cause (backlog) rather than the symptom (CPU) — ideal for workers.

4VerticalPodAutoscaler

VPA observes actual usage and recommends (or applies) requests. Modes: Off (recommendations only — great for right-sizing), Initial (set at Pod creation), Recreate/InPlaceOrRecreate (update running Pods). Don't let VPA and HPA both act on CPU for the same workload.

5Node Autoscaling

• Cluster Autoscaler: scales predefined node groups (ASGs) up when Pods are Pending and down when nodes are underutilized and their Pods can move elsewhere.
• Karpenter: provisions right-sized nodes directly from Pending Pods' requirements (instance types, arch, Spot/on-demand, zones) in about a minute, and consolidates continuously by replacing/removing underused nodes.

apiVersion: karpenter.sh/v1
kind: NodePool
metadata: { name: default }
spec:
  template:
    spec:
      requirements:
        - { key: karpenter.sh/capacity-type, operator: In, values: [spot, on-demand] }
        - { key: kubernetes.io/arch, operator: In, values: [arm64, amd64] }
      nodeClassRef: { group: karpenter.k8s.aws, kind: EC2NodeClass, name: default }
  limits: { cpu: "400" }
  disruption: { consolidationPolicy: WhenEmptyOrUnderutilized, consolidateAfter: 1m }

Protect workloads during scale-down with PodDisruptionBudgets and graceful shutdown; mark Pods that must not move with karpenter.sh/do-not-disrupt.

Under the HoodThe HPA Control Loop▾

Every 15 seconds the HPA controller (in kube-controller-manager) reads metrics for the target's Pods from the metrics APIs (metrics.k8s.io from metrics-server, custom.metrics.k8s.io/external.metrics.k8s.io from adapters such as Prometheus Adapter or KEDA's metrics server). For resource metrics it averages usage over Pods as a percentage of their requests, ignoring Pods that are not yet ready or are missing metrics (conservatively assuming 0% on scale-up / 100% on scale-down for those), and skips scaling if the ratio is within a 10% tolerance. It computes a recommendation per metric, takes the maximum, applies behavior policies and the stabilization window (scale-down uses the highest recommendation over the last 5 minutes to prevent flapping), and updates the scale subresource of the Deployment. New Pods then need to be scheduled — if no node fits, they're Pending until Karpenter or Cluster Autoscaler adds capacity, which is why fast-booting nodes, pre-pulled images and some headroom (overprovisioning with low-priority placeholder Pods) matter for spiky traffic.

⚠️ Common Pitfalls

1. No or wrong requests — HPA percentages and bin-packing are meaningless.
2. HPA on memory for runtimes that never release memory.
3. replicas in Git fighting the HPA.
4. Scale-down too aggressive — flapping; no PDBs during consolidation.
5. Slow startup (big images, long JVM warm-up) making autoscaling too late; use startup probes, pre-scaling or KEDA cron triggers.
6. No maxReplicas/node limits — a bug or attack scales your bill.

📋 Quick Reference

HPA autoscaling/v2: CPU % of requests · custom/external metrics · behavior (stabilization, policies) · max of metrics
KEDA: event sources (SQS, Kafka lag, cron, Prometheus) · scale to zero
VPA: Off (recommend) / Initial / Recreate · not with HPA on same resource
nodes: Cluster Autoscaler (node groups) · Karpenter (NodePools, Spot, consolidation) · PDBs

🎤 Interview Q&A

🛠️ Practice exercises

Each exercise is a file in this topic folder. Code exercises are self-checking: run them and make the checks pass.

✅ Check yourself

Source: k8s_autoscaling_theory.md · generated by tools/build_lessons.py