# Kubernetes Architecture

## 1. What Kubernetes Does

**Kubernetes** (K8s) is a container orchestrator: you declare the **desired state** ("run 3 replicas of `api:1.4.2`, reachable on port 80, with 512 MiB memory each") and Kubernetes continuously works to make the **actual state** match — scheduling containers onto machines, restarting them when they crash, replacing them when nodes die, rolling out new versions and wiring up networking. It builds on the container concepts in `devops-cloud/docker/` and is offered as a managed service by every cloud (EKS — `devops-cloud/aws/16-eks`, GKE, AKS).

## 2. Cluster Components

A cluster has a **control plane** (the brain) and **worker nodes** (where your containers run).

**Control plane:**

| Component | Role |
|---|---|
| **kube-apiserver** | the front door: REST API for every read/write; authentication, authorization (RBAC), admission control; the *only* component that talks to etcd |
| **etcd** | consistent, highly available key-value store (Raft consensus) holding all cluster state |
| **kube-scheduler** | picks a node for each new Pod based on resource requests, affinity, taints/tolerations, topology spread |
| **kube-controller-manager** | runs controllers: Deployment, ReplicaSet, Node, Job, EndpointSlice, ServiceAccount… |
| **cloud-controller-manager** | integrates with the cloud: load balancers for `Service type: LoadBalancer`, node lifecycle, routes |

**Every node:**

| Component | Role |
|---|---|
| **kubelet** | agent that watches the API for Pods assigned to its node and runs them via the container runtime; reports status, runs probes |
| **container runtime** | containerd or CRI-O (via the Container Runtime Interface) pulls images and runs containers |
| **kube-proxy** | programs iptables/IPVS rules (or replaced by eBPF in Cilium) so Service virtual IPs reach Pods |
| **CNI plugin** | gives each Pod an IP and connects Pods across nodes (Calico, Cilium, AWS VPC CNI) |

Cluster add-ons: **CoreDNS** (service discovery via DNS), metrics-server, ingress controllers, CSI storage drivers.

## 3. Objects and the API

Everything is an **object** with `apiVersion`, `kind`, `metadata` (name, namespace, labels, annotations) and usually `spec` (desired) and `status` (observed):

```yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: api
  namespace: petpal
  labels: { app: api, team: core }
spec:
  replicas: 3
  selector: { matchLabels: { app: api } }
  template:
    metadata: { labels: { app: api } }
    spec:
      containers:
        - name: api
          image: ghcr.io/petpal/api:1.4.2
          ports: [{ containerPort: 8080 }]
```

- **Namespaces** partition a cluster (per team/environment) for names, RBAC, quotas and network policies.
- **Labels** + **selectors** connect objects loosely: a Service finds Pods by label, a Deployment owns Pods by label.
- **Annotations** hold non-identifying metadata used by tools (e.g. ingress controller settings).
- Workload objects: **Pod**, **Deployment**/ReplicaSet, **StatefulSet**, **DaemonSet** (one Pod per node — log agents), **Job**/**CronJob**.

## 4. Working With the Cluster

```bash
kubectl get pods -n petpal -o wide            # list with node and IP
kubectl describe pod api-7c9f-abcde -n petpal # events: scheduling, pulls, probe failures
kubectl logs deploy/api -n petpal --since=10m
kubectl apply -f k8s/                          # declarative: create or update to match files
kubectl diff -f k8s/                           # what would change
kubectl explain deployment.spec.strategy      # built-in API docs
kubectl get events -n petpal --sort-by=.lastTimestamp
```

Prefer **declarative** `kubectl apply` (or GitOps — `08-production_practices_and_gitops/`) of files in version control over imperative `kubectl create/edit`.

## 5. Security Model in Brief

- Authentication: client certificates, OIDC tokens (cloud IAM), ServiceAccount tokens for Pods.
- Authorization: **RBAC** — `Role`/`ClusterRole` (verbs on resources) bound to users/groups/ServiceAccounts with `RoleBinding`/`ClusterRoleBinding`.
- **Admission controllers** validate/mutate objects before storage: Pod Security Admission (`restricted` profile), ResourceQuota, LimitRanger, and policy engines (Kyverno, OPA Gatekeeper).

## Under the Hood: The Reconciliation Loop

Nothing in Kubernetes is a one-shot command; everything is a **controller** watching the API server and reconciling. When you `kubectl apply` a Deployment: (1) the API server authenticates, authorizes and runs admission, then persists the object in etcd; (2) the **Deployment controller**, watching Deployments, sees no matching ReplicaSet and creates one; (3) the **ReplicaSet controller** sees 0 of 3 Pods and creates 3 Pod objects with no `nodeName`; (4) the **scheduler**, watching unscheduled Pods, filters nodes (enough CPU/memory requested, taints, affinity), scores them, and writes a **binding**; (5) the **kubelet** on the chosen node sees a Pod bound to it, asks containerd to pull the image and start containers, sets up networking via CNI, starts probes and reports status; (6) the **EndpointSlice controller** adds ready Pod IPs to the Service's endpoints, and kube-proxy on every node updates routing rules. If a node dies, the node controller marks it NotReady, its Pods are evicted, and the ReplicaSet controller creates replacements — the same loop. Watches (long-lived HTTP streams with resource versions) make this efficient: components react to changes rather than polling.

![kubectl sends a Deployment to the API server which stores it in etcd; the Deployment and ReplicaSet controllers create Pods, the scheduler binds them to nodes, and each node's kubelet runs containers via containerd while kube-proxy and the CNI wire up networking](diagrams/01-architecture.svg)

## Common Pitfalls

1. **Running bare Pods** — nothing recreates them; use Deployments/StatefulSets/Jobs.
2. **Imperative changes** (`kubectl edit`) that drift from files in Git.
3. **Everything in the `default` namespace** with cluster-admin for everyone.
4. **Treating etcd casually** in self-managed clusters — no backups means no cluster.
5. **Ignoring events** — `kubectl describe` usually tells you why a Pod is Pending or crashing.

## Quick Reference

```
control plane: apiserver (only etcd client) · etcd · scheduler · controller-manager · cloud-controller-manager
node: kubelet · containerd (CRI) · kube-proxy · CNI · add-ons: CoreDNS, metrics-server, ingress, CSI
object = apiVersion, kind, metadata (labels!), spec (desired), status (observed)
workloads: Deployment · StatefulSet · DaemonSet · Job/CronJob   ·  RBAC + admission (PSA restricted)
kubectl get/describe/logs/apply/diff/explain/events