Why orchestration
You can run one container with docker run. But production means many containers, across many machines, that must survive crashes, reboots, scaling, and rollouts — without you babysitting them. This lesson is the honest bridge from a single docker run to a cluster: the exact problems orchestration solves, the anatomy of a Kubernetes cluster, why a Pod is not the same thing as a container, and — just as important — when not to reach for Kubernetes at all.
docker run stop being enough. This lesson is concepts and reading; nothing here needs a cluster yet. From K2 onward the labs apply real YAML to a real cluster, so install minikube or kind locally now, or plan to use EKS in K5. Verify versions and API fields against the current Kubernetes docs — the project moves fast.Learning objectives
- Name the concrete production problems
docker runleaves unsolved and that orchestration solves. - Describe the anatomy of a cluster: control plane (api-server, etcd, scheduler, controller-manager) and nodes (kubelet, container runtime, kube-proxy).
- Explain the declarative model — you describe desired state, controllers reconcile toward it.
- Distinguish a Pod from a container and say why the Pod is the smallest unit k8s schedules.
- Decide honestly when not to use Kubernetes — the cases where it is the wrong tool.
1 · What `docker run` cannot do
A single docker run is perfect for one container on one machine. Production is not that. The moment you have real traffic, you need answers to questions Docker alone does not answer: what restarts a crashed container at 3am? What moves it when its host dies? How do ten replicas share one stable address? How do you roll out v2 without dropping requests, and roll back in seconds when it's bad? Orchestration is the standing machinery that answers all of these automatically.
| Production need | With bare `docker run` | With an orchestrator |
|---|---|---|
| Restart on crash | --restart flag, per host only | Self-healing across the whole cluster |
| Host dies | Containers on it are gone | Rescheduled onto healthy nodes |
| Scale to N replicas | Run N commands, track them yourself | replicas: N — one field |
| Stable address for N replicas | Wire up your own load balancer | A Service gives one virtual IP + DNS |
| Zero-downtime deploy | Manual, error-prone | Rolling update built in, with rollback |
| Config & secrets | Bake in or pass by hand | ConfigMaps / Secrets, mounted at runtime |
2 · The declarative model — desired vs actual state
This is the single most important idea in Kubernetes, and it's what separates it from a shell script. You do not tell the cluster "start this container, then that one." You declare the desired state — "I want 3 replicas of this image exposed on port 80" — and write it down as YAML. A set of controllers continuously compares desired state to actual state and takes whatever action closes the gap. Delete a Pod and the controller notices actual < desired and makes a new one. This loop is called reconciliation.
3 · Cluster anatomy — control plane + nodes
A cluster is two kinds of machines. The control plane is the brain that decides what should run where; the worker nodes are the muscle that actually runs your containers. You talk only to the control plane (via kubectl); it drives the nodes.
| Component | Lives on | What it does |
|---|---|---|
| kube-apiserver | Control plane | The front door. Every read/write goes through it; it validates and persists. |
| etcd | Control plane | The cluster's database — the single source of truth for all state. |
| kube-scheduler | Control plane | Decides which node an unscheduled Pod should run on (fit, resources, constraints). |
| controller-manager | Control plane | Runs the reconciliation loops (Deployment, ReplicaSet, node controllers…). |
| kubelet | Every node | The node agent — starts/stops Pods and reports health back to the api-server. |
| container runtime | Every node | Actually runs containers (containerd/CRI-O). |
| kube-proxy | Every node | Programs the node's networking so Service virtual IPs route to Pods. |
On a managed service like EKS (K5), AWS runs and secures the entire control plane for you — you only manage nodes. That's the main reason most teams use managed Kubernetes rather than running etcd themselves.
4 · Pods vs containers
Newcomers assume Kubernetes runs containers. It doesn't — directly. The smallest unit it schedules is a Pod: one or more containers that share a network namespace (same IP, same localhost) and can share storage. Usually a Pod is one app container; a second sidecar container (a log shipper, a proxy) is added only when it must live and die with the main one.
pod.yamlapiVersion: v1
kind: Pod
metadata:
name: web
labels:
app: web
spec:
containers:
- name: app
image: ghcr.io/example/web:1.0.0 # pin a version, never :latest
ports:
- containerPort: 8000
run.shkubectl apply -f pod.yaml # create/update from the manifest
kubectl get pods # is it Running?
kubectl describe pod web # events, image, node, why it's Pending/CrashLooping
kubectl logs web # the container's stdout/stderr
NAME READY STATUS RESTARTS AGE
web 1/1 Running 0 12s5 · When NOT to use Kubernetes
Kubernetes is powerful and expensive in complexity. It is the wrong first choice more often than the internet admits. Reach for something simpler when:
| Situation | Why k8s is overkill | Reach for instead |
|---|---|---|
| A single container, low traffic | You'd run a control plane to babysit one app | A PaaS: Fly.io, Render, App Runner, ECS |
| A small team, no platform engineer | k8s needs someone to own it full-time | Managed containers / serverless |
| Bursty, event-driven, scale-to-zero | Idle nodes cost money | Lambda / Cloud Run / Fargate tasks |
| A static site or simple API | Enormous machinery for little benefit | A CDN / a managed app host |
| You just want to ship this week | The learning curve is real | Whatever gets it live; migrate later |
✓ Checkpoint — you can move on when you can…
- Name three production problems
docker runalone doesn't solve. - Explain the reconciliation loop in one sentence (desired vs actual state).
- Label the control-plane components and say what etcd and the scheduler each do.
- Explain why a Pod — not a container — is the unit Kubernetes schedules.
- Give two concrete situations where you'd deliberately not use Kubernetes.
You delete a Pod that was created directly with kubectl apply -f pod.yaml (a bare Pod, not owned by a Deployment). What happens, and why is this different from deleting a Pod managed by a Deployment?
Show answer
A startup with one Flask API and two engineers asks whether to launch on Kubernetes. What's the honest answer?
Show answer
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Before you learn the YAML, you should be able to defend why the cluster exists at all — the reflex a lead expects from you in a design review.
Your task: Write a short, concrete answer to "we already have docker run on a VM — why add Kubernetes?" grounded in real failure modes, not buzzwords.
Requirements:
- Name at least three production problems bare
docker runleaves unsolved - For each, state what an orchestrator does instead
- End with one honest caveat: a case where k8s would be the wrong call
💡 Hint: Anchor each point to a 3am failure: a crash, a dead host, a bad deploy.
Show solution
The pitch. Bare docker run is fine until production stress arrives, and then it leaves gaps an orchestrator fills automatically:
- Crash at 3am → a bare container needs
--restartand only recovers on that one host; Kubernetes self-heals by rescheduling failed Pods anywhere in the cluster. - Host dies → everything on it is lost; the scheduler reschedules those Pods onto healthy nodes.
- Bad deploy → manual stop/start drops requests; a Deployment does a rolling update with an instant rollback.
- Ten replicas need one address → you'd hand-roll a load balancer; a Service gives a stable virtual IP and DNS name.
The honest caveat: for a single low-traffic app and a two-person team, this is all cost and little benefit — a PaaS or serverless host is the right call. Kubernetes earns its complexity only when you run many services across many machines.
Context: Understanding the request path demystifies every later debugging session.
Your task: Describe, step by step, what happens inside the cluster from the moment you run kubectl apply -f pod.yaml until the container is actually running.
Requirements:
- Follow the request through the api-server and etcd
- Say where the scheduling decision is made
- Say which component actually starts the container on the node
💡 Hint: Every write goes through the api-server; nothing talks to etcd directly except it.
Show solution
The path, component by component:
kubectlsends the manifest to the kube-apiserver (the only front door), which authenticates, validates, and writes the desired state to etcd.- The Pod now exists in etcd but is unscheduled (no node assigned). The kube-scheduler watches for such Pods, picks a node that fits (resources, constraints), and writes that assignment back through the api-server.
- The kubelet on the chosen node sees a Pod bound to it, tells the container runtime (containerd) to pull the image and start the container, then reports status back.
- kube-proxy programs node networking so any Service in front of the Pod can route to it.
The mental model: you write desired state → api-server persists it → controllers and the scheduler reconcile → kubelet makes it real. Every debugging step later (describe, logs, events) is just reading one stage of this pipeline.
Context: The reconciliation loop is the heart of k8s; you should be able to demonstrate it, not just define it.
Your task: Explain what happens when a node running one of a Deployment's 3 replicas is abruptly powered off, tracing the loop back to 3 healthy replicas.
Requirements:
- Identify which controller notices the gap and how
- Explain the role of node health / heartbeats
- State what the end state looks like and roughly how fast
💡 Hint: Two loops interact here: node health detection and the ReplicaSet's replica count.
Show solution
Setup: a Deployment declares replicas: 3; its ReplicaSet has 3 Pods, one per node. A node is powered off hard.
- The dead node's kubelet stops sending heartbeats. The node controller waits a grace period, then marks the node
NotReadyand eventually evicts its Pods (deleting them from desired-state's point of view). - The ReplicaSet controller now sees actual (2 running) < desired (3). It creates a new Pod to close the gap.
- That new Pod is unscheduled, so the scheduler places it on a surviving healthy node; its kubelet starts the container.
- End state: 3 running replicas again, no human involved. Timing depends on the node eviction timeouts (tens of seconds to a few minutes) — verify the current default grace/eviction settings against the Kubernetes docs.
The lesson: self-healing is not magic — it's two reconciliation loops (node health and replica count) each independently pulling actual state back toward desired state.
Context: The expert skill is knowing when NOT to use the tool you're learning.
Your task: Given four representative systems, decide for each whether Kubernetes is justified, and name the alternative when it isn't.
Requirements:
- System A: a marketing static site
- System B: a monthly batch job that runs 20 minutes then exits
- System C: a platform of 15 microservices for 40 engineers
- System D: a bursty webhook handler, idle most of the day
💡 Hint: Score each on: many services? many machines? need self-healing/portability? bursty?
Show solution
System A — static site → no. A CDN or managed static host is cheaper, faster, and needs no cluster. k8s adds nothing.
System B — monthly batch → no (mostly). A scheduled serverless job (Lambda, Cloud Run job, or a Fargate task) matches the run-then-exit shape without paying for idle nodes. Only put it on k8s if a cluster already exists and a CronJob is convenient.
System C — 15 services, 40 engineers → yes. This is exactly k8s's sweet spot: many services across many machines, a shared platform, self-healing, and a team large enough to own the cluster. The complexity now buys real leverage.
System D — bursty webhook, idle → probably no. Scale-to-zero serverless (Lambda / Cloud Run) avoids paying for idle capacity; on k8s you'd carry baseline nodes or add KEDA/knative to fake scale-to-zero. Use k8s only if it's already your platform.
The rule: Kubernetes earns its keep on many services × many machines × a team to run it. Miss those and a simpler host wins.
Context: A platform engineer must be able to sketch the cluster and reason about what breaks when a given piece fails.
Your task: Produce a labeled diagram of a cluster (control plane + 3 nodes) and, for each component, state the blast radius if it fails.
Requirements:
- Include api-server, etcd, scheduler, controller-manager, and per-node kubelet/runtime/proxy
- For each, describe what still works and what breaks if it's down
- Note which components a managed service (EKS) takes off your plate
💡 Hint: Ask for each: if this dies, can running Pods still serve traffic? Can new ones be created?
Show solution
Failure-domain analysis:
- api-server down → no new changes (no apply, no scheduling of new work), but existing Pods keep running and serving traffic since kubelets act on already-persisted state. The cluster is frozen, not dead.
- etcd down/lost → the most serious: it's the source of truth. Without a backup, cluster state is unrecoverable. Running Pods may limp on, but the cluster can't be managed. Back up etcd.
- scheduler down → running Pods fine; new Pods stay
Pending(unassigned) until it returns. - controller-manager down → reconciliation stops: crashed Pods aren't replaced, rollouts stall. Existing steady-state keeps serving.
- one node's kubelet down → that node's Pods are eventually rescheduled elsewhere; the rest of the cluster is unaffected.
Managed value: on EKS, AWS runs and replicates the api-server and etcd across AZs and handles their backups/upgrades — you inherit a resilient control plane and only own the nodes. That offload is the main reason to pay for managed Kubernetes.
Context: Representative scenario: a Series-B company with 6 services and growing traffic asks platform engineering for a recommendation on adopting Kubernetes. Leadership wants a one-pager, not a religious war.
Your task: Write a decision memo that recommends for or against adopting Kubernetes now, with explicit criteria, costs, and a migration path.
Requirements:
- State the decision up front, then the criteria behind it
- Quantify the ongoing cost: who owns the cluster, what breaks if no one does
- Include a 'revisit if' trigger list so the decision isn't permanent
- Name the interim option and how you'd migrate to k8s later without a rewrite
💡 Hint: Frame it as reversible: containerize now (portable), choose the runtime as a separate, revisitable decision.
Show solution
Recommendation: not yet — containerize now, defer the cluster. Ship the 6 services as containers on a managed host (ECS/Fargate or Cloud Run) today; adopt Kubernetes when the triggers below fire. Rationale, in one page:
Criteria we scored. (1) Number of services: 6 — meaningful but not yet fleet-scale. (2) Team: no dedicated platform owner — the decisive factor. (3) Portability need: low today. (4) Self-healing/scaling: a managed host already provides both. On these, k8s's benefits are largely already met, while its costs are not yet justified.
The real cost of adopting now. Kubernetes needs an owner: upgrades, CVE patching, networking, RBAC, cost control, and on-call for the cluster itself. Without a named owner it rots into an unpatched, misconfigured liability — the worst of both worlds. That headcount is the true price, not the compute.
Revisit if any of these become true: we cross ~10–15 services; we hire/allocate a platform engineer; we need multi-cloud or on-prem portability; a managed host's limits (custom networking, GPUs, operators) start blocking us; or spend on managed containers exceeds what a cluster would cost with an owner.
Migration path (why this is reversible). Because every service is already a container with a Dockerfile and externalized config/secrets, moving to Kubernetes later is writing manifests — not a rewrite. We lose nothing by waiting, and we avoid paying the cluster tax before it buys us anything.