CI/CD pipelines
The automation that ties it together: a gated test→build→scan→deploy pipeline, staging + approvals, supply-chain security, and owning delivery via the DORA metrics — modeled in Python.
Learning objectives
- Explain CI/CD and why automation beats manual deploys.
- Build a test → build → scan → deploy pipeline.
- Stage with environments, approvals, and secrets.
- Own delivery: the maturity ladder and DORA metrics.
code/cd5-cicd/. Python runs offline; configs are ready to use.1 · What CI/CD is (and why) essential
CI runs tests on every push; CD builds and ships automatically when they pass. Manual deploys are slow and error-prone — someone forgets a step, deploys the wrong tag, skips tests. A pipeline does the same steps identically every time, gated on quality.
This is the whole idea of CI/CD in one line: the instant code lands in the main branch, an automated assembly line runs — no human clicking buttons. CI (Continuous Integration) is the testing part; CD (Continuous Delivery/Deployment) is the shipping part. Read the boxes left to right; each arrow only fires if the box before it succeeded.
- Push/merge (trigger) — someone merges code. That event alone kicks off everything to the right; nobody runs steps by hand.
- Test (CI) — the automated test suite runs first. This is the gate: if tests fail, the arrow to the next box never fires and nothing ships.
- Build + scan + push — only reached when tests pass. It packages the app into a container image, scans that image for security problems, and pushes it to an image registry.
- Deploy (CD) — the final box takes the freshly built image and puts it live. Because it sits last, code can only reach production after passing every earlier stage.
In short: A pipeline is just steps in a fixed order, each guarding the next. The single most important guard is 'tests must pass before deploy' — that is what makes automated shipping safe instead of scary.
2 · A first pipeline essential
deploy.ymlname: CI/CD
on: { push: { branches: [main] } }
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install -r requirements.txt pytest
- run: pytest -q
build-deploy:
needs: test # ONLY runs if tests pass
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
IMG=ghcr.io/${{ github.repository }}:${{ github.sha }}
echo "${{ secrets.GHCR_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker build -t "$IMG" .
docker push "$IMG"
- run: echo "deploy the tagged image (gcloud run deploy / aws ecs update-service)"
This is a GitHub Actions workflow — a YAML config file (not a program you run) that lives in your repo at .github/workflows/. GitHub reads it and does what it says. YAML uses indentation to show what belongs to what, just like Python. Read it as three questions: when should this run, what jobs are there, and what steps does each job take.
on: { push: { branches: [main] } }is the trigger: run this workflow whenever code is pushed to themainbranch.jobs:lists the units of work. The first job,test,runs-on: ubuntu-latest— GitHub spins up a fresh Linux machine for it.- A job's
steps:run top to bottom.uses:pulls in a prebuilt action (check out the code, install Python);run:executes a shell command. Here the steps install dependencies then runpytest -q— that is the test stage. - The second job,
build-deploy, hasneeds: test— it waits for the test job and only starts if tests passed. Its steps log in to the registry,docker buildan image tagged with the exact commit (github.sha),docker pushit, then deploy.
What the output means: Nothing prints locally — GitHub runs this in the cloud on every push. In the Actions tab you'd see two boxes: test (green), then build-deploy (green). If pytest failed, build-deploy would be skipped and nothing would ship.
Try this: Notice the values wrapped in ${{ ... }} — those are filled in by GitHub at run time (the repo name, the commit SHA, and a secret token). secrets.GHCR_TOKEN is a password stored safely by GitHub, never written in the file.
needs: test is the whole pointThe deploy job depends on the test job — a red suite stops the pipeline before deploy. Broken code physically cannot reach production. That one line encodes safe delivery.3 · Model a pipeline in Python intermediate
A pipeline is stages that short-circuit on failure. Modeling it makes the control flow explicit — and it's how a custom deploy tool works under the hood.
pipeline.pydef run_pipeline(stages):
"""stages: [(name, fn)]. Stop at the first failure; report where."""
for name, fn in stages:
ok = fn()
print(f"{'PASS' if ok else 'FAIL'} {name}")
if not ok:
return False, name # nothing after a failure runs
return True, None
ok = [("lint",lambda:True),("test",lambda:True),("build",lambda:True),("deploy",lambda:True)]
print("result:", run_pipeline(ok))
broken = [("lint",lambda:True),("test",lambda:False),("deploy",lambda:True)]
print("result:", run_pipeline(broken)) # stops at test; deploy never runs
PASS lint
PASS test
PASS build
PASS deploy
result: (True, None)
PASS lint
FAIL test
result: (False, 'test')
This tiny Python program models what a pipeline does: run a list of stages in order and stop the moment one fails. Seeing it as plain code makes the 'gate' idea concrete — a real deploy tool does exactly this under the hood.
run_pipeline(stages)takes a list of(name, fn)pairs — each stage has a label and a function to call.fnreturnsTruefor pass,Falsefor fail.- The
forloop runs each stage in turn:ok = fn()runs it, then it printsPASSorFAILnext to the stage name. if not ok: return False, nameis the short-circuit — the first failing stage stops the loop immediately and reports which stage broke. Any stages listed after it never run.- The two example lists show both paths:
okpasses every stage;brokenfails attest, sodeployis never reached.
What the output means: First run prints PASS for all four stages and (True, None) — the whole pipeline succeeded. Second run prints PASS lint, FAIL test, then (False, 'test') — it stopped at test and deploy never ran, exactly like a real CI gate.
Try this: Flip the first lambda:True to lambda:False and run again — the pipeline should stop at lint and never print the later stages.
4 · Security scanning in the pipeline advanced
Professional pipelines scan before deploy: the image for known CVEs (Trivy/Grype), dependencies for vulnerabilities, and code for leaked secrets. Fail the build on high-severity findings.
scan_gate.pydef scan_gate(findings, max_high=0, max_critical=0):
highs = sum(1 for f in findings if f["severity"] == "HIGH")
crits = sum(1 for f in findings if f["severity"] == "CRITICAL")
passed = highs <= max_high and crits <= max_critical
return passed, {"high": highs, "critical": crits}
clean = scan_gate([{"severity":"LOW"},{"severity":"MEDIUM"}])
print("clean image:", clean)
vuln = scan_gate([{"severity":"CRITICAL"},{"severity":"HIGH"},{"severity":"LOW"}])
print("vulnerable image:", vuln, "-> block deploy")
clean image: (True, {'high': 0, 'critical': 0})
vulnerable image: (False, {'high': 1, 'critical': 1}) -> block deploy
Before shipping, professional pipelines scan the container image for known security holes (CVEs). This function is the gate that decides: given a list of findings, is the image safe enough to deploy? It counts the serious ones and says pass or fail.
scan_gate(findings, max_high=0, max_critical=0)takes the scan results plus two thresholds — how many HIGH and CRITICAL issues you will tolerate. The defaults are0: zero tolerance.highs = sum(1 for f in findings if f["severity"] == "HIGH")counts how many findings are HIGH severity; the next line counts CRITICAL the same way.passed = highs <= max_high and crits <= max_criticalis the decision —Trueonly if both counts stay within the allowed limits. It returns that verdict plus the counts.- The two calls show both outcomes: a
cleanimage with only LOW/MEDIUM findings passes; avulnimage with a CRITICAL and a HIGH fails.
What the output means: clean image: (True, {'high': 0, 'critical': 0}) — safe to ship. vulnerable image: (False, {'high': 1, 'critical': 1}) -> block deploy — the gate said no, so the pipeline stops before deploy.
Try this: Raise the bar with scan_gate(findings, max_high=5) and see the vulnerable image pass — that is how teams temporarily accept known risks. Setting limits to 0 is the strict, safe default.
5 · Advanced — environments & approvals advanced
staged.yml deploy-staging:
needs: test
environment: staging # auto-deploy to staging on green
steps: [ ... deploy to staging + run smoke tests ... ]
deploy-prod:
needs: deploy-staging
environment:
name: production # GitHub requires a human approval here
steps: [ ... deploy to prod (canary via CD4) ... ]
This config adds a safety layer for real deploys: ship to a staging copy automatically, but require a human to click 'approve' before touching production. It is two more jobs chained onto the pipeline with needs:.
deploy-staginghasneeds: testandenvironment: staging— once tests are green it auto-deploys to a staging environment (a production-like sandbox) and runs smoke tests there.deploy-prodhasneeds: deploy-staging— it can only start after staging succeeded. This chains the jobs into a strict order.- Under
environment:the nameproductionis special: GitHub can be configured so a named environment requires a human approval before its job runs. That is the manual gate before real users are affected. - The
steps: [ ... ]are shown as placeholders — the real deploy commands (a canary rollout, from lesson CD4) would go there.
Try this: The pattern is automatic to staging, approved to prod. Staging catches problems safely; the human approval is your last chance to say 'not yet' before customers see a change.
6 · Professional — secrets & supply chain professional
CI has powerful credentials — a prime target. Use the platform's secret store (never hardcode), scope tokens to least privilege, pin action versions (a compromised third-party action can steal secrets), and sign/attest images so you can prove what you shipped.
7 · Tech-lead — measure delivery with DORA tech-lead
A delivery lead manages by the DORA metrics: deploy frequency, lead time for changes, change-failure rate, and time-to-restore. They tell you whether delivery is elite or struggling, and where to invest.
dora.pydef dora_tier(deploys_per_week, lead_time_h, change_fail_pct, restore_h):
elite = deploys_per_week>=7 and lead_time_h<=24 and change_fail_pct<=15 and restore_h<=1
high = deploys_per_week>=1 and lead_time_h<=168 and change_fail_pct<=30 and restore_h<=24
return "Elite" if elite else "High" if high else "Medium/Low"
print("team A:", dora_tier(14, 4, 10, 0.5)) # ships daily, recovers in minutes
print("team B:", dora_tier(0.5, 336, 40, 72)) # rare big releases, slow recovery
print("improve B: raise deploy freq (smaller changes) + cut restore time (rollback)")
team A: Elite
team B: Medium/Low
improve B: raise deploy freq (smaller changes) + cut restore time (rollback)
The DORA metrics are the industry-standard way to score how well a team delivers software. This function takes four numbers and classifies the team as Elite, High, or Medium/Low — turning a fuzzy 'are we good at shipping?' into a concrete answer.
- The four inputs are the four DORA metrics:
deploys_per_week(how often you ship),lead_time_h(hours from commit to live),change_fail_pct(percent of deploys that break something), andrestore_h(hours to recover from a failure). elite = ...checks all four against the toughest bar at once withand— every condition must be true to qualify as Elite.high = ...checks a looser bar the same way.return "Elite" if elite else "High" if high else "Medium/Low"picks the best tier the team qualifies for, top-down.- The two example teams contrast the extremes: team A ships often and recovers in minutes; team B does rare, large releases and recovers slowly.
What the output means: team A: Elite (frequent, fast, reliable) and team B: Medium/Low. The last line names the fix for B: ship smaller changes more often and make recovery (rollback) faster.
Try this: Change team B's deploys_per_week from 0.5 to 3 and its restore_h from 72 to 10 — watch it climb toward High. Small, frequent, reversible changes are what move the needle.
Exercise CD5.1 — Build & measure a pipeline
Context: The whole Containers & Deployment track converges in the pipeline: git triggers it, tests gate it, the image is built and scanned, it deploys with canary + rollback, and DORA measures it. Building and then measuring your own pipeline is the professional delivery endgame.
Your task: Write a workflow that tests → builds+pushes a SHA-tagged image → deploys, gated on green, add a scan_gate step, model your release with run_pipeline, and score your team with dora_tier.
Requirements:
- Write a green-gated workflow: test, then build+push a SHA-tagged image, then deploy
- Add a
scan_gatestep that blocks on high-severity findings - Model the release control flow with
run_pipeline - Score your team (or a hypothetical one) with
dora_tier - Name the one DORA metric you'd improve first and exactly how
💡 Hint: Tie the chapter's pieces together — the needs: gate, the scan gate, and DORA — then let the lowest DORA metric point at your first improvement.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: CI runs tests on every push; CD builds and ships automatically when they pass. The pipeline is just steps in a fixed order, each guarding the next — and one guard is what makes automated shipping safe instead of scary.
Your task: Explain the core CI/CD pipeline (push → test → build+scan+push → deploy) and identify the single most important guard the lesson calls 'the whole point'.
Requirements:
- Describe the four stages in order: trigger, test, build+scan+push, deploy
- Explain that each arrow fires only if the stage before it succeeded
- Identify 'tests must pass before deploy' as the critical guard
- Name the GitHub Actions mechanism that encodes it: the deploy job's
needs: test - State the payoff: broken code physically cannot reach production
💡 Hint: Everything hinges on the test stage gating the deploy stage — without that dependency, automation just ships bugs faster.
Show solution
CI runs tests on every push; CD builds and ships automatically when they pass. A pipeline runs the same steps identically every time, each stage guarding the next.
Push/merge (trigger)
-> Test (CI) # the gate: if tests fail, nothing after runs
-> Build + scan + push # package image, scan for CVEs, push to registry
-> Deploy (CD) # put the built image live
The critical guard is tests must pass before deploy. In GitHub Actions that is the deploy job's needs: test line — broken code physically cannot reach production, which is what makes automated shipping safe instead of scary.
Context: A GitHub Actions workflow is a YAML config GitHub reads and runs in the cloud on every push. Splitting test and deploy into separate jobs, with the deploy depending on the test, is the canonical safe-delivery shape.
Your task: Write a GitHub Actions workflow (.github/workflows/deploy.yml) that on push to main runs pytest, then in a separate job (only if tests pass) builds and pushes a SHA-tagged image to GHCR.
Requirements:
- Trigger on
pushto themainbranch - Give the test job steps that check out code, set up Python, install deps, and run
pytest -q - Declare
needs: teston the build-deploy job so it waits for and depends on the tests - Tag the image with the exact commit via
${{ github.sha }} - Read the registry token from
secretsand log in with--password-stdin
💡 Hint: The ${{ ... }} values are filled in by GitHub at run time; a failing pytest skips the build-deploy job so nothing ships.
Show solution
Trigger on push to main; the build-deploy job declares needs: test so it waits for and depends on the tests. Tag the image with the exact commit via github.sha, and read the registry token from GitHub secrets.
name: CI/CD
on: { push: { branches: [main] } }
jobs:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- run: pip install -r requirements.txt pytest
- run: pytest -q
build-deploy:
needs: test # ONLY runs if tests pass
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: |
IMG=ghcr.io/${{ github.repository }}:${{ github.sha }}
echo "${{ secrets.GHCR_TOKEN }}" | docker login ghcr.io -u ${{ github.actor }} --password-stdin
docker build -t "$IMG" .
docker push "$IMG"
- run: echo "deploy the tagged image (gcloud run deploy / aws ecs update-service)"
If pytest fails, build-deploy is skipped and nothing ships. The ${{ ... }} values are filled in by GitHub at run time; the token is stored safely, never in the file.
Context: A pipeline is stages that short-circuit on failure — and a real deploy tool does exactly this under the hood. Modeling it in plain Python makes the 'gate' idea concrete.
Your task: Model a CI pipeline in Python as stages that short-circuit on failure: run (name, fn) stages in order, stop at the first failing one, and report which stage broke.
Requirements:
- Take a list of
(name, fn)stages where each fn returns True (pass) or False (fail) - Run each stage in order and print PASS or FAIL next to its name
- Stop immediately on the first failing stage — nothing after it runs
- Return a success boolean plus the name of the stage that broke (or None)
- Demonstrate a clean run and a run that fails at
testsodeploynever runs
💡 Hint: A single return inside the loop on the first failure is the short-circuit; it mirrors how a red test suite stops a real pipeline before deploy.
Show solution
This is how a real deploy tool works under the hood: run each stage, print pass/fail, and return immediately on the first failure so nothing after it runs.
def run_pipeline(stages):
"""stages: [(name, fn)]. Stop at the first failure; report where."""
for name, fn in stages:
ok = fn()
print(f"{'PASS' if ok else 'FAIL'} {name}")
if not ok:
return False, name # nothing after a failure runs
return True, None
ok = [("lint", lambda: True), ("test", lambda: True),
("build", lambda: True), ("deploy", lambda: True)]
print("result:", run_pipeline(ok))
broken = [("lint", lambda: True), ("test", lambda: False), ("deploy", lambda: True)]
print("result:", run_pipeline(broken)) # stops at test; deploy never runs
The clean run reports (True, None); the broken run prints FAIL at test and returns (False, 'test') — deploy never runs, exactly like a real CI gate.
Context: Professional pipelines scan the image for known CVEs before deploy and fail the build on high-severity findings. The gate that decides pass or fail is a small, configurable function.
Your task: Add a security-scan gate to the pipeline in Python: given a list of scan findings, block the deploy if HIGH or CRITICAL counts exceed configurable thresholds (default zero tolerance).
Requirements:
- Take the findings plus
max_highandmax_criticalthresholds (both default 0) - Count HIGH-severity and CRITICAL-severity findings separately
- Pass only when both counts stay within their allowed limits
- Return the verdict plus the high/critical counts
- Show a clean image passing and a vulnerable image (a CRITICAL and a HIGH) blocking the deploy
💡 Hint: Zero thresholds are the strict, safe default; raising max_high is how a team consciously accepts a known risk for a while.
Show solution
Professional pipelines scan the image for known CVEs (Trivy/Grype) before deploy. Count the serious findings and compare against thresholds.
def scan_gate(findings, max_high=0, max_critical=0):
highs = sum(1 for f in findings if f["severity"] == "HIGH")
crits = sum(1 for f in findings if f["severity"] == "CRITICAL")
passed = highs <= max_high and crits <= max_critical
return passed, {"high": highs, "critical": crits}
clean = scan_gate([{"severity":"LOW"}, {"severity":"MEDIUM"}])
print("clean image:", clean)
vuln = scan_gate([{"severity":"CRITICAL"}, {"severity":"HIGH"}, {"severity":"LOW"}])
print("vulnerable image:", vuln, "-> block deploy")
# clean: (True, {'high': 0, 'critical': 0})
# vuln: (False, {'high': 1, 'critical': 1}) -> block deploy
Zero thresholds are the strict, safe default; raising max_high is how a team temporarily accepts a known risk. The gate fails the build on high-severity findings before deploy.
Context: Real deploys ship to a staging copy automatically but require a human to approve before touching production. And because a pipeline that can deploy to prod is as sensitive as prod, its credentials and third-party actions are a prime attack target.
Your task: Write the staged-deploy config: auto-deploy to staging on green, then a production job that requires a human approval, chained with needs: — and state why the pipeline itself must be treated as production infrastructure.
Requirements:
- Give
deploy-stagingneeds: testand anenvironment: stagingso it auto-deploys on green and runs smoke tests - Chain
deploy-prodwithneeds: deploy-stagingso it can't start until staging succeeds - Use a named
productionenvironment that GitHub can require a human approval for - Summarize the pattern: automatic to staging, approved to prod
- Explain the supply-chain risk: use the secret store, scope tokens to least privilege, and pin every third-party action version
💡 Hint: The named production environment is the manual gate — the last chance to say 'not yet' before customers see the change.
Show solution
Ship to a staging copy automatically, but require a human to approve before touching production. Two jobs chained with needs: enforce the order.
deploy-staging:
needs: test
environment: staging # auto-deploy to staging on green
steps: [ ... deploy to staging + run smoke tests ... ]
deploy-prod:
needs: deploy-staging
environment:
name: production # GitHub requires a human approval here
steps: [ ... deploy to prod (canary via CD4) ... ]
The pattern is automatic to staging, approved to prod. A named production environment can be configured to require manual approval — the last chance to say "not yet." Because a pipeline that can deploy to prod is as sensitive as prod, use the platform's secret store (never hardcode), scope tokens to least privilege, and pin every third-party action version — supply-chain attacks target exactly this.
Context: A delivery lead manages by the four DORA metrics — they turn a fuzzy 'are we good at shipping?' into a concrete tier and show where to invest. The fix for a struggling team is almost always smaller, more frequent, reversible changes.
Your task: As a delivery lead, write a Python function that classifies a team's delivery performance by the four DORA metrics (deploy frequency, lead time, change-failure rate, time-to-restore) into Elite / High / Medium-Low.
Requirements:
- Take the four metrics as inputs: deploys per week, lead time (h), change-failure %, restore time (h)
- Check all four against the toughest bar with
andfor Elite - Check a looser bar the same way for High
- Return the best tier the team qualifies for, chosen top-down
- Contrast a frequent/fast/reliable team (Elite) with a rare-big-release team (Medium/Low)
💡 Hint: All four conditions must hold to reach a tier, so a single weak metric (slow restore, rare deploys) caps the team — that's the metric to improve first.
Show solution
Check all four metrics against the toughest bar with and for Elite, a looser bar for High, and pick the best tier top-down.
def dora_tier(deploys_per_week, lead_time_h, change_fail_pct, restore_h):
elite = deploys_per_week >= 7 and lead_time_h <= 24 and change_fail_pct <= 15 and restore_h <= 1
high = deploys_per_week >= 1 and lead_time_h <= 168 and change_fail_pct <= 30 and restore_h <= 24
return "Elite" if elite else "High" if high else "Medium/Low"
print("team A:", dora_tier(14, 4, 10, 0.5)) # ships daily, recovers in minutes
print("team B:", dora_tier(0.5, 336, 40, 72)) # rare big releases, slow recovery
print("improve B: raise deploy freq (smaller changes) + cut restore time (rollback)")
# team A: Elite / team B: Medium/Low
Team A is Elite (frequent, fast, reliable); team B is Medium/Low. The two changes that most directly move B toward Elite are shipping smaller changes far more often (raising deploy frequency, cutting lead time) and making recovery fast via rollback (cutting time-to-restore) — small, frequent, reversible changes are what move the needle.
✓ Checkpoint — you can move on when you can…
- Explain CI vs CD and the
needs: testgate. - Build a test→build→scan→deploy pipeline.
- Stage with environments/approvals; secure the supply chain.
- Measure delivery with DORA and drive improvement.
Knowledge check check yourself
In the GitHub Actions workflow, the build-deploy job declares needs: test. Why is that one line described as 'the whole point' of the pipeline?
Show answer
A team scores Medium/Low on the DORA metrics with rare, large releases and slow recovery. Which two changes most directly move it toward Elite, and why?