Software Testing · Chapter TQ5

Integration tests, coverage & CI

Q: The coverage lab reports 1/2 branches = 50% and names discount:guard as missing. Why does the lesson insist that 100% coverage is not the same as correctness?

Coverage only tells you a line ran, not that you asserted the right thing about it. It's a gap-finder that flags untested branches (here the error guard), but a line can be executed with a wrong or absent assertion — so you aim for meaningful coverage of important logic (~80-90% on core), not a vanity 100%.

From units to the whole picture: integration tests, honest coverage, CI gates, quality gates, and leading a testing culture — where Git, Testing, and Deployment converge.

⏱️ ~2.5 hours🧪 5 labs🎯 Advanced→Tech-lead

Learning objectives

Write integration tests across components.
Measure coverage honestly.
Run the suite in CI as a required gate.
Own quality gates and a testing culture for a team.

▶ Runnable companionThe code here is also saved under code/tq5-integration-ci/. Python blocks run offline; config files are ready to drop into a project.

1 · Integration tests essential

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.

Python · service + real in-memory DB (runs)

integration.pyimport sqlite3
class OrderStore:
    def __init__(self, conn):
        self.conn = conn
        conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, total REAL)")
    def create(self, total):
        c = self.conn.execute("INSERT INTO orders(total) VALUES (?)", (total,))
        self.conn.commit(); return c.lastrowid
    def get(self, oid):
        return self.conn.execute("SELECT total FROM orders WHERE id=?", (oid,)).fetchone()

store = OrderStore(sqlite3.connect(":memory:"))     # integration: store + DB
oid = store.create(99.0)
assert store.get(oid)[0] == 99.0
print("integration test passed (store + db together)")

integration test passed (store + db together)

▶ How this works

A unit test checks one small piece on its own (often with the database faked out). An integration test checks that two real pieces work together — here, your OrderStore code plus a real SQLite database. If the code and the database disagree about how data is stored, only an integration test can catch it. The trick that keeps this fast is :memory:: a throwaway database that lives in RAM and vanishes when the program ends.

class OrderStore wraps a database connection. Its __init__ immediately runs CREATE TABLE orders, so a fresh store always has a real table to write into.
create(total) runs a real SQL INSERT, calls commit() to save it, and returns lastrowid — the id the database assigned to the new row.
get(oid) runs a SELECT and fetchone() pulls back that one row as a tuple, e.g. (99.0,).
store = OrderStore(sqlite3.connect(":memory:")) is the integration part: a real store talking to a real (in-memory) database. We then create an order, read it back, and assert store.get(oid)[0] == 99.0 — [0] grabs the first column (the total).

What the output means: assert raises an error and stops the program if the value is wrong. Because it matched, we reach the last line and print integration test passed (store + db together).

Try this: Change the assert to == 88.0 and re-run — you'll get an AssertionError, which is the test correctly telling you the stored value doesn't match what you expected.

2 · Coverage intermediate

Coverage shows which lines your tests execute — a gap-finder, not a quality guarantee. Run pytest --cov; here's a tiny coverage tracker to show the idea runnably.

Python · what coverage measures (runs)

coverage.pyexecuted = set()
def tracked(fn_name): executed.add(fn_name)

def discount(price, pct):
    tracked("discount")
    if pct > 100:
        tracked("discount:guard"); raise ValueError
    return price*(1-pct/100)

discount(100, 10)                    # tests exercise the happy path only
all_branches = {"discount", "discount:guard"}
covered = executed & all_branches
print(f"coverage: {len(covered)}/{len(all_branches)} branches = {100*len(covered)//len(all_branches)}%")
print("missing:", all_branches - executed)   # the guard is untested!

coverage: 1/2 branches = 50%
missing: {'discount:guard'}

▶ How this works

Coverage answers one question: which lines of your code did the tests actually run? It is a gap-finder — it shows code no test ever touched. This tiny program fakes a coverage tool by recording the name of each branch as it executes, then comparing what ran against everything that could run.

executed = set() is our record of which parts ran. tracked(fn_name) adds a label to that set — think of it as a checkmark saying "this line was reached".
Inside discount, the normal path calls tracked("discount"). The error path (when pct > 100) calls tracked("discount:guard") — but only if we ever hit that case.
We call discount(100, 10) once. Since 10 > 100 is false, the guard branch never runs, so "discount:guard" is never recorded.
covered = executed & all_branches uses set intersection (&) to find which of the two possible branches actually ran, then prints a percentage.

What the output means: coverage: 1/2 branches = 50% — one of two branches ran. missing: {'discount:guard'} names the untested branch: the error guard. That is the whole point — coverage flags the risky line no test exercised.

Try this: Add a second call discount(100, 150) above the report (wrap it in try/except ValueError so it doesn't crash). Now the guard runs and coverage jumps to 2/2 = 100%.

100% coverage != correctCoverage says a line ran, not that you asserted the right thing. Aim for meaningful coverage of important logic (~80-90% on core), not a vanity 100%.

3 · Advanced — CI gate advanced

config · tests run on every push

tests.ymlname: tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements.txt pytest pytest-cov
      - run: pytest --cov=src --cov-report=term-missing
      # a failing test fails the job -> the PR check goes red

▶ How this works

This is not Python — it's a configuration file (YAML) that tells GitHub Actions what to do automatically. CI (Continuous Integration) means: every time someone pushes code or opens a pull request, a fresh machine in the cloud checks out the code and runs your tests for you, so nobody can forget. YAML uses indentation to show nesting — the deeper a line is indented, the more it "belongs to" the line above it.

on: [push, pull_request] is the trigger: run this whenever code is pushed or a pull request is opened/updated.
jobs: lists the work to do. Here there's one job called test, and runs-on: ubuntu-latest says "do it on a fresh Ubuntu Linux machine".
steps: are run top to bottom. The two uses: lines pull in ready-made actions — checkout downloads your code onto the machine, setup-python installs Python 3.12.
The two run: lines are shell commands: first pip install the dependencies plus pytest, then pytest --cov=src to run the tests and measure coverage.

What the output means: There's no console output to read here — this file just defines the pipeline. As the last comment says, if any test fails, pytest exits with an error, which fails the job, which turns the PR's check red — blocking a broken merge.

Try this: Save this as .github/workflows/tests.yml in a real repo and push. Open the "Actions" tab on GitHub to watch it run your tests automatically — green means all passed.

4 · Professional — quality gates professional

A quality gate is an automated pass/fail on a PR: tests green, coverage above a floor, no new lint errors. Encode it so it's objective, not a reviewer's mood.

Python · a quality gate check (runs)

quality_gate.pydef quality_gate(results):
    failures = []
    if not results["tests_pass"]:             failures.append("tests failing")
    if results["coverage"] < results["min_coverage"]: failures.append(
        f"coverage {results['coverage']}% < {results['min_coverage']}%")
    if results["new_lint_errors"] > 0:        failures.append(
        f"{results['new_lint_errors']} new lint errors")
    return (not failures), failures

ok, why = quality_gate({"tests_pass": True, "coverage": 82, "min_coverage": 80, "new_lint_errors": 0})
print("PR mergeable:", ok)
ok2, why2 = quality_gate({"tests_pass": True, "coverage": 71, "min_coverage": 80, "new_lint_errors": 3})
print("PR mergeable:", ok2, "|", why2)

PR mergeable: True
PR mergeable: False | ['coverage 71% < 80%', '3 new lint errors']

▶ How this works

A quality gate turns "is this PR good enough to merge?" into an objective yes/no, instead of a reviewer's gut feeling. This function takes a summary of a PR's results and returns whether it passes plus a list of reasons it failed — so the feedback is specific.

failures = [] starts an empty list. Each rule that is violated appends a short reason string to it; a clean PR leaves the list empty.
Rule 1: if not results["tests_pass"] — if the tests didn't pass, record "tests failing".
Rule 2: if coverage is below min_coverage (the floor), record a message showing both numbers. Rule 3: if there are any new lint errors, record that count.
return (not failures), failures returns two things: not failures is True only when the list is empty (an empty list is "falsy"), and the list itself explains any failure.

What the output means: First call has good numbers, so PR mergeable: True. The second has coverage 71 (below the 80 floor) and 3 lint errors, so it prints False followed by the exact reasons: ['coverage 71% < 80%', '3 new lint errors'].

Try this: Lower min_coverage to 70 in the second call and re-run — the coverage complaint disappears, leaving only the lint failure. This is how a real gate is tuned per project.

5 · Tech-lead — build a testing culture tech-lead

Tools don't create quality — culture does. A lead makes the healthy path the easy path: required green CI, fast tests, flaky tests fixed immediately, tests expected with every change, and bug-fix-first (TQ4). This is where Git (DF3), Testing (TQ), and Deployment (CD) converge on the PR.

Practice	Why it holds the line
required green CI to merge	no broken code reaches main
fast suite (mock slow deps)	people actually run it
fix/delete flaky tests now	preserves trust in red = real
tests with every feature/fix	coverage grows with the code
coverage floor on core modules	risk-weighted, not vanity

This is the professional endgameA codebase where every PR is gated on a fast, trusted, meaningful test suite is one a team can change quickly and safely for years. Building and defending that culture is a defining tech-lead responsibility.

Exercise TQ5.1 — Gate a repo

Context: The capstone wires everything together: a genuine integration test, coverage in CI, a composed quality gate, and proof — via a deliberately failing PR — that the gate actually blocks bad merges.

Your task: Add an integration test backed by a real in-memory DB, wire pytest --cov into CI, encode a quality_gate (tests green plus a coverage floor), make the check required, and open a PR that violates the gate to confirm it is blocked.

Requirements:

An integration test runs against a real in-memory sqlite3 database
CI runs pytest --cov with an enforced coverage floor
A quality_gate combines the test result and the coverage floor into one pass/fail
The CI check is marked required so it blocks merges
A deliberately gate-violating PR is opened and shown to be blocked
The green suite plus the gate — not reviewer goodwill — enforce the bar

💡 Hint: Prove the gate by breaking it on purpose: open a PR whose coverage dips below the floor (or whose tests fail) and confirm the required check turns the merge red.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Integration test with a real in-memory DBBeginner

Context: Integration tests exercise components together rather than in isolation. Using a real in-memory sqlite3 database gives genuine SQL behavior — constraints, aggregates — at unit-test speed, with no mocks.

Your task: Test an OrderStore against a real in-memory sqlite3 database, inserting rows and asserting an aggregate over real SQL.

Requirements:

Back the store with sqlite3.connect(":memory:"), not a mock
The store creates its table and exposes add and total operations
total() runs a real SUM query against the DB
setUp gives each test a fresh in-memory database
Assert the aggregate after inserting a couple of rows
Runs offline under unittest

💡 Hint: An in-memory DB is real SQL that vanishes when the connection closes — you get authentic integration behavior without any external server or fixture files.

Show solution

Use a real DB in memory so the integration is genuine but fast:

import unittest, sqlite3

class OrderStore:
    def __init__(self, conn):
        self.conn = conn
        conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, amt REAL)")
    def add(self, amt): self.conn.execute("INSERT INTO orders (amt) VALUES (?)", (amt,))
    def total(self):
        return self.conn.execute("SELECT COALESCE(SUM(amt),0) FROM orders").fetchone()[0]

class TestOrderStore(unittest.TestCase):
    def setUp(self):
        self.store = OrderStore(sqlite3.connect(":memory:"))   # real DB, fresh
    def test_total(self):
        self.store.add(10); self.store.add(5)
        self.assertEqual(self.store.total(), 15)

if __name__ == "__main__":
    unittest.main(verbosity=2)     # exercises store + real SQL together

The in-memory DB gives real SQL behavior (constraints, aggregates) with unit-test speed — the sweet spot for integration tests.

Exercise 2 · Measure coverage honestlyIntermediate

Context: Coverage is a map of what your tests do not touch. Modeling it by hand — given which lines ran — demystifies the number and shows that high coverage still does not prove correctness.

Your task: Model line coverage offline: given a total line count and the executed lines, compute the coverage percentage and list the uncovered lines.

Requirements:

coverage(total_lines, executed_lines) returns a percentage and the missing lines
Compute covered lines as the intersection of executed lines with the valid range
Return the percentage rounded to one decimal place
Return the sorted list of lines that were never executed
Demonstrate it on a small example and print the gap
Note that 100% coverage still does not prove correctness

💡 Hint: Work with sets over the line range: covered is the intersection, missing is the difference — the gap is a to-do list, not a verdict.

Show solution

Coverage is a map of what's untested — computed here without the tool:

def coverage(total_lines, executed_lines):
    covered = len(set(executed_lines) & set(range(1, total_lines + 1)))
    pct = 100.0 * covered / total_lines
    missing = sorted(set(range(1, total_lines + 1)) - set(executed_lines))
    return round(pct, 1), missing

pct, missing = coverage(total_lines=10, executed_lines=[1,2,3,4,5,6,7,8])
print(f"{pct}% covered; missing lines: {missing}")
# 80.0% covered; missing lines: [9, 10]

The 20% gap is a to-do list, not a failure — coverage tells you where tests are missing, but 100% coverage still doesn't prove correctness.

Exercise 3 · The CI gate (worked YAML)Advanced

Context: A test suite that no one runs is worthless. Wiring the suite into CI as a required gate makes a red test block the merge automatically — tests become a gate, not a suggestion.

Your task: Write a GitHub Actions workflow (worked YAML) that installs dependencies and runs the test suite on every push and pull request, failing the build on red. (Needs a CI runner.)

Requirements:

Trigger on both push and pull_request
Check out the repo and set up a pinned Python version
Install the project and pytest
Run pytest as the test step
A non-zero pytest exit fails the job and blocks the merge
The YAML lives at .github/workflows/

💡 Hint: The gate is automatic: pytest exits non-zero on any failure, and a non-zero step fails the whole job — no extra wiring needed to block a red merge.

Show solution

Worked YAML — the minimal test gate:

# .github/workflows/ci.yml   (needs GitHub Actions)
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -e . pytest
      - name: Test
        run: pytest -q          # non-zero exit fails the job = blocks merge

Because a non-zero pytest exit fails the job, a red test blocks the merge automatically — tests become a gate, not a suggestion.

Exercise 4 · Coverage gate in CI (worked YAML)Expert

Context: Coverage that is never enforced silently rots. A floor in CI — not a target of 100% — blocks regressions in coverage without incentivizing meaningless tests that chase the last percent.

Your task: Extend the CI workflow to run coverage and fail the build if it drops below a threshold, using --cov-fail-under. (Needs pytest-cov.)

Requirements:

Install pytest-cov alongside pytest
Run pytest with --cov against the source package
Enforce a floor with --cov-fail-under=<N> (e.g. 80)
Below the floor, pytest exits non-zero and fails the build
Use a floor rather than demanding 100% coverage
Report term-missing so uncovered lines are visible in the log

💡 Hint: The floor keeps the gate honest — it catches coverage regressions while leaving room to skip tests that would only pad the number.

Show solution

Worked YAML — enforce a coverage floor so it can't silently rot:

# .github/workflows/coverage.yml   (needs pytest + pytest-cov)
name: coverage
on: [push, pull_request]
jobs:
  cov:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -e . pytest pytest-cov
      - name: Test with coverage floor
        run: pytest --cov=src --cov-report=term-missing --cov-fail-under=80
      # --cov-fail-under=80 exits non-zero (fails the build) below 80% coverage.

A floor (not a target of 100%) keeps the gate honest: it blocks regressions in coverage without incentivizing meaningless tests to chase the last percent.

Exercise 5 · Compose quality gatesProfessional

Context: Professional CI gates on more than tests. A single gate function that requires tests to pass, coverage to clear a floor, and no critical lint errors makes the build green only when every check passes.

Your task: Model a quality gate offline that ANDs several checks — tests pass, coverage ≥ floor, zero critical lint errors — and returns both a pass/fail and the blocking reasons.

Requirements:

quality_gate(results, coverage_floor) inspects a results dict
Fail if tests are not passing
Fail if coverage is below the floor
Fail if there are any critical lint errors
Return a boolean plus a list of human-readable blocking reasons
Demonstrate one passing and one failing set of results

💡 Hint: Accumulate reasons as you check each condition, then the overall pass is simply "no reasons" — the reasons list tells the author exactly what to fix.

Show solution

A single gate that ANDs the quality checks — modeled offline:

def quality_gate(results, coverage_floor=80):
    reasons = []
    if not results["tests_pass"]:
        reasons.append("tests failing")
    if results["coverage"] < coverage_floor:
        reasons.append(f"coverage {results['coverage']}% < {coverage_floor}%")
    if results["critical_lint"] > 0:
        reasons.append(f"{results['critical_lint']} critical lint errors")
    return (not reasons), reasons

ok  = {"tests_pass": True, "coverage": 88, "critical_lint": 0}
bad = {"tests_pass": True, "coverage": 72, "critical_lint": 2}
print(quality_gate(ok))    # (True, [])
print(quality_gate(bad))   # (False, ['coverage 72% < 80%', '2 critical lint errors'])

The build is green only when every check passes; the reasons list tells the author exactly what to fix — the gate that maps to the CI YAML above.

Exercise 6 · Lead a testing culture (health check)Industry scenario

Context: Tech-lead reality: testing culture is measurable. A scorecard over signals — flaky-test rate, suite time, coverage trend, share of PRs with tests — turns "how healthy is our testing?" into a grade and a concrete next investment.

Your task: Encode a health scorecard over testing signals that outputs a health grade and names the single top risk to address.

Requirements:

health(signals) scores each signal against a threshold
Cover flaky-test rate, median suite time, coverage trend, and percent of PRs with tests
Sum the passing signals into a score and map it to a named grade
Identify and return the single biggest risk when any signal fails
Demonstrate it on a realistic set of signals
The output is actionable — a grade plus one place to invest

💡 Hint: Award a point per healthy signal and index a grade list by the total; the top risk is whichever failing signal you surface, giving a lead a concrete target instead of "test more".

Show solution

Turn "how healthy is our testing?" into a scored, actionable answer:

def health(signals):
    score = 0
    notes = []
    if signals["flaky_rate"] <= 0.01: score += 1
    else: notes.append(("flaky tests", signals["flaky_rate"]))
    if signals["suite_seconds"] <= 300: score += 1
    else: notes.append(("slow suite", signals["suite_seconds"]))
    if signals["coverage_trend"] >= 0: score += 1
    else: notes.append(("coverage falling", signals["coverage_trend"]))
    if signals["pct_prs_with_tests"] >= 0.8: score += 1
    else: notes.append(("PRs lack tests", signals["pct_prs_with_tests"]))
    grade = ["at risk","weak","fair","good","strong"][score]
    top_risk = max(notes, key=lambda x: 1)[0] if notes else "none"
    return grade, top_risk

s = {"flaky_rate": 0.03, "suite_seconds": 120,
     "coverage_trend": +2, "pct_prs_with_tests": 0.9}
print(health(s))   # ('good', 'flaky tests')

The scorecard grades the suite and names the single biggest risk (here, flaky tests) — giving a lead a concrete place to invest rather than a vague "test more".

✓ Checkpoint — you can move on when you can…

Write integration tests.
Measure and read coverage honestly.
Run tests in CI as a required gate.
Own quality gates and a testing culture.

Knowledge check check yourself

✓ Knowledge check

The coverage lab reports 1/2 branches = 50% and names discount:guard as missing. Why does the lesson insist that 100% coverage is not the same as correctness?

Show answer

Coverage only tells you a line ran, not that you asserted the right thing about it. It's a gap-finder that flags untested branches (here the error guard), but a line can be executed with a wrong or absent assertion — so you aim for meaningful coverage of important logic (~80-90% on core), not a vanity 100%.

✓ Knowledge check

The lesson's quality_gate turns "is this PR mergeable?" into an objective check. What three conditions does it enforce, and why encode it rather than leave it to a reviewer?

Show answer

It requires tests passing, coverage at or above a minimum floor, and zero new lint errors, returning the specific failing reasons. Encoding it makes the merge decision objective and consistent instead of a reviewer's mood, and it becomes the automated CI gate that keeps broken code out of main.

Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.

Learning objectives

1 · Integration tests essential

2 · Coverage intermediate

3 · Advanced — CI gate advanced

4 · Professional — quality gates professional

5 · Tech-lead — build a testing culture tech-lead

Exercise TQ5.1 — Gate a repo

🪜 Practice ladder beginner → industry

✓ Checkpoint — you can move on when you can…

Knowledge check check yourself

More in Testing & quality