Integration tests, coverage & CI

1 · Integration tests essential

Here's a bug that has embarrassed more engineers than any other. Every unit test is green. You have 400 of them, all passing, all fast. You deploy on Friday. By Saturday, orders are silently saving with the total set to NULL, because your code writes a column called amount but the real database column is spelled total. Not one of your 400 unit tests caught it — because every one of them faked the database out. The pieces were individually correct. The seams between them were where the truth lived, and nobody tested a seam.

That is exactly the gap integration tests fill. Think of it like a car: unit tests check that the engine spins, the wheels turn, and the fuel pump pumps — each part on a bench, in isolation. An integration test bolts the parts together and actually drives around the block. Most catastrophic failures aren't a broken part; they're two good parts that disagree about the interface between them — a mismatched column name, a JSON field the other service never reads, a timezone one layer assumes and the other doesn't.

So the precise definition: an integration test exercises two or more real components together — your code plus the database, your service plus the queue, your API plus its client — and asserts they cooperate correctly. The word "real" is load-bearing. The moment you replace the database with a mock, you're back to a unit test, and you've stopped testing the very thing (the SQL, the schema, the driver's type coercion) that broke on Saturday.

The catch integration tests always run into is speed and setup. A real Postgres server means spinning up a container, seeding a schema, and tearing it down — slow enough that people stop running the suite, which defeats the purpose. The trick you'll use throughout this chapter sidesteps that: an in-memory SQLite database (sqlite3.connect(":memory:")). It is genuine SQL — real tables, real constraints, real aggregates — but it lives in RAM and disappears when the connection closes. You get authentic integration behavior at unit-test speed, with no external server to install and no cleanup to forget.

• Unit test: one component, dependencies faked — fast, precise, but blind to seam bugs.
• Integration test: multiple real components together — catches the mismatches, at the cost of more setup.
• The common mistake: mocking the very thing you were trying to integrate. If you mock the DB in an "integration" test, you've tested nothing new.

The lab below is the smallest possible real integration: an OrderStore class and an actual SQLite database, writing a row and reading it back. Watch that no mock appears anywhere — that absence is the whole point.

integration.pyimport sqlite3
class OrderStore:
    def __init__(self, conn):
        self.conn = conn
        conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, total REAL)")
    def create(self, total):
        c = self.conn.execute("INSERT INTO orders(total) VALUES (?)", (total,))
        self.conn.commit(); return c.lastrowid
    def get(self, oid):
        return self.conn.execute("SELECT total FROM orders WHERE id=?", (oid,)).fetchone()

store = OrderStore(sqlite3.connect(":memory:"))     # integration: store + DB
oid = store.create(99.0)
assert store.get(oid)[0] == 99.0
print("integration test passed (store + db together)")

integration test passed (store + db together)

2 · Coverage intermediate

You've written tests. But which lines of your code did they actually touch? Almost every engineer, at some point, discovers the answer the hard way: a bug ships in a function they were sure was tested — and when they look, the failing branch (the error path, the edge case, the else nobody wrote a test for) was never once executed by the suite. The tests were real; they just never walked down that particular hallway. Coverage is the flashlight that shows you which hallways are still dark.

Concretely: coverage is the percentage of your code that your test suite executes at least once. A coverage tool runs your tests while quietly ticking off every line (and, more usefully, every branch) as it runs. At the end it reports the fraction reached and — the part that actually matters — names the lines that were never reached. That list of untouched lines is a to-do list: "here is code you shipped that no test has ever exercised."

Now the trap, and it is the single most misunderstood idea in testing: coverage measures execution, not correctness. A line being covered means only that a test caused it to run — not that any test asserted the right thing about it. You can achieve 100% coverage with tests that assert nothing at all, or assert the wrong value. Coverage is a good gap-finder (it tells you what you definitely haven't tested) and a terrible quality certificate (it can't tell you whether what you tested is right). Think of it like checking that a security guard walked past every door — reassuring, but it doesn't prove any door was actually locked.

The healthy way to use it, and what real teams do: chase meaningful coverage of the code that matters — the core business logic, the money-touching paths, the error handling — aiming for something like 80–90% there, rather than a vanity 100% that pushes people to write pointless tests just to color a getter green. In practice you run pytest --cov (from the pytest-cov plugin) and read the term-missing report to see exactly which lines slipped through. The lab below fakes that whole mechanism in a few lines so you can watch it work: it records which branches ran, then subtracts that from every branch that could have run — and the difference is the gap the flashlight finds.

coverage.pyexecuted = set()
def tracked(fn_name): executed.add(fn_name)

def discount(price, pct):
    tracked("discount")
    if pct > 100:
        tracked("discount:guard"); raise ValueError
    return price*(1-pct/100)

discount(100, 10)                    # tests exercise the happy path only
all_branches = {"discount", "discount:guard"}
covered = executed & all_branches
print(f"coverage: {len(covered)}/{len(all_branches)} branches = {100*len(covered)//len(all_branches)}%")
print("missing:", all_branches - executed)   # the guard is untested!

coverage: 1/2 branches = 50%
missing: {'discount:guard'}

3 · Advanced — CI gate advanced

A test suite that nobody runs is worth exactly nothing. You know this because you've lived it: the team has great tests, but running them takes a manual command, so people skip it when they're in a hurry — which is always. Someone pushes a change on a deadline, forgets to run the suite, and the broken code lands on main. Monday morning, the whole team is blocked because the shared branch is red and nobody's sure who broke it. The tests existed. The discipline to run them didn't. Humans are unreliable about repetitive chores; that's not a character flaw, it's just biology.

Continuous Integration (CI) removes the human from that loop entirely. The idea: a fresh, clean machine in the cloud automatically checks out your code and runs the tests every single time anyone pushes or opens a pull request. No one can forget, because no one is asked to remember. Think of it as a tireless robot gatekeeper standing at the door to main — every change has to walk past it, and it runs the exact same checks the exact same way, on every commit, forever.

The tool you'll see here is GitHub Actions, and the thing that trips people up first is that its config is not code — it's a YAML file, a plain-text format where indentation defines structure. A line indented under another line "belongs to" it, the way bullet points nest. That's why YAML is famously whitespace-sensitive: a stray space can change which block a step lives in. The file lives at a fixed, magic path — .github/workflows/ — and GitHub watches that folder; drop a workflow file there and it activates automatically.

The mental model for the file has three layers, top to bottom: on (the trigger — when should this run, e.g. on push and pull_request), jobs (what work to do, each on a fresh virtual machine), and steps (the ordered commands inside a job — check out the code, install Python, install dependencies, run pytest). The part that turns this from "runs my tests" into a gate is quietly powerful and worth internalizing: every shell step must exit with status 0 (success) or the job fails. pytest exits non-zero the moment any test fails — so a red test automatically fails the step, which fails the job, which turns the pull request's check red and blocks the merge. You don't write any "if tests fail, block" logic; that behavior falls out of Unix exit codes for free. That is the whole trick behind a required CI gate.

tests.ymlname: tests
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with: { python-version: "3.12" }
      - run: pip install -r requirements.txt pytest pytest-cov
      - run: pytest --cov=src --cov-report=term-missing
      # a failing test fails the job -> the PR check goes red

4 · Professional — quality gates professional

Picture the argument that happens on every team without a written bar. A pull request comes in: tests pass, but coverage on the new code is thin and there are a couple of fresh lint warnings. One reviewer says ship it, we're behind. Another says no, that's below our standard. The truth is neither of them knows the standard, because it only exists as a vague feeling that shifts with mood, deadline pressure, and who's asking. The same PR gets waved through on Friday and blocked on Monday. That inconsistency is corrosive — it makes "quality" feel arbitrary and political instead of objective.

A quality gate ends that argument by writing the bar down as code. It's an automated pass/fail check that a pull request must clear before it can merge, combining several conditions with a logical AND: tests must be green, and coverage must sit at or above an agreed floor, and there must be no new critical lint errors. If every condition holds, the gate opens; if any fails, it stays shut and reports exactly which conditions blocked it. The analogy is airport security: it's not one guard's judgment call about whether you look trustworthy — it's a fixed checklist (boarding pass, ID, nothing flagged in the scanner) applied identically to everyone, every time.

Why bother encoding it rather than trusting good reviewers? Two reasons. First, consistency: a function that checks the numbers gives the same verdict at 2pm and 2am, for the senior and the intern, this week and next quarter — it can't be tired, rushed, or lobbied. Second, specificity: a good gate doesn't just say "no," it hands back the list of reasons ("coverage 71% below the 80% floor; 3 new lint errors"), so the author knows precisely what to fix instead of guessing what the reviewer disliked. That frees human reviewers to spend their attention on the things machines can't judge — is this the right design, is the code readable, does it solve the actual problem — while the mechanical bar enforces itself.

One nuance worth stating plainly, because it's a frequent mistake: the coverage condition should be a floor, not a demand for 100%. A floor (say 80% on core modules) blocks regressions — you can't quietly make things worse — without pushing people to write meaningless tests that chase the last few percent on trivial code. The lab below models exactly this gate as a small pure function: it accumulates a list of blocking reasons, and the PR is mergeable only when that list is empty. That same logic is what you wire into the CI YAML from §3 so the gate enforces itself on every push.

quality_gate.pydef quality_gate(results):
    failures = []
    if not results["tests_pass"]:             failures.append("tests failing")
    if results["coverage"] < results["min_coverage"]: failures.append(
        f"coverage {results['coverage']}% < {results['min_coverage']}%")
    if results["new_lint_errors"] > 0:        failures.append(
        f"{results['new_lint_errors']} new lint errors")
    return (not failures), failures

ok, why = quality_gate({"tests_pass": True, "coverage": 82, "min_coverage": 80, "new_lint_errors": 0})
print("PR mergeable:", ok)
ok2, why2 = quality_gate({"tests_pass": True, "coverage": 71, "min_coverage": 80, "new_lint_errors": 3})
print("PR mergeable:", ok2, "|", why2)

PR mergeable: True
PR mergeable: False | ['coverage 71% < 80%', '3 new lint errors']

5 · Tech-lead — build a testing culture tech-lead

Every engineer has watched a beautiful test suite rot. A team buys the best tools, wires up CI, hits 90% coverage — and eighteen months later the suite takes twenty minutes to run so people skip it, three tests fail "randomly" so everyone ignores red, and new features ship with no tests because "we'll add them later." Nothing was technically broken. The culture decayed, one small tolerated compromise at a time. This is the humbling lesson of leading quality: tools are necessary and completely insufficient. A gate you can bypass, a red build you learn to ignore, a slow suite you route around — the technology is fine; the discipline around it eroded.

So the tech-lead's real job isn't installing better tools — it's making the healthy path the easy path, so doing the right thing is also the path of least resistance. When the tested route is faster and smoother than the untested one, people take it without being nagged. That's the whole game, and it comes down to a handful of load-bearing habits, each defending against a specific failure mode:

• Required green CI to merge. Not "please run the tests" — can't merge until they pass. This is the difference between a suggestion and a gate; without it, everything below erodes.
• A fast suite (mock the slow dependencies). If the suite takes twenty minutes, people stop running it locally and quality drops. Speed isn't a luxury — it's what makes the tests get used at all.
• Fix or delete flaky tests immediately. The most dangerous test isn't one that fails — it's one that fails sometimes. A single flaky test teaches the whole team that "red" might not mean "broken," and once red loses its meaning, the entire safety net is gone.
• Tests expected with every feature and fix. Coverage should grow with the code, as a normal part of the work, not bolted on in a doomed "testing sprint" later.
• A coverage floor on core modules, not everywhere. Risk-weighted, not vanity — protect the code that matters rather than chasing a number across trivial files.

Notice that this section is where the whole stack of skills converges on one artifact: the pull request. Git collaboration (DF3) delivers the change, the testing chapters (TQ1–TQ4) fill it with meaningful tests, CI and deployment (CD) gate and ship it. A codebase where every PR clears a fast, trusted, meaningful suite is one a team can keep changing quickly and safely for years — and building and defending that culture, long after the tools are installed, is the defining responsibility of a technical lead. The table and exercises below turn that culture into something measurable: signals you can score, so "how healthy is our testing?" becomes a grade and a concrete next investment rather than a vague "we should test more."

import unittest, sqlite3

class OrderStore:
    def __init__(self, conn):
        self.conn = conn
        conn.execute("CREATE TABLE orders (id INTEGER PRIMARY KEY, amt REAL)")
    def add(self, amt): self.conn.execute("INSERT INTO orders (amt) VALUES (?)", (amt,))
    def total(self):
        return self.conn.execute("SELECT COALESCE(SUM(amt),0) FROM orders").fetchone()[0]

class TestOrderStore(unittest.TestCase):
    def setUp(self):
        self.store = OrderStore(sqlite3.connect(":memory:"))   # real DB, fresh
    def test_total(self):
        self.store.add(10); self.store.add(5)
        self.assertEqual(self.store.total(), 15)

if __name__ == "__main__":
    unittest.main(verbosity=2)     # exercises store + real SQL together

def coverage(total_lines, executed_lines):
    covered = len(set(executed_lines) & set(range(1, total_lines + 1)))
    pct = 100.0 * covered / total_lines
    missing = sorted(set(range(1, total_lines + 1)) - set(executed_lines))
    return round(pct, 1), missing

pct, missing = coverage(total_lines=10, executed_lines=[1,2,3,4,5,6,7,8])
print(f"{pct}% covered; missing lines: {missing}")
# 80.0% covered; missing lines: [9, 10]

# .github/workflows/ci.yml   (needs GitHub Actions)
name: ci
on: [push, pull_request]
jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -e . pytest
      - name: Test
        run: pytest -q          # non-zero exit fails the job = blocks merge

# .github/workflows/coverage.yml   (needs pytest + pytest-cov)
name: coverage
on: [push, pull_request]
jobs:
  cov:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-python@v5
        with:
          python-version: "3.12"
      - run: pip install -e . pytest pytest-cov
      - name: Test with coverage floor
        run: pytest --cov=src --cov-report=term-missing --cov-fail-under=80
      # --cov-fail-under=80 exits non-zero (fails the build) below 80% coverage.

def quality_gate(results, coverage_floor=80):
    reasons = []
    if not results["tests_pass"]:
        reasons.append("tests failing")
    if results["coverage"] < coverage_floor:
        reasons.append(f"coverage {results['coverage']}% < {coverage_floor}%")
    if results["critical_lint"] > 0:
        reasons.append(f"{results['critical_lint']} critical lint errors")
    return (not reasons), reasons

ok  = {"tests_pass": True, "coverage": 88, "critical_lint": 0}
bad = {"tests_pass": True, "coverage": 72, "critical_lint": 2}
print(quality_gate(ok))    # (True, [])
print(quality_gate(bad))   # (False, ['coverage 72% < 80%', '2 critical lint errors'])

def health(signals):
    score = 0
    notes = []
    if signals["flaky_rate"] <= 0.01: score += 1
    else: notes.append(("flaky tests", signals["flaky_rate"]))
    if signals["suite_seconds"] <= 300: score += 1
    else: notes.append(("slow suite", signals["suite_seconds"]))
    if signals["coverage_trend"] >= 0: score += 1
    else: notes.append(("coverage falling", signals["coverage_trend"]))
    if signals["pct_prs_with_tests"] >= 0.8: score += 1
    else: notes.append(("PRs lack tests", signals["pct_prs_with_tests"]))
    grade = ["at risk","weak","fair","good","strong"][score]
    top_risk = max(notes, key=lambda x: 1)[0] if notes else "none"
    return grade, top_risk

s = {"flaky_rate": 0.03, "suite_seconds": 120,
     "coverage_trend": +2, "pct_prs_with_tests": 0.9}
print(health(s))   # ('good', 'flaky tests')

Knowledge check check yourself