Software Testing · Chapter TQ1

Why test, and your first pytest

Why testing lets you change code fearlessly — from a first assertion to pytest to a risk-based test strategy a team follows. (Software testing, distinct from LLM evals.)

⏱️ ~2.5 hours🧪 6 labs🎯 Beginner→Tech-lead

🌱 Start here — from zero Testing, from scratch — a test is just code that checks your code — the habit that lets you change things fearlessly.

A test runs your code and asserts the result. This chapter climbs from your first assertion to running pytest, and up to designing a test strategy a team follows. So every block runs anywhere, the runnable examples use plain assert (pytest uses the same assertions).

The words you'll hear (in plain terms):

Term	What it actually means
test	code that runs your code and asserts it's correct.
assertion	a check that must hold (`assert x==5`).
pytest	the standard runner that finds and runs tests.
regression	a bug that returns; a test stops it recurring.
test pyramid	many unit, some integration, few end-to-end tests.

What you need before starting:

Python functions (py3).
pip install pytest for the pytest labs.
Runnable blocks use plain assert so they work with zero setup.

New to the topic? Read this box, then take the chapters in order — each section is tagged essential → expert so you always know the depth you're at.

Learning objectives

Explain what tests buy you and the test pyramid.
Write assertions and your first pytest test.
Read a failure and interpret expected-vs-got.
Shape a test strategy for a codebase.

▶ Runnable companionThe code here is also saved under code/tq1-why-test/. Python blocks run offline; config files are ready to drop into a project.

1 · Why test — and a first assertion essential

Without tests every change is a gamble. With them, one command tells you if you broke anything. At heart a test is an assert.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.

Python · the essence of a test (runs)

essence.pydef add(a, b):
    return a + b

# a "test" is just: run the code, assert the result
assert add(2, 3) == 5
assert add(-1, -1) == -2
print("all assertions passed")

all assertions passed

▶ How this works

This is the whole idea of testing in six lines. A test is nothing magic: you run a bit of your code, then assert that the result is what you expected. If it is, nothing happens and the program keeps going. If it isn't, Python stops with an error — that's the test failing, and it's how a test tells you something broke.

The first two lines define a tiny function add that returns a + b. This is the code under test — the thing we want to be sure works.
assert add(2, 3) == 5 means "I claim this equals 5." Python computes add(2, 3), gets 5, checks 5 == 5, and since that's true it stays silent and moves on.
The second assert checks a different case (adding two negatives). Good tests check more than one example so you catch mistakes that only show up sometimes.
The final print only runs if every assertion above passed — so seeing the message is your proof that all the checks held.

What the output means: You see all assertions passed. That line printing at all means both asserts were true; if one had been false, Python would have raised an AssertionError and the print would never run.

Try this: Change the first assertion to == 6 and run it again. Now the program stops with an AssertionError and never prints the message — that is exactly what a failing test looks like.

2 · Your first pytest test essential

pytest finds functions named test_* and runs them. Plain assert — pytest rewrites it to show values on failure.

pytest · calc.py + test_calc.py

test_calc.py# calc.py
def add(a, b): return a + b

# test_calc.py
from calc import add
def test_add():            assert add(2, 3) == 5
def test_add_negatives():  assert add(-1, -1) == -2

# run:  pytest -q   ->  2 passed

▶ How this works

Here the same assertions move into a real pytest test file. pytest is the standard tool that finds your tests, runs them, and reports pass/fail — so you don't call each check by hand. The one rule you must follow: names have to start with test_, because that's how pytest discovers which functions are tests.

The top two lines are the code being tested (add), which normally lives in its own file calc.py. The from calc import add line pulls that function into the test file so the tests can use it.
Each function whose name starts with test_ is one test. Inside, a single assert states what should be true. You can have as many test_ functions as you like — here there are two.
You do not call these functions yourself. pytest scans the file, sees the test_ names, and runs each one for you — that automatic finding is called test discovery.
The last comment shows how you run it: type pytest -q in your terminal (-q just means "quiet", less noise). pytest replies with a summary line.

What the output means: 2 passed means both test_ functions ran and every assertion inside them was true. This all-passing state is what people call green; a failing run is red.

Try this: Run pip install pytest, save these two functions in a file named test_calc.py, and run pytest -q. Then break one assertion and watch the summary flip to 1 failed, 1 passed.

3 · Reading a failure intermediate

A failing test shows expected vs got — learning to read it is half of testing.

Python · simulate what pytest reports

failure.pydef add(a, b): return a + b

def check(expr_desc, got, want):
    status = "PASS" if got == want else "FAIL"
    print(f"{status}: {expr_desc}  got={got!r} want={want!r}")

check("add(2, 2)", add(2, 2), 5)     # FAIL — shows 4 vs 5, like pytest does
check("add(2, 3)", add(2, 3), 5)

FAIL: add(2, 2)  got=4 want=5
PASS: add(2, 3)  got=5 want=5

▶ How this works

Passing tests are boring; the skill that matters is reading a failure. This little program imitates what pytest prints when a test fails, so you learn to read the two numbers that matter: what your code actually produced (got) versus what you expected (want).

check(...) is a helper we wrote to mimic pytest: it takes a description, the value your code got, and the value you want, then compares them.
The line that decides pass or fail is the comparison got == want: if they're equal it labels the line PASS, otherwise FAIL.
It then prints a line showing both values. The !r inside the f-string just prints them in a precise, quote-showing form — handy so 4 and "4" don't look identical.
The first call, add(2, 2), is meant to fail: 2+2 is 4, but we told it we want 5. The second call is a genuine pass. Running both lets you compare the two output lines side by side.

What the output means: FAIL: add(2, 2) got=4 want=5 is the failure line — it says your code returned 4 but you expected 5. Real pytest shows this same got-vs-want information; reading it tells you whether the bug is in your code or in your expectation.

Try this: Fix the expectation: change the 5 in the first check(...) call to 4 and re-run. Both lines now say PASS — you just "fixed" a failing test by correcting a wrong expectation.

4 · The test pyramid intermediate

Level	Tests	Speed	How many
Unit	one function	ms	many
Integration	parts together	seconds	some
End-to-end	whole system	slow	few

5 · Professional — organize the suite professional

Convention: a tests/ folder mirroring src/, files test_*.py, config in pyproject.toml. pytest discovers it all.

config · pyproject test config

pyproject.toml[tool.pytest.ini_options]
testpaths = ["tests"]
addopts = "-q --strict-markers"
markers = ["slow: long-running tests (deselect with -m 'not slow')"]

▶ How this works

Once you have more than a handful of tests, you tell pytest where they live and how to behave, once, in a config file. pyproject.toml is the standard place; pytest reads this section automatically every time you run it, so nobody has to remember flags.

The header [tool.pytest.ini_options] is the section name pytest looks for. Everything under it configures pytest.
testpaths = ["tests"] tells pytest to only look inside a folder called tests — the usual convention is a tests/ folder that mirrors your source code.
addopts lists flags applied to every run, so pytest alone behaves like pytest -q --strict-markers. --strict-markers makes pytest error on any label you didn't declare — a guard against typos.
markers declares custom labels (here slow) you can attach to tests so you can, for example, skip slow ones during quick runs.

Try this: Drop this into a real project's pyproject.toml, put your test files in a tests/ folder, and just run pytest — no flags needed, because the config supplies them.

6 · Tech-lead — a test strategy tech-lead

A lead defines what to test and to what depth so effort goes where risk is. A simple, codified policy beats vague "write more tests." Here's a risk model you can actually compute.

Python · a risk-based coverage policy (runs)

strategy.py# Decide required coverage from a module's risk (blast radius x change rate).
def required_coverage(blast_radius, change_rate):   # each 1-5
    risk = blast_radius * change_rate
    if risk >= 16: return 95      # core, changes often -> near-total
    if risk >= 9:  return 85
    if risk >= 4:  return 70
    return 50                     # low-risk glue -> light

modules = {
    "payments":  (5, 4),   # huge blast radius, changes often
    "auth":      (5, 2),
    "reporting": (2, 3),
    "scripts":   (1, 1),
}
for name, (b, c) in modules.items():
    print(f"{name:10} -> require {required_coverage(b, c)}% coverage")

payments   -> require 95% coverage
auth       -> require 85% coverage
reporting  -> require 70% coverage
scripts    -> require 50% coverage

▶ How this works

This is a tech-lead's answer to "how much should we test each part?" Instead of testing everything equally, it scores each module by risk and turns that score into a required coverage target — so effort lands where a bug would hurt most. It's ordinary Python, no testing library needed.

required_coverage(blast_radius, change_rate) takes two 1–5 ratings: how much breaks if this module fails (blast radius), and how often it changes (change rate). It multiplies them into a single risk number.
The chain of if checks maps that risk to a coverage percent: very high risk demands 95, and it steps down to 50 for low-risk "glue" code. Higher risk, more testing required.
modules is a dictionary pairing each module name with its (blast_radius, change_rate) pair — the real judgement calls a lead makes.
The for loop walks every module, calls the function, and prints the required coverage. The {name:10} just pads the name to 10 characters so the output lines up in a neat column.

What the output means: Each line pairs a module with its required coverage — payments needs 95% (high blast radius, changes often) while scripts needs only 50%. The point: the policy is computed and consistent, not a vague "write more tests."

Try this: Change reporting to (4, 4) and re-run. Its risk jumps and the required coverage rises — you've just watched a testing policy respond to a change in risk.

Strategy > volume"100% coverage everywhere" wastes effort on low-risk code and still misses edge cases. A lead targets coverage by risk, requires tests on high-blast-radius modules, and lets glue code be light. That's how you get reliability without drowning in tests.

Exercise TQ1.1 — Test + triage

Context: Validation logic is the natural first thing to test, and the risk model tells you where to spend effort next — pairing a small test suite with a triage of what to test more.

Your task: Write is_valid_email(s) covered by four or more assertions, then score three modules of a project you know with the risk model and state the coverage tier you would require for each.

Requirements:

is_valid_email handles valid addresses, a missing @, a missing domain, and the empty string
At least four assertions, one per case above
Reuse the blast-radius / change-frequency risk model on three real modules
For each module, state the required coverage tier and one sentence of justification
Both the test and the triage run/read offline — no live project needed

💡 Hint: Pick modules with genuinely different risk profiles (a payments path vs. a logging helper) so the tiers you assign actually differ.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Your first assertionBeginner

Context: A test is nothing more than code that checks other code. Before any framework, the bare habit is to call a function and state, in code, what its answer must be.

Your task: Write add(a, b) and back it with a couple of plain assert statements that pin down its expected results — no pytest, no test runner, just Python.

Requirements:

Use the built-in assert statement, not a framework
Assert at least one positive case (e.g. add(2, 3) == 5)
Assert a case that crosses zero (e.g. add(-1, 1) == 0)
Print a confirmation line only after every assertion has passed
The whole thing runs offline as an ordinary script

💡 Hint: An assert that does not raise is a claim about your code that just held — that silent success is the entire point.

Show solution

The whole idea, in two lines:

def add(a, b):
    return a + b

assert add(2, 3) == 5
assert add(-1, 1) == 0
print("all assertions passed")

An assert that doesn't raise means the code did what you claimed. This is the habit that lets you change code fearlessly.

Exercise 2 · A unittest test case (runs without pytest)Intermediate

Context: Bare asserts do not scale — you want named cases, a runner, and a pass/fail report. The stdlib unittest ships with Python and runs anywhere, so it is the zero-dependency way to write a real test.

Your task: Wrap the add check in a unittest.TestCase with two separate test methods, and run it with the built-in runner.

Requirements:

Subclass unittest.TestCase
Give it two distinct test methods (e.g. positives and mixed signs)
Use self.assertEqual rather than a bare assert
Invoke unittest.main(verbosity=2) under a __main__ guard so each test name prints
The suite runs offline with no third-party packages

💡 Hint: The pytest version of this is just plain functions with assert; the assertions themselves are identical, so what you learn here transfers.

Show solution

unittest ships with Python — a first test that runs offline:

import unittest

def add(a, b): return a + b

class TestAdd(unittest.TestCase):
    def test_positives(self):
        self.assertEqual(add(2, 3), 5)
    def test_mixed_signs(self):
        self.assertEqual(add(-1, 1), 0)

if __name__ == "__main__":
    unittest.main(verbosity=2)
# test_mixed_signs (...) ok
# test_positives (...) ok
# Ran 2 tests ... OK

The pytest equivalent is just plain functions with assert; the assertions are identical.

Exercise 3 · Read a failure and fix itAdvanced

Context: A failing test is not a nuisance — it is a precise bug report you wrote in advance. Watching a test go red and then flip to green is the core loop of all testing.

Your task: Write a deliberately buggy median, let an assertion catch it, then fix the function so the same assertion passes — demonstrating the red→green flip.

Requirements:

The first median mishandles the even-length case (returns the wrong element)
Catch the AssertionError and print the actual value so the failure localizes the bug
The failing assertion targets an even-length list (e.g. median([1,2,3,4]) == 2.5)
The fixed version averages the two middle elements for even-length input
Show the odd-length case still returns the single middle element
Print a RED line before the fix and a GREEN line after

💡 Hint: The even-length branch is exactly the edge the test pins down — average the two central values only when the length is even.

Show solution

The failure message localizes the defect:

def median(xs):
    xs = sorted(xs)
    n = len(xs)
    # BUG: integer division picks the wrong element for even-length lists
    return xs[n // 2]

try:
    assert median([1, 2, 3, 4]) == 2.5, f"got {median([1,2,3,4])}"
except AssertionError as e:
    print("RED:", e)     # RED: got 3

def median_fixed(xs):
    xs = sorted(xs); n = len(xs); m = n // 2
    return xs[m] if n % 2 else (xs[m-1] + xs[m]) / 2

assert median_fixed([1, 2, 3, 4]) == 2.5
assert median_fixed([1, 2, 3]) == 2
print("GREEN: fixed")

The even-length case is exactly the edge the test pinned down — a failing test is a precise bug report you wrote in advance.

Exercise 4 · Classify tests into the pyramidExpert

Context: The test pyramid says a healthy suite has many fast unit tests, fewer integration tests, and fewest end-to-end tests. An inverted pyramid — mostly slow e2e — is the classic anti-pattern.

Your task: Given a suite of tests each tagged by type, write a check that asserts the suite is pyramid-shaped: unit ≥ integration ≥ e2e.

Requirements:

Count tests per type (a Counter over the tags is enough)
Define is_pyramid() returning whether unit ≥ integration ≥ e2e
Assert the healthy suite passes and print the per-type counts
Give the assertion a message naming the failure ("top-heavy: too many slow tests")
The check turns "are we pyramid-shaped?" into a runnable test

💡 Hint: You are comparing three tallies, not the tests themselves — reduce the tagged list to counts first, then compare them in order.

Show solution

Shape the suite so most tests are fast and cheap:

from collections import Counter

suite = ["unit", "unit", "unit", "unit", "integration", "integration", "e2e"]
c = Counter(suite)

def is_pyramid(c):
    return c["unit"] >= c["integration"] >= c["e2e"]

print(dict(c))            # {'unit': 4, 'integration': 2, 'e2e': 1}
assert is_pyramid(c), "suite is top-heavy: too many slow tests"
print("pyramid OK")

An inverted pyramid (mostly slow e2e tests) is the classic anti-pattern — the assertion turns "are we pyramid-shaped?" into a check.

Exercise 5 · Organize the suite by responsibilityProfessional

Context: As a suite grows, discoverability keeps it maintainable: anyone should be able to find (or create) the test file for a given module without hunting. A predictable source→test mapping is what lets tooling enforce that every module has a test.

Your task: Document a project layout and naming convention that groups tests by the module they cover, then write a helper that maps a source path to its test path.

Requirements:

State the convention (e.g. src/orders.py → tests/test_orders.py)
Write test_path_for(src_path) that derives the test path from a source path
Strip the directory and the .py extension, then prefix the module with test_
Place the result under a tests/ directory
Assert the mapping on at least two example paths
Use os.path so the split works portably

💡 Hint: Split the path into directory and filename, drop the extension to get the module name, and reassemble as tests/test_<module>.py.

Show solution

Discoverable structure keeps a growing suite maintainable:

# Layout:
#   src/orders.py      -> tests/test_orders.py
#   src/pricing.py     -> tests/test_pricing.py
# Convention: test_<module>.py, one test class per public class.

import os

def test_path_for(src_path):
    d, fname = os.path.split(src_path)
    module = fname[:-3] if fname.endswith(".py") else fname
    return os.path.join("tests", f"test_{module}.py")

assert test_path_for("src/orders.py") == "tests/test_orders.py"
assert test_path_for("src/pricing.py") == "tests/test_pricing.py"
print("mapping OK")

A predictable source→test mapping means anyone can find (or create) the right test file, and tooling can enforce that every module has one.

Exercise 6 · A risk-based test strategyIndustry scenario

Context: Tech-lead reality: not everything deserves equal testing effort. A component's blast radius (how much breaks if it fails) and its change frequency together say how much rigor it warrants — and where you are dangerously under-tested.

Your task: Encode a risk-based strategy that assigns a required coverage tier from a component's blast radius and change frequency, then audit a component list to flag high-risk items lacking the tests they need.

Requirements:

required_tier(blast_radius, change_freq) maps the two low/med/high inputs to critical/standard/light
Higher combined blast-plus-churn yields a higher required tier
audit() returns the names of components whose required tier is critical but which lack critical-tier tests
A high-blast, high-churn component (e.g. payments) is flagged; a low/low one (e.g. logging) is not
The strategy is explicit and reviewable, not a gut call

💡 Hint: Rank each dimension 0/1/2 and add them, then threshold the sum into tiers — the audit is just a filter for critical components missing their tests.

Show solution

Spend testing effort where failure hurts most:

def required_tier(blast_radius, change_freq):
    # blast_radius, change_freq in {"low","med","high"}
    rank = {"low": 0, "med": 1, "high": 2}
    score = rank[blast_radius] + rank[change_freq]
    return "critical" if score >= 3 else "standard" if score >= 1 else "light"

COMPONENTS = [
    {"name": "payments", "blast": "high", "freq": "high", "has_critical_tests": False},
    {"name": "logging",  "blast": "low",  "freq": "low",  "has_critical_tests": False},
]
def audit(components):
    gaps = []
    for c in components:
        tier = required_tier(c["blast"], c["freq"])
        if tier == "critical" and not c["has_critical_tests"]:
            gaps.append(c["name"])
    return gaps

print(required_tier("high", "high"))   # critical
print("under-tested:", audit(COMPONENTS))   # ['payments']

Payments (high blast radius, high churn) is flagged as needing critical-tier tests; logging is fine with light coverage. The strategy is explicit and reviewable, not vibes.

✓ Checkpoint — you can move on when you can…

Explain what tests buy you and the pyramid.
Write assertions and a pytest test.
Read a failure's expected-vs-got.
Set risk-based testing depth for a codebase.

Knowledge check check yourself

✓ Knowledge check

This lesson's risk model computes required coverage from blast_radius × change_rate. Why does a lead target coverage by risk instead of mandating 100% everywhere?

Show answer

Because "100% coverage everywhere" wastes effort on low-risk glue code and still misses edge cases. Scoring each module by risk (how much breaks if it fails, times how often it changes) sends test effort where a bug would hurt most — e.g. payments needs ~95% while scripts needs only ~50%.

✓ Knowledge check

In the test pyramid the lesson describes, why should a suite have many unit tests but few end-to-end tests?

Show answer

Unit tests exercise one function, run in milliseconds, and pinpoint failures, so you can afford many of them. End-to-end tests exercise the whole system, are slow, and are harder to diagnose, so you keep only a few — the pyramid is wide at the fast/cheap unit base and narrow at the slow e2e top.

Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.

Learning objectives

1 · Why test — and a first assertion essential

2 · Your first pytest test essential

3 · Reading a failure intermediate

4 · The test pyramid intermediate

5 · Professional — organize the suite professional

6 · Tech-lead — a test strategy tech-lead

Exercise TQ1.1 — Test + triage

🪜 Practice ladder beginner → industry

✓ Checkpoint — you can move on when you can…

Knowledge check check yourself

More in Testing & quality