Validation, Structured Outputs & Resiliency

Why the boundary matters motivation

Picture the demo that goes perfectly, then the launch that doesn't. Your agent parses the model's JSON, calls the deploy tool, and everyone claps. Two weeks later — under real traffic, with a real model that occasionally returns "replicas": "three" instead of 3, and a cluster API that throws a 503 every few minutes — the same agent scales a service to a nonsense number and then retries a half-finished deploy twice. Nothing in the code changed. The world got messier. Everything in this part exists to make the second story impossible.

Here's the mental model worth carrying through the whole page: your agent sits behind a boundary, like the airlock on a spacecraft. Two very different dangers press on it from opposite sides. From the inside-out side, the model and the tools produce data you can't trust — a hallucinated field, a malformed argument, a string where you expected a number. From the outside-in side, the network and downstream services fail in ways you can't control — rate limits, timeouts, servers that briefly fall over. A robust boundary has to be strict on the way in (reject bad data before it triggers a real action) and resilient on the way out (survive transient failures instead of turning them into outages).

The common mistake is treating these as the same problem, or skipping one entirely. Beginners tend to trust model output because "it looked right in testing" — then a single bad field crashes a batch or, worse, executes silently as a wrong action. Or they wrap everything in a blind try/except that retries, which quietly hammers a dead server and re-runs state-changing operations. The two disciplines are genuinely different tools for genuinely different failures, and you need both.

So the precise framing: validation keeps bad data out — it's the parser and type-checker at the door that turns untrusted JSON into typed, rule-checked objects (Pydantic, sections 1–4). Resiliency keeps transient failures from becoming outages — the error taxonomy, retries, circuit breakers, timeouts, idempotency, and fallbacks that let the system bend without breaking (sections 5–8). An agent takes untrusted input (users, tools, model output) and drives real actions; if a malformed tool argument or a hallucinated field slips through you get wrong actions or crashes, and if a transient 429/503 isn't retried a whole batch fails. Put the two together and you get the thing every production system is really selling: trustworthiness — the confidence that the agent does the right thing, exactly as many times as intended, even when its inputs and its dependencies misbehave.

1 · Pydantic foundations intermediate → advanced

Think about the last time you filled in a paper form that had rules printed on it. "Date of birth (DD/MM/YYYY)." "Amount, whole numbers only." "Tick one." The form isn't just collecting data — it's enforcing a shape, and it rejects nonsense at the point of entry so nobody downstream has to guess what you meant. A Pydantic model is exactly that form, written once in code, and applied automatically to every piece of data that tries to come through your door.

Why this matters so much for LLM work specifically: a model hands you back text that looks like JSON, and your instinct is to json.loads() it and start reading fields. That's the trap. Raw JSON gives you a dictionary with no promises — data["confidence"] might be 0.9, might be "0.9", might be 2.0, might be missing entirely, and you won't find out until three functions later when something divides by it and blows up far from the real cause. Pydantic moves that discovery to the edge: you declare the shape you require, and every value is checked, converted, and rule-verified the instant it arrives. Bad data fails here, with a precise message, instead of leaking inward as a mystery bug.

The common beginner mistake is hand-writing this checking: a wall of if not isinstance(x, str): ... and if conf < 0 or conf > 1: ... scattered through the code. It's verbose, it drifts out of sync with reality, and it usually reports only the first problem it hits. Pydantic replaces all of that with a single declarative class: you say what you want (a string 1–500 chars, a float in 0–1, one of four allowed words) and it figures out how to check it — reporting every violation at once, which is what makes debugging an LLM's output bearable.

Now the precise version. Pydantic parses and validates data against typed models — the de-facto standard for LLM structured output, config, and API boundaries. You subclass BaseModel, annotate each field with its type and optional constraints (via Field(...)), and Pydantic gives you three things for free on every instantiation: validation (rules are enforced), coercion (a "3" becomes 3 where sensible), and clear errors (a structured ValidationError listing each bad field). Define the shape once; get all three everywhere you use it.

pythonfrom pydantic import BaseModel, Field
from typing import Literal

class Diagnosis(BaseModel):
    summary: str = Field(min_length=1, max_length=500)
    severity: Literal["low", "medium", "high", "critical"]   # enum-constrained
    confidence: float = Field(ge=0.0, le=1.0)                    # 0..1
    affected: list[str] = []                                     # default empty

d = Diagnosis(summary="pod crash-looping", severity="high", confidence=0.9)
print(d.model_dump())            # dict, ready to serialize
# Diagnosis(summary="", severity="bad", confidence=2)  -> ValidationError with
# precise messages for EACH bad field at once

2 · Validators & coercion advanced

Types get you surprisingly far, and then they hit a wall. A type can say "this is an integer" but not "this integer must not exceed the max_replicas field two lines up." It can say "this is a string" but not "trim the whitespace and lowercase it first." Real-world rules are relational and messy in ways a type annotation simply can't express — and that gap is exactly where a validator lives.

The analogy is the difference between a bouncer checking IDs and a bouncer who also knows the house rules. The type check is the ID scan: is this a valid driver's license, yes or no. But the club also has rules the license can't encode — "no more guests than the fire code allows," "members-only after 10pm." Those need someone who can look at the whole situation, not just one card. A field validator is the ID scanner with a little extra polish (it can clean and reshape a single value); a model validator is the manager who sees every field at once and enforces the rules that only make sense in combination.

Here's a subtlety that trips people up, and it's worth slowing down for: Pydantic coerces by default. Hand it "3" where it expects an int and it will happily give you 3 — which is usually a gift (LLMs and web forms love to send numbers as strings) but occasionally a landmine. If a field arriving as a string instead of a number is itself a sign that something upstream is broken, silent coercion hides the bug. That's what strict mode is for: it turns off the helpful conversions so "3" is rejected as "not an int." The mistake to avoid is assuming coercion is always on or always off — it's a deliberate choice you make per model or per field, trading convenience against precision.

The mechanics, precisely: beyond types you often need custom rules and cross-field checks. A @field_validator("x") method receives one field's value, can clean or reject it, and whatever it returns replaces the stored value (raising ValueError is how you reject). A @model_validator(mode="after") runs once every field is populated, so it can compare fields against each other — the cross-field rules a single-field check can't reach. And coercion is the default behavior you opt out of with strict mode when you'd rather catch a type mismatch than paper over it.

pythonfrom pydantic import BaseModel, field_validator, model_validator

class ScaleAction(BaseModel):
    service: str
    replicas: int
    max_replicas: int = 10

    @field_validator("service")
    @classmethod
    def normalize(cls, v):
        return v.strip().lower()            # clean the value

    @field_validator("replicas")
    @classmethod
    def non_negative(cls, v):
        if v < 0:
            raise ValueError("replicas cannot be negative")
        return v

    @model_validator(mode="after")
    def within_limit(self):
        if self.replicas > self.max_replicas:   # cross-field rule
            raise ValueError("replicas exceeds max_replicas")
        return self

print(ScaleAction(service="  Web ", replicas=3).service)   # 'web'

3 · Discriminated unions — parse "one of several shapes" expert intermediate

Every tool-calling agent hits this problem on day one. The model says "I want to do something," and that something could be scale a service, or restart it, or roll it back — three actions with three different sets of required fields. A rollback needs a to_version; a restart doesn't. You get a raw dict and have to answer: which kind of action is this, and does it have the right fields for that kind? Get it wrong and you either crash or, worse, execute a malformed action against real infrastructure.

The naive fix is a staircase of ifs: "if action is 'rollback', check for to_version; elif it's 'scale', check for replicas…" — hand-written dispatch that's tedious, easy to get subtly wrong, and silently accepts a rollback that forgot its version. The parcel-sorting analogy makes the better idea obvious: a sorting facility doesn't open every package to guess where it goes: it reads the label. One field on the outside tells the machine exactly which bin, and the contents are checked against that bin's rules. A discriminated union is that label-driven sorter for your data.

Here's how it works: you designate one field as the discriminator (the tag — action here), and each candidate model pins that field to a single literal value (Literal["rollback"]). Pydantic reads the tag first, jumps straight to the matching model, and validates against only that shape — requiring exactly its fields. This is dramatically better than an untagged union, where Pydantic would try each shape in turn and hand you confusing errors from every failed attempt; the tag turns an ambiguous "figure out what this is" into a single, unambiguous, typed dispatch. The common mistake is forgetting the discriminator and using a bare Union: it often still works, but the error messages become a wall of noise and validation gets slower because every branch is attempted.

Precisely, then: a tool-calling agent receives actions that could be any of several types, each with different fields, and a discriminated union uses a tag field to tell Pydantic which model to validate against — turning that mess into a typed, exhaustive dispatch. The result is safety at the boundary: LLM JSON becomes a correctly-typed action object, or it's rejected with a message naming exactly what's missing, before it ever reaches your executor.

pythonfrom pydantic import BaseModel, Field, TypeAdapter
from typing import Literal, Union, Annotated

class Scale(BaseModel):
    action: Literal["scale"] = "scale"       # the discriminator
    service: str
    replicas: int

class Restart(BaseModel):
    action: Literal["restart"] = "restart"
    service: str

class Rollback(BaseModel):
    action: Literal["rollback"] = "rollback"
    service: str
    to_version: str

Action = Annotated[Union[Scale, Restart, Rollback], Field(discriminator="action")]
adapter = TypeAdapter(Action)

a = adapter.validate_python({"action": "rollback", "service": "api", "to_version": "v1.2"})
print(type(a).__name__, a.to_version)      # Rollback v1.2
# wrong shape for the tag -> ValidationError naming the missing field

4 · Structured LLM outputs — the reliable pattern advanced

The single most fragile line of code in most early agent projects is the one that reads the model's answer. Someone asks the model to "reply with the summary and a confidence score," gets back a paragraph of prose with the number buried in it, and writes a regex to fish it out. It works in the demo. Then the model phrases it differently next Tuesday, the regex misses, and the whole pipeline silently ingests garbage. Parsing prose is building on sand — the model's job is to be fluent and varied, which is exactly the opposite of what a parser needs.

The fix reframes the whole interaction. Instead of asking a question and hoping to extract structure from the reply, you hand the model the shape you require up front — your Pydantic schema — and constrain it to fill that shape in. Think of it as the difference between asking a witness to "tell me what happened" versus handing them a form with labelled boxes. The free-text answer is richer but unparseable; the form is boring and machine-readable. For data you're going to act on programmatically, boring and machine-readable wins every time.

But — and this is the honest part beginners skip — even schema-constrained output isn't guaranteed. Models still occasionally emit JSON that's malformed or violates a constraint (a confidence of 1.4, a missing required field). So the reliable pattern has two moves, not one: bind and validate, then retry-with-feedback. You validate the response against your schema; if it fails, you don't just throw it away and try the identical prompt again — you feed the validation error itself back to the model ("your last output failed: confidence must be ≤ 1.0; return valid JSON"). Models are strikingly good at fixing their own mistakes when shown the specific complaint, and the second attempt usually succeeds. The mistake to avoid is retrying blindly without the error message — that just rolls the same dice again.

Precisely: don't parse prose; ask the model for structured output bound to your schema and validate the result, retrying with the validation error fed back if it doesn't conform. This is the single biggest reliability win for agents — it converts the model from an unpredictable text generator into a component that returns typed, validated objects or fails loudly. As the note after the lab stresses, prefer your SDK's native structured-output path (which does the bind-validate-retry loop for you) and hand-roll it only when you must; either way, the iron rule is that you never trust unvalidated model text as data.

pythonfrom pydantic import BaseModel, ValidationError

# Preferred: the SDK parses INTO your model for you —
#   Claude: resp = client.messages.parse(model="claude-opus-4-8",
#             messages=[...], output_format=Diagnosis); resp.parsed_output
#   OpenAI: resp = client.responses.parse(model="gpt-5.5",
#             input="...", text_format=Diagnosis); resp.output_parsed
#   either way you get a validated Diagnosis back.
#
# When you must validate raw text yourself, retry feeding back the error:
def parse_with_retry(call_model, schema, prompt, tries=3):
    err = None
    for _ in range(tries):
        raw = call_model(prompt if err is None
                         else f"{prompt}\n\nYour last output failed validation: {err}\nReturn valid JSON.")
        try:
            return schema.model_validate_json(raw)    # success
        except ValidationError as e:
            err = str(e)                              # feed the error back to the model
    raise ValueError(f"model never produced valid output: {err}")

5 · An error taxonomy — retryable vs fatal advanced

Here's the failure that costs teams real money, and it comes from a good instinct applied without judgment. "Retries make things reliable," someone reasons, so they wrap every call in a retry loop. Then a request goes out with a typo'd API key — a 401 that will never succeed — and the loop dutifully retries it five times with backoff, burning seconds and, if it's a paid API, money, before failing anyway. The retry didn't help; it made a fast, clear failure into a slow, expensive one. Resiliency without classification is just wasted effort dressed up as robustness.

The idea that fixes this is triage, in the exact medical sense. An emergency room doesn't treat everyone identically — it sorts first, because the right response depends entirely on what kind of problem walked in. A cut that needs stitches and a heart attack get completely different handling, and giving one the other's treatment is worse than doing nothing. Errors are the same: before you decide how to respond, you have to decide what kind of error this is. That sorting step — the taxonomy — is the foundation every later resiliency pattern in this part is built on.

The essential split is transient versus permanent. A transient error is a passing condition of the world — a 429 rate limit, a 503 service-unavailable, a 529 overload, a timeout, a dropped connection. The request was fine; the server was just briefly unable, and trying again later has a real chance of working. A permanent (client) error is a problem with the request itself — a 400 bad request, a 401 auth failure, a 404, a validation error. No amount of retrying fixes a malformed request; the only cure is changing what you send. The table below adds two more buckets worth naming explicitly — server overload (retry, but back off harder or fall back to another model) and safety refusals (not an error to retry at all, but a decision to handle).

Precisely, then: resiliency starts with classification — some errors are transient so retrying helps, others are permanent so retrying just wastes time and money, and you decide the response per error type. The practical trick shown in the lab is to translate the raw signal (an HTTP status code) into your own small set of exception types — Retryable and Fatal — right at the boundary. Once the error carries its classification in its type, the retry loop in the next section becomes trivially simple: retry one type, immediately re-raise the other. Get the taxonomy right and everything downstream falls into place; get it wrong and no amount of clever backoff will save you.

pythonclass Retryable(Exception): ...
class Fatal(Exception): ...

def classify(status):
    if status in (429, 500, 502, 503, 529):
        raise Retryable(f"transient {status}")
    if 400 <= status < 500:
        raise Fatal(f"client error {status} — do not retry")

6 · Retry with exponential backoff + jitter expert advanced

The most tempting retry code is also the most dangerous. A server returns 503, so you try again immediately. Still failing? Try again immediately. This "tight retry loop" feels responsive, but picture what happens at scale: a service hiccups, ten thousand clients all get the error at the same instant, and all ten thousand slam it again a millisecond later — and again, and again. Your retries become a denial-of-service attack on a server that was just about to recover. This has a name in operations: the thundering herd, and it's one of the classic ways a small blip becomes a full outage.

Two ideas, layered, defuse it. The first is exponential backoff: instead of retrying at a fixed pace, you wait longer each time — 1 second, then 2, then 4, then 8. The intuition is polite persistence: a brief stumble gets a quick second chance, but a server that keeps failing is clearly struggling, so you back off and give it room to breathe rather than crowding the door. The doubling matters because it grows the breathing room fast without you having to tune each step by hand.

The second idea is jitter, and it's the one people forget. Exponential backoff alone doesn't actually break the herd — if all ten thousand clients back off on the same schedule, they just retry in synchronized waves, hammering the server at 1s, then 2s, then 4s, in perfect lockstep. Jitter adds randomness to each client's wait ("sleep a random amount up to the current delay"), so the crowd smears out across time instead of arriving together. The analogy is leaving a packed stadium: if everyone rushes the exit at once, nobody moves; if arrivals are staggered, everyone gets out faster. A little randomness, counterintuitively, makes the whole system quicker to recover.

The third piece is the discipline from the last section: retry only errors classified as retryable. This loop re-raises Fatal immediately (never waste a backoff cycle on a request that can't succeed) and only backs-off-and-retries the Retryable ones. Precisely, then: retry only retryable errors, and space attempts out exponentially (1s, 2s, 4s…) with jitter so many clients don't retry in lockstep and re-crush a recovering server. One real-world refinement flagged in the lab and the note: when a server sends a Retry-After header it's telling you exactly how long to wait — honor it over your computed delay. And know that mature SDKs (both Anthropic's and OpenAI's) already do this for their own calls; you build it yourself for your tools — the database, the internal API, the webhook — that don't come with it.

class _Any:
    '''stands in for any undefined demo value; supports call/attr/index/
    iteration and basic arithmetic (as 0.7) so demo snippets run.'''
    def __call__(self, *a, **k): return _Any()
    def __getattr__(self, k): return _Any()
    def __getitem__(self, k): return _Any()
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __contains__(self, o): return True
    def __enter__(self, *a): return _Any()
    def __exit__(self, *a): return False
    def __float__(self): return 0.7
    def __int__(self): return 1
    def __lt__(self, o): return True
    def __gt__(self, o): return False
    def __le__(self, o): return True
    def __ge__(self, o): return False
    def __add__(self, o): return o
    def __radd__(self, o): return o
    def __bool__(self): return True
    def __repr__(self): return 'demo'
    def __str__(self): return 'demo'
Fatal = _Any()
Retryable = _Any()

pythonimport random, time

def retry(fn, tries=5, base=1.0, cap=30.0):
    for attempt in range(tries):
        try:
            return fn()
        except Fatal:
            raise                               # never retry permanent errors
        except Retryable:
            if attempt == tries - 1:
                raise
            # exponential backoff with full jitter
            delay = min(cap, base * (2 ** attempt))
            time.sleep(random.uniform(0, delay))    # jitter avoids thundering herd

7 · Circuit breaker — stop hammering a dead dependency expert advanced

Backoff-and-jitter handles a service that stumbles. But what about one that's genuinely, lengthily down? Say a downstream API has been dead for ten minutes. Every request you send waits for its timeout, fails, waits out its backoff, and fails again — and meanwhile your threads are piling up in those waits, your latency is climbing, and the failure of one dependency is dragging your whole service down with it. Retrying was the right tool for a blip; it's the wrong tool for an outage. You need something that recognizes "this thing is dead, stop asking" and gets out of the way.

The pattern borrows its name and its logic straight from household electrical wiring, and the metaphor is worth taking literally. A circuit breaker in your fuse box doesn't keep pushing current into a short — it trips, cutting the connection to protect the house from a fire. You don't rewire anything; you just wait, then flip it back and see if the fault has cleared. A software circuit breaker does the same for a call to a flaky dependency: after enough failures it "trips" and stops the calls entirely, protecting both your system (no more piled-up waits) and the struggling dependency (no more pile-on load while it tries to recover).

The mechanism is a little three-state machine, and it's worth naming the states because they're the whole idea. Closed is normal: calls flow through, and a running count tracks consecutive failures. Hit the failure threshold and the breaker flips to open: for a cooldown period, every call fails instantly ("circuit OPEN — failing fast") without even touching the dead service. When the cooldown expires it goes half-open: it lets exactly one trial call through as a probe. If that probe succeeds, the service is back — reset to closed. If it fails, trip open again for another cooldown. A single success resets the failure count, so a healthy service is never penalized for old, unrelated failures.

Notice how this composes with the previous section rather than replacing it, because this is the distinction people miss: retry works within a single call (try this operation a few times), while the breaker works across calls (has this dependency been failing so much that we should stop trying at all?). They're different scopes of the same goal, and production systems use both together — retries to ride out brief turbulence, a breaker to bail out of a sustained outage. Precisely: if a downstream service is down, retrying every call just piles on load and slows your system; a circuit breaker opens after N consecutive failures — failing fast for a cooldown — then half-opens to test recovery before resuming.

pythonclass CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30.0, now=None):
        self.threshold, self.cooldown = threshold, cooldown
        self.failures = 0
        self.opened_at = None
        self._now = now or (lambda: __import__("time").monotonic())

    def call(self, fn):
        if self.opened_at is not None:
            if self._now() - self.opened_at < self.cooldown:
                raise RuntimeError("circuit OPEN — failing fast")
            self.opened_at = None            # half-open: allow one trial
        try:
            result = fn()
        except Exception:
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = self._now()   # trip the breaker
            raise
        self.failures = 0                    # success resets
        return result

8 · Idempotency & graceful fallbacks advanced

Everything we've built so far — retries, breakers — has a dark side we've been quietly ignoring. Retrying is safe when the operation just reads something. But what if the operation changes the world? You send a "charge the customer $99" request, the network hiccups before the response comes back, so your retry logic dutifully sends it again — and now the customer is charged twice. The request may have succeeded the first time; you just never heard the "yes." Retrying state-changing actions is how good resiliency code causes double-charges, double-deploys, and duplicate emails. This is the problem you must solve before you're allowed to retry anything that mutates state.

The concept that saves you has an intimidating name — idempotency — but a simple meaning: an operation is idempotent if doing it twice has the same effect as doing it once. Pressing a "close elevator door" button is idempotent (the door is closed either way); pressing "vend one soda" is not (you get two sodas). Most reads are naturally idempotent; most writes are not — and the trick is to make the dangerous writes idempotent by attaching a unique idempotency key to each intended action. The server (or your own dedup layer) records "I've already processed key deploy-42" and, on seeing it again, returns the stored result instead of running the operation a second time. The retry becomes safe not because it stops happening, but because the second execution is quietly turned into a no-op.

The partner idea is the graceful fallback, which answers a different question: not "how do I retry safely" but "what do I do when everything I tried has genuinely failed?" The mistake here is letting a single failed call crash the whole request — a dead end for the user. A fallback is the Plan B you reach for instead: if the frontier model is overloaded or refuses, fall back to a cheaper model, a cached answer, or an "escalate to a human" path. The system degrades — the answer might be less good — but it stays useful, which is almost always better than a stack trace. Think of it as the restaurant that's out of the special offering you a solid second choice rather than sending you home hungry.

Precisely: when you retry an action that changes state you risk doing it twice, and an idempotency key makes repeats safe by having the server or dedup layer recognize the key and apply the operation once; when a call ultimately fails, a fallback keeps the system useful. As the closing note ties together, idempotency is a correctness guardrail that complements the safety gate from the capstone — the gate decides whether a state-changing action may run, and the idempotency key guarantees how many times it actually does. Together with everything above, that's the full boundary: strict on the way in, resilient on the way out, and exactly-once where it counts.

python_done = {}                                # idempotency store (dict; a DB/Redis in prod)

def execute_once(key, action):
    if key in _done:                       # already ran -> return cached result
        return _done[key]
    result = action()                     # the real, state-changing call
    _done[key] = result
    return result

def with_fallback(primary, fallback):
    try:
        return primary()
    except Exception:
        return fallback()                # e.g. cheaper model, cached answer, "escalate"

# Agent example: try the frontier model; on refusal/overload, fall back.
# answer = with_fallback(lambda: ask_opus(q), lambda: cached_or_escalate(q))

Exercises expert

🎯 Interview practice interview

The interview questions this topic gets asked — worked, with code. For the full pattern catalog see A9 · Big Tech AI-engineering patterns.

pythonfrom pydantic import BaseModel, ValidationError
class Answer(BaseModel):
    summary: str
    confidence: float
def parse_with_retry(call, prompt, tries=3):
    err = None
    for _ in range(tries):
        raw = call(prompt if not err else f"{prompt}\nFix: {err}")
        try: return Answer.model_validate_json(raw)
        except ValidationError as e: err = str(e)
    raise ValueError("no valid output")

pythonimport random, time
def retry(fn, tries=5, base=1.0):
    for attempt in range(tries):
        try: return fn()
        except Exception:
            if attempt == tries - 1: raise
            time.sleep(random.uniform(0, base * 2 ** attempt))

Checkpoint expert

def parse_user(d):
    name = d.get("name")
    age = d.get("age")
    if not isinstance(name, str) or not name:
        raise ValueError("name must be a non-empty string")
    if not isinstance(age, int) or isinstance(age, bool) or age < 0:
        raise ValueError("age must be a non-negative int")
    return {"name": name, "age": age}

print(parse_user({"name": "Ada", "age": 36}))
try:
    parse_user({"name": "", "age": -1})
except ValueError as e:
    print("rejected:", e)

def coerce_age(v):
    if isinstance(v, bool):
        raise ValueError("bool is not a valid age")
    if isinstance(v, int):
        return v
    if isinstance(v, str) and v.strip().lstrip("-").isdigit():
        return int(v)
    raise ValueError(f"cannot coerce {v!r} to int age")

print(coerce_age(30))       # 30
print(coerce_age("42"))     # 42
try:
    coerce_age("old")
except ValueError as e:
    print("rejected:", e)

import math

def parse_shape(d):
    t = d.get("type")
    if t == "circle":
        r = float(d["radius"]); return ("circle", math.pi * r * r)
    if t == "rect":
        return ("rect", float(d["w"]) * float(d["h"]))
    raise ValueError(f"unknown shape type: {t!r}")

print(parse_shape({"type": "circle", "radius": 2}))     # ('circle', 12.566...)
print(parse_shape({"type": "rect", "w": 3, "h": 4}))     # ('rect', 12.0)
try:
    parse_shape({"type": "blob"})
except ValueError as e:
    print("rejected:", e)

class Retryable(Exception): pass      # e.g. timeout, 429, 503
class Fatal(Exception): pass          # e.g. 400, auth, bad schema

def should_retry(exc):
    return isinstance(exc, Retryable)

def run(fn, retries=3):
    for attempt in range(retries):
        try:
            return fn()
        except Exception as e:
            if not should_retry(e) or attempt == retries - 1:
                raise
    # unreachable

state = {"n": 0}
def flaky():
    state["n"] += 1
    if state["n"] < 2:
        raise Retryable("503")
    return "ok"

print(run(flaky))                     # ok (retried once)
try:
    run(lambda: (_ for _ in ()).throw(Fatal("400 bad request")))
except Fatal as e:
    print("not retried:", e)

import random

def backoff_delays(retries=5, base=0.1, cap=5.0, seed=0):
    rng = random.Random(seed)
    delays = []
    for attempt in range(retries):
        exp = min(cap, base * (2 ** attempt))   # exponential, capped
        delays.append(round(rng.uniform(0, exp), 3))  # full jitter
    return delays

print(backoff_delays())
# growing ceiling (0.1,0.2,0.4,0.8,1.6...) each randomized in [0, ceiling]
# e.g. [0.084, 0.15, 0.005, 0.63, 1.14]  -- no two clients align

import time

class CircuitBreaker:
    def __init__(self, threshold=3, cooldown=0.5):
        self.threshold, self.cooldown = threshold, cooldown
        self.fails = 0; self.opened_at = None; self.state = "closed"
    def _can_try(self):
        if self.state == "open":
            if time.monotonic() - self.opened_at >= self.cooldown:
                self.state = "half-open"; return True
            return False
        return True
    def call(self, fn, fallback):
        if not self._can_try():
            return fallback()                      # fail fast -> fallback
        try:
            res = fn()
        except Exception:
            self.fails += 1
            if self.fails >= self.threshold:
                self.state = "open"; self.opened_at = time.monotonic()
            return fallback()
        self.fails = 0; self.state = "closed"        # recovered
        return res

cache = {"last": "stale-but-ok"}
def dead():
    raise RuntimeError("dependency down")
def fallback():
    return cache["last"]                             # idempotent, safe

cb = CircuitBreaker(threshold=2, cooldown=0.2)
print(cb.call(dead, fallback), cb.state)   # stale-but-ok closed
print(cb.call(dead, fallback), cb.state)   # stale-but-ok open (fails hit threshold)
print(cb.call(dead, fallback), cb.state)   # stale-but-ok open (fast fail)
time.sleep(0.25)
print(cb.call(lambda: "live!", fallback), cb.state)  # live! closed (half-open probe ok)

Knowledge check check yourself