Case Studies & Reference Architectures · Part 3

Case study: compliance doc agent

Q: A proposed obligation comes back with confidence 0.97, but its cited quote does not appear at the claimed offsets in the source document. What should the agent do, and why does the high confidence not save it?

Drop the obligation and escalate — the citation failed to verify. Confidence is the model's own opinion; citation verification is an objective fact checked in code. A high-confidence claim you cannot ground in the source is exactly the dangerous hallucination case, so integrity is checked before confidence is even considered. Asserting it would risk filing a wrong obligation — a legal liability.

Q: Why is prompt caching the right tool for a Q&A agent over a 200-page rulebook, and what one operational detail must you get right for it to stay correct?

Reading the huge document is the dominant cost; caching lets you pay to read it once (a ~1.25× cache-write premium) and then answer many questions at a deeply discounted cache-read (~90% off), so "ask it anything" becomes affordable — verify the exact numbers in Anthropic's current docs. The operational detail: key the cache by document version/hash and invalidate on change, or a stale cache will answer questions against an outdated document and cite the wrong version.

A representative regulated-industry case study, end to end: a bank builds an agent that extracts obligations, flags risk, and answers questions over large regulatory and contract documents — with a citation for every claim, a confidence gate, human sign-off, and an immutable audit trail. Real Anthropic features (tool use, structured output, citations, prompt caching, evals) as the load-bearing parts; a wrong obligation is legal liability, so the discipline is abstain, cite, and gate.

⏱️ ~2.5 hours🧪 6 steps🎯 Advanced→Tech-lead

Learning objectives

State the requirements that make a regulated document agent different: accuracy, auditability, no-hallucination, and strict data handling.
Design an architecture that ingests large documents, caches them, extracts to a schema with citations, gates on confidence, and logs an audit trail.
Use Anthropic's real features — tool use, structured output with a JSON schema, citations, and prompt caching — as the load-bearing building blocks.
Enforce the no-hallucination discipline: every obligation must cite a source span, or the agent abstains and escalates to a human.
Evaluate the agent on three axes: extraction accuracy, citation integrity, and abstention correctness.
Reason about cost with prompt caching on big documents, and harden against the failure modes that turn a wrong obligation into legal liability.

Representative scenario — not a real customerEverything below is a composite, illustrative regulated-industry scenario built to teach the design. It is not a real Anthropic customer, and it contains no named-customer facts, metrics, or incidents. Where a product detail could change over time, we say "verify in Anthropic's current docs" — treat the API shapes as illustrative and confirm them against the live documentation before you build.

1 · The scenario & its requirements advanced

A mid-size bank has a shelf of long documents: vendor master-services agreements, regulatory rulebooks, and internal policies — each tens to hundreds of pages. A compliance analyst's job is to read them and answer three questions: What are we obligated to do? What's the risk if we don't? Where exactly does the document say so? Doing this by hand is slow, and a missed clause can become a regulatory finding. The bank wants an agent that extracts obligations, flags risk, and answers questions over these documents — but only if it can be trusted the way a human analyst is trusted: with a citation for every claim and a human sign-off before anything is filed.

That trust requirement is the whole design. This is not a chatbot where a wrong answer is embarrassing; it is a system where a wrong obligation is a legal liability. So the requirements read like a controls document, not a feature list:

Requirement	What it means for the design
Accuracy	Extracted obligations must match the document. Measured against a labeled gold set per field, not judged by vibe.
Auditability	Every output is traceable: which document, which span, which model + prompt version, what confidence, who signed off. An immutable audit log.
No hallucination	The agent may only assert what it can cite. If it cannot ground a claim in a source span, it must abstain, not guess.
Data handling	Contracts contain confidential and often personal data. Retention, access control, and (where required) redaction are part of the pipeline, not an afterthought — verify your obligations under the applicable regime.
Human sign-off	No obligation is treated as authoritative until a person with authority reviews and approves it. The agent proposes; a human disposes.

Why an agent and not just searchKeyword search finds where a word appears; it doesn't tell you the obligation ("the Bank shall notify the regulator within 30 days") or its risk. The value is turning prose into structured, cited, reviewable obligations — exactly the schema-constrained-extraction pattern, applied to documents too large to eyeball.

2 · The architecture, end to end advanced

Read the pipeline left to right. A large document is chunked and cached so we don't re-pay to read it on every question; each chunk is run through a schema-constrained extraction that must return a source span (a citation) for every obligation; a confidence gate decides what is trustworthy enough to auto-accept; everything uncertain goes to human sign-off; and every step writes to an append-only audit log.

🗺️ How to read this diagram

This is the whole compliance agent in one row, read left to right. It shows how a large document becomes a set of trustworthy, cited obligations — and, crucially, where the agent hands off to a human.

Large doc — the raw input: a contract or rulebook, often hundreds of pages, far too big to re-read on every question.
Chunk + cache — the document is split on clause/section boundaries and marked with a cache breakpoint (cache_control), so you pay to read it once, not once per question.
Extract + cite (the purple, highlighted box) — the one schema-constrained call per chunk. The schema requires a citation, so every obligation comes back pointing at the exact source span it came from.
Confidence gate (the amber box) — plain code decides what's trustworthy: a verified citation plus high confidence auto-accepts; anything uncertain or ungrounded abstains and moves right.
Human sign-off (green) — a person with authority approves before anything is treated as authoritative. The agent proposes; the human disposes.
Audit log — every step is written to an append-only record, so you can replay exactly what was claimed, from which source, and who approved it.

In short: follow the single path left to right, then notice the safety valve in the middle — the confidence gate — and the human at the end. Nothing becomes authoritative until a person signs off, and the whole path is logged.

Three design commitments make this safe rather than merely clever. First, extraction is grounded: the schema requires a citation, so an obligation with no supporting span is invalid by construction. Second, the gate is conservative: uncertainty routes to a human, never to auto-accept. Third, the log is immutable: you can reconstruct exactly what the agent claimed, from which source, at which prompt version, and who approved it — which is what an auditor will ask for.

This is doc-intel with the safety dial turned to maxIf you did the Document Intelligence project, you know the shape: structured output + confidence + a review queue. A regulated compliance agent keeps that skeleton and adds two non-negotiables — a citation for every claim and an audit trail — because here the cost of a silent error is legal, not cosmetic.

3 · Ingesting & caching big documents advanced

A 200-page rulebook is far too big to re-send on every question. Anthropic's prompt caching is built for exactly this: you mark a large, stable prefix (the document) with a cache breakpoint, pay a one-time cache-write premium, and then every subsequent request that reuses that prefix pays a deeply discounted cache-read instead of full input price. That is what makes "ask twenty questions of one big contract" economically sane.

Real caching mechanics (verify in Anthropic's current docs)Public guidance at time of writing: you add cache_control ({"type": "ephemeral"}) to a content block to set a breakpoint; the cached prefix must clear a minimum token count (about 1024 for the larger models); a cache write costs roughly 1.25× the base input price and a cache read roughly 0.1× (about 90% off); the default cache lifetime is about 5 minutes, refreshed on use, with a longer 1-hour option. Numbers drift — confirm them live before you budget on them.

Two ingestion decisions matter. Chunking: long documents are split so that each obligation lives whole inside one chunk (split on clause/section boundaries, not blindly every N characters), because a citation that straddles a chunk boundary is useless. Cache placement: the document (and the stable system prompt + schema) go in the cached prefix; only the small, changing question goes after the breakpoint. The lab below models the cost of that arrangement with pure arithmetic — no network.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.

Python · prompt-caching cost model for a big-document Q&A session (runs offline)

caching_cost.py# stdlib only, no network. Models the economics of caching one big document
# across many questions. Prices are $ per 1M tokens; multipliers reflect Anthropic's
# PUBLIC caching guidance (cache write ~1.25x input, cache read ~0.1x input).
# Verify the exact numbers in Anthropic's current docs before budgeting.

def session_cost(doc_tokens, n_questions, q_tokens, out_tokens,
                 in_price, out_price, write_mult=1.25, read_mult=0.10):
    """Compare re-sending the whole doc every question vs caching it once."""
    # No cache: every question re-pays full input price for the whole document.
    no_cache_in = (doc_tokens + q_tokens) * n_questions
    no_cache = (no_cache_in * in_price + out_tokens * n_questions * out_price) / 1_000_000

    # With cache: pay the write premium once for the doc, then cheap reads thereafter.
    write = doc_tokens * in_price * write_mult
    reads = doc_tokens * in_price * read_mult * (n_questions - 1)
    questions_in = q_tokens * in_price * n_questions          # the small changing part
    outputs = out_tokens * out_price * n_questions
    cached = (write + reads + questions_in + outputs) / 1_000_000

    return {
        "no_cache_usd": round(no_cache, 2),
        "cached_usd": round(cached, 2),
        "saved_usd": round(no_cache - cached, 2),
        "saved_pct": round(100 * (no_cache - cached) / no_cache),
    }

# A 200-page rulebook (~150k tokens), 20 questions, on a mid model ($3 in / $15 out).
r = session_cost(doc_tokens=150_000, n_questions=20, q_tokens=80, out_tokens=350,
                 in_price=3.00, out_price=15.00)
for k, v in r.items():
    print(f"{k}: {v}")

no_cache_usd: 9.11
cached_usd: 1.53
saved_usd: 7.58
saved_pct: 83

▶ How this works

This tiny program answers the money question behind the whole design: if analysts ask many questions of one huge document, is it affordable? It compares re-sending the document every time against caching it once — pure arithmetic, no network.

no_cache assumes the naive approach: every one of the n_questions re-pays full input price to read the whole doc_tokens document. That's the expensive baseline.
The cached path pays a one-time write premium (~1.25× input price) to store the document, then cheap reads (~0.10×, about 90% off) for the remaining questions. Only the small changing question and the output are billed normally each time.
The function returns both totals plus the dollars and percentage saved, so you can see the trade at a glance.

What the output means: Reading the 150k-token document dominates, so caching it once instead of 20 times saves about 83% — roughly $9.11 down to $1.53 for the session.

Try this: Bump n_questions from 20 to 100 and watch saved_pct climb: the more times you reuse the same cached document, the more the one-time write premium is dwarfed by cheap reads. Confirm the exact multipliers in Anthropic's current docs.

Caching turns "ask it anything" affordableReading the document is the dominant cost, and you pay it once instead of twenty times. That's what lets analysts interrogate a contract conversationally instead of rationing questions — the discount scales with how many times you reuse the same cached document within its lifetime.

4 · Schema-constrained extraction with citations expert

The heart of the agent is one disciplined call per chunk: structured output with a JSON schema forces the model to return obligations in an exact shape, and the schema requires a citation on every obligation. Anthropic's citations feature is designed to return grounded spans that point back into the source document; combined with a required-citation schema, an obligation the model cannot ground simply cannot be emitted as valid.

Real features doing real work (verify in Anthropic's current docs)This design leans on three published capabilities: tool use / structured output to constrain the reply to a JSON schema; the citations feature to get source-grounded spans rather than free text; and a system prompt that states the no-hallucination rule explicitly. The exact request fields and citation payload shape evolve — confirm them in the current docs; the discipline below does not change.

Below is the deterministic, offline stand-in for that call. A real system would send the chunk to Claude with a required-citation schema; here a fake extractor returns obligations, each with a claimed quote and character offsets, plus a confidence. The point of the lab is the shape of the data and the contract it must satisfy — not the model call, which is why it runs with no key.

Python · schema-shaped extraction (offline fake stands in for the model call)

extract.py# stdlib only, no network. A deterministic fake extractor that returns the SAME
# shape a schema-constrained Claude call would: obligations, each REQUIRED to carry a
# citation (quote + character span into the source) and a confidence score.
from dataclasses import dataclass, field

DOC = (
    "Section 4.1 The Bank shall notify the Regulator within 30 days of any material "
    "breach. Section 7.2 The Vendor shall maintain insurance of at least $5,000,000. "
    "Section 9.0 Either party may terminate for convenience upon 90 days written notice."
)

@dataclass
class Citation:
    quote: str
    start: int
    end: int

@dataclass
class Obligation:
    party: str
    text: str
    risk: str                      # low | medium | high
    citation: Citation
    confidence: float

def fake_extract(doc):
    """Pretend-model output. In production this is one schema-constrained Claude call
    per chunk, with the citations feature returning the grounded spans."""
    spans = [
        ("Bank",   "notify the Regulator within 30 days of any material breach",
         "high",   0.94),
        ("Vendor", "maintain insurance of at least $5,000,000",
         "medium", 0.88),
        ("Either party", "terminate for convenience upon 90 days written notice",
         "low",    0.61),
    ]
    obligations = []
    for party, quote, risk, conf in spans:
        start = doc.find(quote)
        end = start + len(quote)
        obligations.append(Obligation(
            party=party, text=quote, risk=risk,
            citation=Citation(quote=quote, start=start, end=end),
            confidence=conf))
    return obligations

for o in fake_extract(DOC):
    print(f"[{o.risk:6}] {o.party:12} conf={o.confidence:.2f} "
          f"@({o.citation.start},{o.citation.end})")

[high  ] Bank         conf=0.94 @(27,85)
[medium] Vendor       conf=0.88 @(116,157)
[low   ] Either party conf=0.61 @(188,241)

▶ How this works

This is the shape of the agent's core call, with a deterministic fake standing in for the model so it runs offline. In production this is one schema-constrained Claude call per chunk, with the citations feature returning the grounded spans — but the data contract is exactly what you see here.

The @dataclass types Citation and Obligation are the schema in miniature: an obligation must carry a citation with a quote and character offsets. "An obligation with no source" isn't a bad answer here — it's an impossible shape.
fake_extract returns three obligations. For each, doc.find(quote) computes where that quote actually starts in the document, and start + len(quote) gives the end — so the offsets are real, not invented.
Each obligation also carries a risk level and a confidence score, the two signals the gate will use next.

What the output means: Three obligations print with their risk, party, confidence, and the @(start,end) span each citation points at — the exact structured data the rest of the pipeline consumes.

Try this: Add a fourth obligation whose quote is NOT in DOC. doc.find(...) returns -1, giving a nonsensical span — which is precisely the kind of ungrounded claim the next lab's verifier is built to catch.

The schema is the contractBecause the schema makes citation a required field, "an obligation with no source" is not a bad answer the model might give — it's an invalid shape the schema rejects. You've moved a safety rule from a hope into a type guarantee, the same way the doc-intel project moved arithmetic sanity into a validator.

5 · The no-hallucination discipline — abstain & cite expert

A required-citation schema stops the model from omitting a citation, but it cannot stop the model from fabricating one — quoting text that isn't actually in the document, or pointing at the wrong span. So the citation is not trusted; it is verified by code. This is the single most important control in the whole system, and it needs no model: you check that the claimed quote appears verbatim at the claimed offsets in the source. If it doesn't, the obligation is dropped and the chunk is escalated — the agent abstains rather than assert something it cannot ground.

A wrong obligation is legal liability — abstain, cite, and gateIn this domain, a confidently-wrong obligation that flows into a compliance filing is far worse than a gap flagged for a human. The rule is absolute: the agent may assert only what it can cite, and only citations that verify against the source count. Everything else abstains and routes to sign-off. Being right or abstaining always beats being confidently wrong.

Python · citation-integrity verification + confidence gate (runs offline)

gate.py# stdlib only, no network. Verifies each claimed citation against the source text,
# then GATES: verified + high-confidence -> AUTO-ACCEPT; verified + low-confidence ->
# REVIEW; a citation that does NOT verify is a hallucination -> DROP + ESCALATE.
CONFIDENCE_THRESHOLD = 0.80

def verify_citation(doc, cite):
    """A citation is valid only if the exact quote sits at the claimed offsets.
    This catches fabricated quotes and misaligned spans — with no model."""
    if not (0 <= cite["start"] < cite["end"] <= len(doc)):
        return False
    return doc[cite["start"]:cite["end"]] == cite["quote"]

def gate(doc, obligations):
    results = []
    for o in obligations:
        if not verify_citation(doc, o["citation"]):
            results.append((o["party"], "DROP+ESCALATE (citation failed to verify)"))
        elif o["confidence"] >= CONFIDENCE_THRESHOLD:
            results.append((o["party"], "AUTO-ACCEPT"))
        else:
            results.append((o["party"], "REVIEW (low confidence)"))
    return results

DOC = "The Bank shall notify the Regulator within 30 days of any material breach."
obligations = [
    # verified + confident -> auto-accept
    {"party": "Bank", "confidence": 0.94,
     "citation": {"quote": "notify the Regulator within 30 days", "start": 15, "end": 50}},
    # verified but low confidence -> review
    {"party": "Bank-B", "confidence": 0.55,
     "citation": {"quote": "material breach", "start": 58, "end": 73}},
    # FABRICATED quote (not in doc) -> drop + escalate
    {"party": "Ghost", "confidence": 0.99,
     "citation": {"quote": "pay a penalty of $1,000,000", "start": 15, "end": 42}},
]
for party, decision in gate(DOC, obligations):
    print(f"{party:8} -> {decision}")

Bank     -> AUTO-ACCEPT
Bank-B   -> REVIEW (low confidence)
Ghost    -> DROP+ESCALATE (citation failed to verify)

▶ How this works

This is the single most important control in the system. A required-citation schema stops the model from omitting a citation, but not from fabricating one. So citations are never trusted — they're verified in code against the source, before confidence is even considered.

verify_citation first bounds-checks the offsets, then does the real test: doc[start:end] == quote. The claimed quote must sit at exactly the claimed position. This catches both fabricated quotes and misaligned spans, with no model.
gate applies the rule in strict order: if the citation fails to verify, the obligation is dropped and escalated — no matter how confident the model was. Only a verified citation reaches the confidence check.
A verified, high-confidence obligation auto-accepts; a verified but low-confidence one goes to human review. Uncertainty and ungroundedness both route away from auto-accept.

What the output means: "Bank" (verified + 0.94) auto-accepts; "Bank-B" (verified + 0.55) goes to review; "Ghost" is dropped and escalated even at confidence 0.99, because its quote isn't in the document.

Try this: Give "Ghost" a real quote from DOC but leave the wrong offsets. It still fails, because verification checks the position, not just "is this text somewhere?" — the stronger guarantee for a legal document.

Confidence high, citation fake — still droppedNotice the "Ghost" obligation had confidence 0.99 and was still dropped. That is deliberate: model confidence is an opinion, but citation verification is a fact. A high-confidence claim you cannot ground in the source is exactly the dangerous case, so integrity is checked before confidence ever matters.

6 · Human-in-the-loop & the audit trail professional

The gate produces three buckets — accept, review, escalate — but "accept" still means proposed, not authoritative. A person with compliance authority reviews the proposed obligations and signs off. What makes the sign-off defensible is the audit trail: an append-only log where every event — extraction, verification result, gate decision, and human approval — is recorded with enough context to reconstruct exactly what happened and why. An auditor's question is always the same: "show me where this came from and who approved it." The log is the answer.

Good audit records share five fields: what (the obligation), where (the document + span), how (model + prompt/schema version), how sure (confidence + verification result), and who (the signer + timestamp). Make the log append-only so history cannot be quietly rewritten — that immutability is the point.

Python · append-only audit log with human sign-off (runs offline)

audit.py# stdlib only, no network. An append-only audit trail: every decision is recorded
# with provenance (document, span, model+prompt version, confidence) and, on sign-off,
# who approved it and when. Records are frozen so history cannot be rewritten in place.
from dataclasses import dataclass, asdict
import json, hashlib

@dataclass(frozen=True)                 # frozen -> a written record can't be mutated
class AuditRecord:
    obligation: str
    doc_id: str
    span: tuple
    model_version: str
    prompt_version: str
    confidence: float
    citation_verified: bool
    decision: str
    signed_off_by: str = ""             # empty until a human approves
    signed_at: str = ""

class AuditLog:
    def __init__(self):
        self._records = []              # append-only; never edited in place

    def append(self, rec):
        self._records.append(rec)
        return len(self._records) - 1   # the record's immutable index

    def sign_off(self, idx, approver, at):
        """Approval does not edit the original record — it appends a NEW one."""
        old = self._records[idx]
        approved = AuditRecord(**{**asdict(old),
                                  "decision": "APPROVED",
                                  "signed_off_by": approver, "signed_at": at})
        return self.append(approved)

    def digest(self):
        """A hash over the whole log — any tampering changes the digest."""
        blob = json.dumps([asdict(r) for r in self._records], sort_keys=True)
        return hashlib.sha256(blob.encode()).hexdigest()[:12]

log = AuditLog()
i = log.append(AuditRecord(
    obligation="Bank shall notify the Regulator within 30 days",
    doc_id="MSA-2026-0142", span=(15, 73),
    model_version="claude-x", prompt_version="oblig-extract-v3",
    confidence=0.94, citation_verified=True, decision="AUTO-ACCEPT"))
log.sign_off(i, approver="a.nguyen@bank.example", at="2026-09-06T10:22:00Z")

for n, r in enumerate(log._records):
    who = r.signed_off_by or "(unsigned)"
    print(f"#{n} {r.decision:11} verified={r.citation_verified} by={who}")
print("log digest:", log.digest())

#0 AUTO-ACCEPT verified=True by=(unsigned)
#1 APPROVED    verified=True by=a.nguyen@bank.example
log digest: 8d79848114bc   # varies per run — timestamp is hashed

▶ How this works

This is the audit trail — what makes a human sign-off defensible to an auditor. Every decision is stored with full provenance, and approval appends a new record rather than editing the old one, so history can't be quietly rewritten.

@dataclass(frozen=True) makes each AuditRecord immutable once written — you cannot mutate a record in place. Each carries the five audit fields: what (obligation), where (doc + span), how (model + prompt version), how sure (confidence + citation_verified), and — after sign-off — who and when.
append only ever adds to the list and returns the record's fixed index. sign_off does not edit record #0; it builds a new APPROVED record (copying the old fields, adding the approver) and appends it. Both the machine proposal and the human approval survive.
digest hashes the entire log, so any tampering with earlier records would change the digest — a cheap integrity check.

What the output means: Record #0 is the unsigned AUTO-ACCEPT proposal; record #1 is the human APPROVED sign-off. The proposal and the approval are both preserved — the agent proposed, the human disposed.

Try this: Call sign_off twice with different approvers and watch a third record appear. The trail grows; nothing is overwritten — which is exactly what an auditor wants to see.

Sign-off appends, never overwritesThe approval didn't edit record #0 — it appended record #1. That's what "append-only" buys you: the original machine proposal and the later human approval both survive, so the trail shows the agent proposed and the human disposed. (The digest line varies per run because the timestamp is part of the hashed content; in a real system you'd pin the clock and store the digest to detect tampering.)

7 · Evaluating the agent — accuracy, citations, abstention professional

You cannot ship this on faith. Three evals, run against a labeled gold set of documents, tell you whether it's safe to trust — and they map directly onto the three risks. Extraction accuracy: did it find the obligations that are actually there, without inventing ones that aren't? Citation integrity: do the citations it returns actually verify against the source? Abstention correctness: when it's genuinely uncertain or ungrounded, does it abstain — and does it abstain only then, not on everything?

Anthropic's published guidance on evals is the frame here: build a representative test set, score deterministically where you can, and treat the safety metric as a hard gate. The lab computes all three from a tiny gold set with pure Python.

Python · three evals: accuracy, citation integrity, abstention (runs offline)

evals.py# stdlib only, no network. Scores a run against a labeled gold set on the three
# axes that matter for a compliance agent. All deterministic — no model, no judge.
GOLD = {                          # doc_id -> set of true obligation ids present
    "A": {"o1", "o2"},
    "B": {"o3"},
}

# What the agent produced: (predicted ids, ids whose citation verified, ids abstained on)
RUN = {
    "A": {"predicted": {"o1", "o2"}, "verified": {"o1", "o2"}, "abstained": set()},
    # doc B: found o3 but its citation did NOT verify, and it correctly abstained on it
    "B": {"predicted": set(),        "verified": set(),        "abstained": {"o3"}},
}

def extraction_accuracy(gold, run):
    tp = fp = fn = 0
    for doc, truth in gold.items():
        pred = run[doc]["predicted"]
        tp += len(truth & pred)
        fp += len(pred - truth)
        fn += len(truth - pred)
    precision = tp / (tp + fp) if (tp + fp) else 1.0
    recall = tp / (tp + fn) if (tp + fn) else 1.0
    return round(precision, 2), round(recall, 2)

def citation_integrity(run):
    """Of everything the agent PREDICTED, what fraction had a verifiable citation?"""
    pred = sum(len(r["predicted"]) for r in run.values())
    ok = sum(len(r["verified"]) for r in run.values())
    return round(ok / pred, 2) if pred else 1.0

def abstention_correct(gold, run):
    """A missed true obligation is SAFE only if the agent abstained on it (didn't
    silently drop it). Silent misses are the dangerous failure."""
    for doc, truth in gold.items():
        missed = truth - run[doc]["predicted"]
        if missed - run[doc]["abstained"]:      # missed but NOT abstained = silent
            return False
    return True

prec, rec = extraction_accuracy(GOLD, RUN)
print(f"precision: {prec}  recall: {rec}")
print(f"citation_integrity: {citation_integrity(RUN)}")
print(f"abstention_correct (hard gate): {abstention_correct(GOLD, RUN)}")

precision: 1.0  recall: 0.67
citation_integrity: 1.0
abstention_correct (hard gate): True

▶ How this works

These are the three evals that tell you whether the agent is safe to trust, scored against a labeled gold set — all deterministic, no model or judge needed. Each maps onto one of the three risks: accuracy, grounding, and never-silently-wrong.

extraction_accuracy computes precision (of what it predicted, how much was true) and recall (of what was true, how much it found) from true-positive / false-positive / false-negative counts.
citation_integrity asks: of everything the agent predicted, what fraction had a citation that actually verified? Here it's 1.0 — nothing ungrounded slipped through.
abstention_correct is the hard gate. A missed true obligation is only safe if the agent abstained on it (flagged it for a human) rather than silently dropping it. missed - abstained being non-empty means a silent miss — an automatic fail.

What the output means: Precision 1.0, recall 0.67 (it missed one), citation integrity 1.0, and abstention_correct: True — the miss is acceptable because the agent abstained on it instead of pretending it found everything.

Try this: Change doc B so the agent neither predicts o3 nor abstains on it (empty abstained set). Re-run: abstention_correct flips to False — a silent miss, the dangerous failure this gate exists to catch.

Recall can dip; silent misses cannotRecall here is 0.67 — the agent missed one true obligation. That's acceptable because it abstained on it (routed it to a human) rather than silently dropping it. The hard gate isn't "find everything"; it's "never pretend you found everything". A run where abstention_correct is False fails no matter how high precision looks.

8 · Tech-lead — failure modes & hardening tech-lead

A lead owns the ways this system can hurt the business. Each failure mode below is a silent-error path, and each control turns the silent error into a visible one — a dropped claim, an escalation, an alert — because in compliance a visible gap is recoverable and a silent wrong answer is not.

Failure mode	Why it's dangerous	Hardening
🔴 Fabricated citation	A confident obligation quoting text not in the doc gets filed as real.	Verify every citation against the source in code (Step 5); a citation that doesn't verify drops the claim and escalates.
🔴 Chunk-boundary loss	An obligation split across two chunks is seen whole by neither, so it silently vanishes.	Chunk on clause/section boundaries with overlap; reconcile duplicates by citation span; eval recall against a gold set.
🟠 Stale cache	A cached document outlives an amendment; answers cite the old version.	Key the cache by document version/hash; invalidate on change; record the document version in every audit record.
🟠 Threshold drift	Model or prompt changes shift confidence, so the gate lets weaker claims through.	Re-run the eval set on every model/prompt version; pin and log `prompt_version`; alert on abstention-rate swings.
🔴 Sign-off theater	Humans rubber-stamp so many proposals that review is meaningless.	Show the citation span inline; keep the review queue small by gating conservatively; sample-audit approved records.
🟠 Data leakage	Confidential contract text lands somewhere it shouldn't (logs, prompts, retention).	Minimize what's logged; access-control the review UI and audit log; follow your retention/redaction obligations — verify the regime.

The mental model for a lead: the agent is a proposal engine wrapped in controls. Its job is to draft cited obligations fast; the controls' job is to guarantee that nothing untrustworthy is ever treated as authoritative. Get citation verification, conservative gating, the immutable audit trail, and versioned evals right, and the system degrades safely: under stress it abstains and escalates more, it does not start asserting confident nonsense.

The one invariant you never trade awayEvery asserted obligation is cited, every citation is verified, everything else abstains, and a human signs off before anything is authoritative — with the whole path in an immutable audit log. If a change would weaken any link in that chain to ship faster or cheaper, the answer is no. A wrong obligation is legal liability.

📋 Grade this design — compliance document agent

Dimension	Meets the bar	Above the bar (tech-lead)
Grounding / citations	Every obligation carries a citation (required by schema); the agent asserts only what it cites.	Citations are verified against the source in code before use; a non-verifying citation drops the claim and escalates, and citation integrity is a tracked eval.
No-hallucination	The agent abstains when it cannot ground a claim rather than guessing.	Abstention is a first-class, hard-gated metric; a silent miss scores worse than an abstention, matching the legal cost of a wrong obligation.
Auditability	Decisions and provenance are logged (doc, span, confidence, decision).	The log is append-only/immutable with model + prompt versions and human sign-off; any record can be replayed back to its source span on demand.
Human-in-the-loop	Uncertain results route to a human before anything is authoritative.	The queue is calibrated so reviewers see the risky cases without drowning; approvals are sampled-audited to prevent rubber-stamping.
Cost / caching	Large documents are cached so repeated questions don't re-pay to read them.	Cache is keyed by document version and invalidated on change; cost per reviewed obligation is a monitored metric.
Robustness under change	Chunking keeps obligations whole; the pipeline doesn't crash on messy input.	Every model/prompt change re-runs the eval set; threshold and abstention-rate drift trigger alerts before bad claims ship.

Score each row 0 (missing) / 1 (meets) / 2 (above). A passing design is 9+/12 with No-hallucination and Grounding both at 2 — an agent that can emit an un-verified obligation with no abstention path is an automatic fail, because a confident wrong obligation in a compliance filing is a legal liability, not a bug.

Exercise CS3.1 — Break the citation verifier

Context: A citation that is real text but points at the wrong offsets is the subtle attack a substring check misses. Predicting the verifier's verdict before running it is how you prove you understand the control.

Your task: In gate.py, add a fourth obligation whose quote is real document text but whose start/end offsets point at the wrong location; predict the decision, then explain why checking offsets beats a substring check for a legal document.

Requirements:

Add a misaligned-span obligation: a genuine quote at deliberately wrong offsets
Predict the decision before running — it should be DROP+ESCALATE because the text at those offsets won't match
Explain that verification checks position, not just "is this quote somewhere in the doc"
Argue why position-checking is the stronger integrity guarantee for a legal document
Connect it to the rule: citation verification is a fact checked in code, independent of model confidence

💡 Hint: Ask what the substring-anywhere check would wave through that the exact-offset check catches.

Exercise CS3.2 — Design the escalation policy

Context: The pipeline over 500 contracts produced a 4,000-item review queue and you have two analysts. Every lever you could pull trades safety against throughput — and one of them you must refuse even under deadline pressure.

Your task: Using evals.py as your measuring stick, decide what you change to shrink the queue — the confidence threshold, the chunking, the schema, or staffing — and state which change you would refuse to make even under deadline pressure, tied to the danger callout.

Requirements:

Use the eval metrics (accuracy, citation validity, abstention) as the measuring stick, not gut feel
Evaluate the candidate levers: raise the confidence threshold, fix chunking, tighten the schema, or add staffing
Show that each choice trades queue size against safety and defend your pick with the eval numbers
Name the change you would refuse: weakening citation verification or the abstain path to auto-accept more
Tie the refusal to the invariant that a confident wrong obligation is a legal liability, not a bug

💡 Hint: The one lever you can never pull is the one that lets an un-verified obligation reach "accepted" to clear the backlog.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Write the requirements for a compliance extractorBeginner

Context: In a regulated domain the usual default flips: a confident wrong answer is the worst possible outcome. Extracting clauses from contracts means the requirements read like a controls document, not a feature list.

Your task: For a compliance clause extractor (governing law, liability cap, termination notice over long contracts), write three requirements that reflect the compliance stakes.

Requirements:

Structured output: return each field with its value and a citation (document + location) — a bare value is unusable in compliance
Abstention: if a clause isn't present, return "not found" rather than guessing — a fabricated liability cap is worse than a blank
Auditability: log every extraction with model version, prompt, and source span so a reviewer can reconstruct why
Frame the design to optimize for citations and abstention, not coverage
State the flipped default explicitly: a confident wrong answer is the worst outcome

💡 Hint: Write each requirement as if a wrong extraction becomes a legal liability — because here it does.

Show solution

Structured output: return each field with its value and a citation (document + location) — a bare value with no source is unusable in compliance.
Abstention: if a clause isn’t present, return “not found” rather than guessing — a fabricated liability cap is worse than a blank.
Auditability: every extraction is logged with the model version, prompt, and source span so a reviewer can reconstruct why the agent said what it said.

Compliance flips the usual default: a confident wrong answer is the worst outcome, so the design optimizes for citations and abstention, not coverage.

Exercise 2 · Handle a document too big for one promptIntermediate

Context: A 200-page contract won't fit sensibly in one call, and re-reading it per question is ruinously expensive. The ingest design has to keep extraction both grounded and affordable.

Your task: Design the ingest + retrieval for a document too big for one prompt so extraction stays grounded and affordable.

Requirements:

Chunk on structural boundaries (sections/clauses), keeping a stable location id per chunk for citations
Index chunks (embeddings) once and cache the parsed/chunked form so re-runs don't re-ingest
Per field, retrieve only the few chunks most likely to contain it rather than stuffing the whole doc
Extract from just those chunks, citing the location id
Note that prompt caching (public Anthropic feature) further cuts the cost of a large stable instruction prefix

💡 Hint: Keep each call small for cost and accuracy, and make citations precise by carrying a stable location id per chunk.

Show solution

Chunk the document on structural boundaries (sections/clauses), keeping a stable location id per chunk for citations.
Index chunks (embeddings) once; cache the parsed/chunked form so re-runs don’t re-ingest.
Per field, retrieve the few chunks most likely to contain it (e.g. search “termination notice period”) rather than stuffing the whole doc.
Extract from just those chunks, citing the location id.

This keeps each call small (cost + accuracy) and makes citations precise. For very large stable prefixes, prompt caching (public Anthropic feature) further cuts the cost of the repeated instruction block.

Exercise 3 · Schema-constrained extraction with citationsAdvanced

Context: The heart of the agent is one disciplined call that returns machine-checkable output where every value carries a citation. A required-citation schema turns "did it cite?" from a judgment call into a boolean.

Your task: Design the extraction call so the output is machine-checkable and every value carries a citation — show the target schema and the grounding instruction.

Requirements:

Define a per-field schema with value, a citation (doc + location), and a found boolean
Allow explicit nulls: value=null, citation=null, found=false for an absent field
Write an instruction that extracts only from the provided sources and never infers an unwritten value
Have the instruction reward saying "not present" so the model doesn't drift toward always producing a value
Add post-parse checks: reject a row if found=true but the citation is null, or the cited location isn't among retrieved chunks
Explain that the schema makes grounding a boolean your code enforces, not a judgment call

💡 Hint: Make an obligation with no source an invalid shape the schema rejects, not a bad answer you hope to catch.

Show solution

Target schema (one object per field):
{
  "field": "liability_cap",
  "value": "12 months of fees" | null,
  "citation": {"doc": "MSA.pdf", "loc": "§8.2"} | null,
  "found": true | false
}

Instruction to the model:
"Extract each field ONLY from the provided sources. For each, return value,
 the citation (doc + location) it came from, and found=true. If a field is
 not present in the sources, return value=null, citation=null, found=false.
 Never infer a value that is not written in the sources."

Post-parse checks: reject the row if found=true but citation is null, or if the cited location isn’t among the retrieved chunks. The schema makes “did it cite?” a boolean your code can enforce, not a judgment call.

Exercise 4 · The no-hallucination discipline: abstain & citeExpert

Context: A required-citation schema stops the model omitting a citation but not from fabricating one. In compliance, recall of correctness beats coverage — the agent must be allowed and expected to say "not found."

Your task: Explain why "abstain and cite" beats "always answer," design the two mechanisms that operationalize it, and give the failure it prevents.

Requirements:

State the principle: a missing field routes to a human (cheap, safe); a hallucinated field can pass silently into a legal decision (expensive)
Mechanism 1: an explicit abstain path in the schema (found=false) plus a prompt that rewards saying "not present"
Mechanism 2: citation verification in code — a claimed value with no verifiable source is downgraded to "needs review," never surfaced
Note that model confidence is an opinion but citation verification is a fact — verify before confidence matters
Name the prevented failure: the model, pressured to answer, invents a plausible clause that sounds like boilerplate but isn't in this contract

💡 Hint: The schema stops omission; only code-side verification stops fabrication — you need both, in that order.

Show solution

Principle: in compliance, recall of correctness > coverage. A missing field routes to a human (cheap, safe); a hallucinated field can pass silently into a legal decision (expensive, dangerous). So the agent must be allowed and expected to say “not found.”

Two mechanisms:

Explicit abstain path in the schema (found=false) plus a prompt that rewards saying “not present” — otherwise models drift toward always producing a value.
Citation verification in code: a claimed value with no verifiable source in the retrieved chunks is downgraded to “needs review,” never surfaced as an answer.

Prevents: the classic RAG failure where the model, pressured to answer, invents a plausible clause that sounds like boilerplate but isn’t in this contract.

Exercise 5 · Human-in-the-loop and the audit trailProfessional

Context: "Accept" from the gate still means proposed, not authoritative. What makes a human sign-off defensible months later is an append-only audit trail that answers the auditor's only question.

Your task: Design the review workflow: what the agent hands a human, what the human does, and what gets logged so the whole thing is auditable months later.

Requirements:

Hand the reviewer, per field: value, citation linked to the exact source span, confidence, and any "needs review"/found=false flags surfaced first
Let the reviewer approve, correct (storing the correction), or reject — showing low-confidence and abstained fields first
Log an immutable record: document id + hash, model version, prompt template version, retrieved chunk ids, raw model output, reviewer decision, timestamps/user
Ensure the trail answers the dispute question: on what basis was this field set, and who approved it
Keep corrections — they become the best eval data for the next model change

💡 Hint: Store enough provenance that any field can be replayed to its source span and its approver on demand.

Show solution

Agent output to reviewer: for each field — value, citation (linked to the exact source span), confidence, and any found=false / “needs review” flags surfaced at the top.
Reviewer action: approve, correct (with the correction stored), or reject; low-confidence and abstained fields are shown first so human time goes where it matters.
Audit log (immutable): document id + hash, model version, prompt template version, retrieved chunk ids, the raw model output, the reviewer’s decision, and timestamps/user.

The audit trail answers the only question that matters in a dispute: “on what basis was this field set, and who approved it?” Store corrections — they become your best eval data for the next model change.

Exercise 6 · Evaluate the agent: accuracy, citations, abstentionIndustry scenario

Context: You can't deploy a compliance agent on faith. Three evals — mapped to the three risks — decide whether it's safe, and a release gate must prefer a version that abstains more over one that fabricates more.

Your task: Design the eval that proves the agent is safe to deploy in compliance, and the gate that blocks a bad release.

Requirements:

Field accuracy: value matches gold on present fields — core correctness
Citation validity: the cited span actually contains the value — grounding is the whole point (must be 100%)
Abstention recall: fraction of truly-absent fields returned as not-found — catches hallucination pressure
False-fabrication rate: present-with-value but wrong/unsupported — the dangerous failure, target ~0
Gate: block if false-fabrication rate rises above baseline or citation validity < 100%, even if raw accuracy improved
Build the eval set from reviewed real docs (de-identified) plus deliberately-absent-clause cases to test abstention directly

💡 Hint: Treat citation validity and fabrication as hard gates — in compliance you'd rather ship the version that abstains more.

Show solution

📋 Compliance-extractor eval

Metric	Definition	Why it matters here
Field accuracy	value matches gold on present fields	Core correctness
Citation validity	cited span actually contains the value	Grounding is the whole point
Abstention recall	% of truly-absent fields returned as not-found	Catches hallucination pressure
False-fabrication rate	% present-with-value but wrong/unsupported	The dangerous failure — target ~0

Gate: block release if false-fabrication rate rises above the current baseline or if citation validity < 100%, even if raw accuracy improved. In compliance you would rather ship a version that abstains more than one that fabricates more. Build the eval set from reviewed real docs (de-identified) plus deliberately-absent-clause cases to test abstention directly.

✓ Checkpoint — you can move on when you can…

State the four requirements (accuracy, auditability, no-hallucination, data handling) and why human sign-off is non-negotiable.
Draw the pipeline: large doc → chunk+cache → extract+cite → confidence gate → human sign-off → audit log.
Explain how prompt caching makes many questions over one big document affordable, and roughly why (write once, read cheap).
Say why a required-citation schema plus code-side citation verification is stronger than either alone.
Describe the three evals and why abstention correctness is the hard gate.
Name three failure modes and the control that turns each silent error into a visible one.

✓ Knowledge check

A proposed obligation comes back with confidence 0.97, but its cited quote does not appear at the claimed offsets in the source document. What should the agent do, and why does the high confidence not save it?

Show answer

Drop the obligation and escalate — the citation failed to verify. Confidence is the model's own opinion; citation verification is an objective fact checked in code. A high-confidence claim you cannot ground in the source is exactly the dangerous hallucination case, so integrity is checked before confidence is even considered. Asserting it would risk filing a wrong obligation — a legal liability.

✓ Knowledge check

Why is prompt caching the right tool for a Q&A agent over a 200-page rulebook, and what one operational detail must you get right for it to stay correct?

Show answer

Reading the huge document is the dominant cost; caching lets you pay to read it once (a ~1.25× cache-write premium) and then answer many questions at a deeply discounted cache-read (~90% off), so "ask it anything" becomes affordable — verify the exact numbers in Anthropic's current docs. The operational detail: key the cache by document version/hash and invalidate on change, or a stale cache will answer questions against an outdated document and cite the wrong version.

Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.

Learning objectives

1 · The scenario & its requirements advanced

2 · The architecture, end to end advanced

3 · Ingesting & caching big documents advanced

4 · Schema-constrained extraction with citations expert

5 · The no-hallucination discipline — abstain & cite expert

6 · Human-in-the-loop & the audit trail professional

7 · Evaluating the agent — accuracy, citations, abstention professional

8 · Tech-lead — failure modes & hardening tech-lead

Exercise CS3.1 — Break the citation verifier

Exercise CS3.2 — Design the escalation policy

🪜 Practice ladder beginner → industry

✓ Checkpoint — you can move on when you can…

More in Case studies