Deep Research Agent

The brief advanced

You have probably watched a "research agent" demo that dazzled you — until you clicked one of its citations and the linked page said nothing of the sort. That single moment of broken trust is the failure this entire project exists to prevent. Building an agent that reads fast is easy; building one you can trust to tell you only what a real source actually says is the hard, valuable part — and the difference lives in the design, not the model.

Picture the gap between a keen intern and a seasoned research analyst. The intern hands you a fluent, confident summary — and half of it is quietly invented, because "sound plausible" is exactly what a language model is best at. The analyst hands you the same summary with a footnote on every sentence, and when you check the footnotes, they hold up. This project builds the analyst, not the intern: a system whose output is only ever as strong as the evidence it can point to. The brief below states the job in business terms — market scans, due diligence, literature reviews — but read it with that intern-vs-analyst tension in mind, because every design choice on this page (planning, verification, citation integrity, budget caps) is really a mechanism for turning a fluent guesser into a disciplined citer. The common mistake teams make is treating this as a prompting problem ("just tell it to cite its sources") when it is an architecture problem — trust has to be enforced in code, not politely requested in a prompt, and that is what the rest of the page shows you how to do.

"Do the hours of reading-and-synthesizing that go into a briefing — with sources." Market scans, competitive analysis, literature reviews, due diligence — all involve gathering many sources and distilling them. An agent that plans the research, gathers and cross-checks sources, and produces a cited draft turns a day of work into minutes, with a human editing the final.

1 · Discovery — where does the time go? advanced

The seductive trap of any agent project is to automate the whole job at once — and it is exactly how you build something impressive that nobody uses. If you hand the agent the final judgment call, one hallucinated fact in a board briefing costs more than every hour it saved. So before writing a line of code, you do discovery: you look at how a human actually spends the day and ask, task by task, "does the agent have leverage here, and is the downside of a mistake survivable?"

Think of it like handing work to a talented but occasionally careless junior. You gladly delegate the grinding parts — reading forty tabs, pulling the same fact from five places, drafting the outline — because those are tedious, high-volume, and easy to check. You do not hand over the one decision that carries the reputation of the report: which sources to trust and what the final narrative should be. That is the logic behind the table's stars. The columns aren't rating difficulty; they're rating fit — how repetitive and verifiable the task is (agent wins) versus how much irreversible judgment it demands (human keeps it). Read each row as a deliberate line drawn between "draft this for me" and "I decide this," because that line is what makes the whole system safe to ship: the agent does the hours of gathering and cross-checking, and a person still owns the call at the end.

2 · Architecture advanced

The first instinct — "give the model a search tool and let it figure it out" — produces an agent that wanders. It searches, gets distracted by an interesting tangent, forgets what it was answering, and stops when it feels done rather than when the question is covered. The fix is not a smarter prompt; it is a shape — an explicit pipeline that forces the work through named stages in order, so no step can be skipped and every claim passes through the same gate before it reaches the page.

The analogy that makes this click is a newsroom, not a search box. A story doesn't go straight from tip to print: an editor breaks the assignment into angles, reporters chase each angle, a fact-checker challenges every claim against its source, and only then does anything get published — with an editor deciding whether there's enough to run. This architecture is that newsroom drawn as boxes: Plan (the editor splitting the question into sub-questions), the three gather tools (reporters — web search, fetch, and RAG over your own docs), Verify (the fact-checker, the amber box that is the whole point of the design), an enough? decision that either loops back for more or stops, and finally the synthesis into a cited report.

Why draw it as a fixed pipeline instead of a free-roaming agent? Because the shape is what gives you the guarantees. When gathering, verifying, and writing are separate stages, you can force the writer to see only verified findings — so an unsupported claim physically cannot reach the report. A free-form agent that does everything in one blur has no such choke point. The one loop (the dashed arrow back from "enough?") is the deliberate exception: research genuinely needs iteration, so the design allows exactly one controlled cycle, bounded by a budget, rather than open-ended wandering. Read the diagram once left-to-right, then trace the dashed arrow, and you have the entire project in your head.

A genuinely agentic loop (Ch 4): the agent plans sub-questions, gathers evidence from web search/fetch and internal docs (RAG), verifies each claim against its source, decides whether it has enough (looping back to search gaps), and finally synthesizes a cited report. This is the most "Tier 4" of the projects.

3 · Risk & quality model advanced

Every other project on this site has a "safety model" about not deleting things. A DevOps agent could drop a table; a support agent could refund the wrong customer. So your instinct arriving here is to ask "what can this agent break?" — and the answer is nothing. It only reads. That absence is a trap, because it makes a research agent feel low-risk when it is actually one of the more dangerous patterns you can ship.

The danger is a different kind of harm, and naming it is the point of this section. A research agent's damage is epistemic: it puts false beliefs into a human's head. The classic image is a forged document — it does no physical harm sitting on a desk, but the moment someone acts on it, the harm is real and often irreversible. A confident, fluently-written report with three fabricated statistics is exactly that forged document: the more polished it reads, the more it gets trusted, and the trust is what does the damage. This is why, for a research agent, the quality controls are the safety model — there is no separate "don't break prod" layer, because the way this agent breaks things is by being wrong persuasively.

Read the table below as a threat model, not a QA checklist. Each red row is a way the agent can hand you a plausible lie — invented facts, faked citations — and each control is a mechanism that makes that specific failure hard to commit, not just discouraged. The mistake to avoid is trusting any control that lives only in the prompt ("please don't make things up"); the durable controls in this table are the ones enforced in code and by a mandatory verify pass, which is why the golden-rule callout that closes the section is stated as an absolute.

Research agents don't touch infrastructure — the risks are epistemic: fabricated facts, fake citations, and shallow coverage. Quality controls are the safety model.

4 · Tool surface advanced

An agent is only as capable — and only as safe — as the tools you give it. The tool surface is the complete list of actions the model is allowed to take in the world; everything else is off-limits by construction. So designing this list is not a chore before the real work, it is the security boundary, and the striking thing about a research agent's list is what's missing from it: there is no write, no delete, no send. Every tool is read-only or a scratchpad. That is deliberate — the fastest way to keep an agent from doing damage is to never hand it a tool that can.

The one tool worth pausing on is note_finding, because it does the quiet heavy lifting of the whole design. It looks like a trivial "write this down" scratchpad, but it is actually the mechanism that makes citations honest. When the agent gathers a fact, it must record the claim together with the id of the source it came from — and later, the writer is allowed to cite only from that recorded set. Think of it as an evidence locker with a chain of custody: nothing can be cited in the final report unless it was logged here first, by a tool, at the moment it was actually retrieved. The model never gets to conjure a citation, because the set of citable sources is built by your code, not by the model's imagination. Read the risk column as the design's conscience: green everywhere, and the two truly powerful capabilities (web search, fetch) come with the standing warning that fetched web content is untrusted — a page can contain instructions trying to hijack your agent, which the next chapters teach you to defend against.

5 · The research loop — plan, gather, verify, synthesize advanced

Ask a beginner to build a research agent and you get one giant prompt: "research this question and write a cited report." It works in the demo and falls apart in production — the model gathers and writes in one breath, so it has no moment where it could stop and check whether a claim it just wrote is actually true. The loop below exists precisely to break that one big step into separate, inspectable stages, and the separation is the whole trick.

The reason a good analyst never merges these steps is worth internalizing: you cannot fact-check yourself in the same motion as writing. The mindset that generates fluent prose ("make it sound complete") is the opposite of the mindset that catches a bad claim ("prove this sentence"). So the loop physically separates them — plan, then gather, then verify as its own adversarial pass, then assess coverage, then and only then synthesize. Each stage hands a clean artifact to the next: the plan is a list of sub-questions, gathering produces findings-with-sources, verification produces the subset that survived scrutiny, and synthesis sees only that survivor set. Because verification is its own step with its own strict prompt, a claim the evidence doesn't support gets dropped before the writer ever sees it — which is impossible in the one-giant-prompt version.

The step to respect most is "assess coverage," because it's where cost and quality collide. Real research needs iteration — the first pass always leaves gaps — but iteration is also where an agent burns money and time endlessly. So this stage is a bounded loop: a "what's missing?" critic can trigger one more round of searching, but only within a budget. That single guarded cycle is the difference between an agent that digs deeper when it should and one that spirals forever chasing marginal facts.

This is the richest agent pattern in the course. It composes several techniques:

1. Plan — decompose the question into sub-questions (structured output: a list of angles to cover).
2. Gather — for each sub-question, search + fetch (often in parallel), and note_finding with the source.
3. Verify — for each noted claim, re-check it against its source (an adversarial "is this actually supported?" pass — the Ch 5 idea). Drop unsupported claims.
4. Assess coverage — a "what's missing?" critic; if there are gaps, loop back and search more (bounded by a budget).
5. Synthesize — write the report from verified findings only, citing each claim.

6 · Evaluation expert

Here is the trap that catches every first-time builder of a research agent: the output reads beautifully, so you assume it's good. Fluency is precisely what a language model optimizes for, which means "sounds authoritative" tells you nothing about "is true." Eyeballing a polished report and nodding is the single most common way a broken research agent ships — it feels like evaluation, but you're only measuring prose, not correctness.

The mental shift is to stop grading the writing and start auditing the evidence. An editor doesn't judge a piece by how confident it sounds; they pull three citations at random and check that the sources actually say what the sentence claims. That is exactly what these evals mechanize. The keystone metric is citation accuracy — for a sample of claims, does the cited source genuinely support them? — because it directly measures the failure mode that matters. Alongside it sit groundedness (what fraction of claims are backed at all), coverage (did it answer the sub-questions a good analyst would), and cost.

Notice the crucial distinction the table draws between two kinds of check, because it's a pattern you'll reuse everywhere. Some evals are deterministic — "every citation was actually fetched by a tool" is a plain set-membership test in code, and it either passes or fails with no judgment involved. Others are judged — "does this source support this claim" needs an LLM-judge plus spot human checks, because it's a matter of degree. The common mistake is reaching for a fuzzy LLM-judge on a question that could have been a hard-coded assertion: whenever a rule can be checked deterministically (like citation integrity), do it in code, where it's free, instant, and never wrong — and save the expensive judged evals for the genuinely subjective calls.

7 · Phased rollout expert

The exciting version of this project — plan, parallel search, verify, coverage critic, full cited reports — is also the version most likely to die half-built. You spend three weeks wiring every stage, nothing works end-to-end until the last day, and you can't tell which of five moving parts is producing garbage. That is the failure a phased rollout is designed to avoid: instead of building the cathedral in one go, you ship the smallest thing that proves the hardest idea, then add capability one rung at a time.

The ordering here is not arbitrary, and seeing the logic is the lesson. Phase 1 does the least ambitious task — answer one focused question with citations — but it proves the scariest property first: that the agent never cites a source it didn't fetch. Nail citation discipline on an easy case and you've de-risked the whole project; get it wrong and nothing built on top can be trusted anyway. Only then does Phase 2 add multi-step planning and the verify pass, and Phase 3 the coverage critic, internal RAG, and budget controls. Each rung is independently shippable and independently testable, so a regression has one obvious suspect: the thing you just added.

The rung that never changes is the last one, and it is the most important design decision on the page: a human edits before anything is published or acted on. This isn't a temporary training-wheels measure you remove once the agent is "good enough" — it's the permanent boundary that keeps the epistemic risk from §3 survivable. The agent drafts; a person decides. Ship the autonomy in phases, but never ship away that final human check.

Skills & course map expert

Build-along plan expert

1. Single-pass first (Ch 4): a web-search + fetch loop that answers a focused question with citations. Prove it never cites a source it didn't fetch.
2. Add planning (P4 schema): decompose the question into 3–5 sub-questions before searching.
3. Add verification (Ch 5): for each claim, a check that the cited source supports it; drop the ones that fail.
4. Add a coverage critic: "what's missing?" → one more search round, bounded by a budget.
5. Synthesize: write a structured, cited report from verified findings only; stream it out.
6. Evals (Ch 5): sample claims and check citation accuracy + coverage on a set of test questions.

What you'll build expert

Ask a research question; the agent plans sub-questions, gathers from a source corpus, verifies each claim against its source, and writes a cited report using only verified findings. A human edits the final — the agent drafts.

Step 1 · A mock source corpus expert

Step 2 · Plan the sub-questions expert

agent/engine.pydef plan(question):
    p = client.messages.parse(model=MODEL, max_tokens=400,
        system="Break the research question into 3-5 focused sub-questions.",
        messages=[{"role":"user","content":question}],
        output_format=Plan)
    return p.parsed_output.sub_questions

agent/engine.pydef plan(question):
    p = client.responses.parse(model=MODEL, max_output_tokens=400,
        instructions="Break the research question into 3-5 focused sub-questions.",
        input=question,
        text_format=Plan)
    return p.output_parsed.sub_questions

Step 3 · Gather — with citation integrity expert

agent/engine.py# track the source_ids we actually gathered
gathered = {}
for sq in sub_questions:
    for hit in sources.search(sq, k=2):
        gathered[hit["id"]] = {"text": sources.fetch(hit["id"]), ...}

# model extracts 'claim | source_id' lines from ONLY these sources
for line in extracted.splitlines():
    claim, _, sid = line.rpartition("|")
    if sid.strip() in gathered:          # REJECT invented citations
        findings.append(Finding(claim=claim.strip(), source_id=sid.strip()))

Step 4 · Verify each claim expert

agent/engine.pydef verify(finding, gathered):
    src = gathered.get(finding.source_id)
    v = client.messages.parse(model=MODEL, max_tokens=300,
        system="Does the source text actually support the claim? Be strict.",
        messages=[{"role":"user","content":
            f"Source: {src['text']}\n\nClaim: {finding.claim}\n\nSupported?"}],
        output_format=Verdict)
    return v.parsed_output.supported

verified = [f for f in findings if verify(f, gathered)]   # drop unsupported

agent/engine.pydef verify(finding, gathered):
    src = gathered.get(finding.source_id)
    v = client.responses.parse(model=MODEL, max_output_tokens=300,
        instructions="Does the source text actually support the claim? Be strict.",
        input=
            f"Source: {src['text']}\n\nClaim: {finding.claim}\n\nSupported?",
        text_format=Verdict)
    return v.output_parsed.supported

verified = [f for f in findings if verify(f, gathered)]   # drop unsupported

Step 5 · Synthesize from verified findings only expert

terminalpython agent/engine.py       # needs key

The global EV market grew strongly in 2025: sales reached 17 million
units, up 25% year over year, with China about 60% of the total [s1].
Battery pack costs fell to $110/kWh [s2], and public charging points
passed 5 million globally [s3]...

Sources:
  [s1] https://ex.com/ev-2025
  [s2] https://ex.com/batteries
  [s3] https://ex.com/charging

6 · Tests & evals expert

Two layers of check guard the agent: fast, offline pytest that needs no API key (deterministic rules like the citation-integrity guard), and live evals.py that call the real model to catch fabricated or off-topic citations. The diagram shows both seams and the pass/fail gate.

terminalpython -m pytest tests/ -v      # no key
python evals.py                 # needs key

# tests
test_search_finds_relevant_sources PASSED
test_fetch_returns_text PASSED
test_citation_integrity_rule PASSED
4 passed

# evals
invalid citations: [] (expect [])
cited irrelevant source: False (expect False)
EV sources cited: 3/3
✅ evals passed

Troubleshooting expert

import re
def toks(s): return set(re.findall(r"[a-z0-9]+", s.lower()))

SOURCES = {
    "s1": "Churn rose to 8 percent in Q3 driven by onboarding friction.",
    "s2": "Net revenue retention held at 105 percent.",
    "s3": "Support wait times doubled after the March release.",
}
def search(query, k=2):
    q = toks(query)
    scored = sorted(SOURCES, key=lambda i: len(q & toks(SOURCES[i])),
                    reverse=True)
    return [{"id": i} for i in scored[:k]]
def fetch(sid): return SOURCES[sid]

print(search("why did churn rise"))   # [{'id': 's1'}, ...]
print(fetch("s1")[:20])

from dataclasses import dataclass
@dataclass
class Finding:
    claim: str
    source_id: str

SOURCES = {"s1": "Churn rose to 8 percent in Q3.",
           "s2": "NRR held at 105 percent."}

def gather(sub_questions, search, fetch):
    gathered, findings = {}, []
    for sq in sub_questions:
        for hit in search(sq, k=2):
            gathered[hit["id"]] = fetch(hit["id"])
    # model output (mocked): claim | source_id, incl. one INVENTED id (s9)
    extracted = "Churn rose to 8 percent | s1\nMade up fact | s9"
    for line in extracted.splitlines():
        claim, _, sid = line.rpartition("|")
        if sid.strip() in gathered:              # reject invented citations
            findings.append(Finding(claim.strip(), sid.strip()))
    return findings, gathered

def search(q, k=2): return [{"id": "s1"}]
def fetch(i): return SOURCES[i]
fs, _ = gather(["why churn"], search, fetch)
print([(f.claim, f.source_id) for f in fs])   # only the s1 claim survives

import re
def content(s): return set(re.findall(r"[a-z0-9]{3,}", s.lower()))

def supports(source, claim, thresh=0.8):
    c = content(claim)
    if not c: return False
    return len(c & content(source)) / len(c) >= thresh

def verify_all(findings, gathered):
    return [f for f in findings
            if supports(gathered[f["id"]], f["claim"])]

gathered = {"s1": "Churn rose to 8 percent in Q3 due to onboarding."}
findings = [
    {"claim": "churn rose to 8 percent", "id": "s1"},   # supported
    {"claim": "churn fell to 2 percent", "id": "s1"},   # NOT supported
]
kept = verify_all(findings, gathered)
print([f["claim"] for f in kept])   # ['churn rose to 8 percent']

def synthesize(verified):
    if not verified:
        return "Insufficient verified evidence to answer."   # refuse, don't fabricate
    lines = [f"- {f['claim']} [{f['source_id']}]" for f in verified]
    return "Report:\n" + "\n".join(lines)

verified = [{"claim": "churn rose to 8 percent", "source_id": "s1"}]
report = synthesize(verified)
print(report)
assert "[s1]" in report                       # every claim is cited
assert synthesize([]).startswith("Insufficient")  # empty -> refuse
print("ok")

def research_loop(sub_questions, search, fetch,
                  max_subs=3, max_fetches=5):
    gathered, fetches = {}, 0
    for sq in sub_questions[:max_subs]:            # breadth cap
        for hit in search(sq, k=3):
            if fetches >= max_fetches:             # total-fetch cap
                return gathered
            if hit["id"] not in gathered:          # dedupe by id
                gathered[hit["id"]] = fetch(hit["id"]); fetches += 1
    return gathered

CORPUS = {f"s{i}": f"source {i}" for i in range(10)}
def search(q, k=3): return [{"id": "s1"}, {"id": "s1"}, {"id": "s2"}]
def fetch(i): return CORPUS[i]
g = research_loop(["a", "b", "c", "d"], search, fetch)
print(sorted(g))          # ['s1', 's2']  -- deduped, capped

# --- real structured calls (needs creds / API key — Claude OR OpenAI) ---
# from pydantic import BaseModel
# class Plan(BaseModel):    sub_questions: list[str]
# class Verdict(BaseModel): supported: bool
#
# Claude (Anthropic):
# import anthropic
# client = anthropic.Anthropic(); MODEL = "claude-opus-4-8"
# def plan(question):
#     p = client.messages.parse(model=MODEL, max_tokens=400,
#         system="Break into 3-5 focused sub-questions.",
#         messages=[{"role":"user","content":question}], output_format=Plan)
#     return p.parsed_output.sub_questions
#
# OpenAI: same Pydantic models, Responses API with structured parse:
# from openai import OpenAI
# client = OpenAI(); MODEL = "gpt-5.5"
# def plan(question):
#     p = client.responses.parse(model=MODEL, max_output_tokens=400,
#         instructions="Break into 3-5 focused sub-questions.",
#         input=question, text_format=Plan)
#     return p.output_parsed.sub_questions
# # verify() follows the same shape: swap messages.parse/output_format for
# # responses.parse/text_format, and parsed_output for output_parsed.

# --- offline fallback (runnable) ---
import os
def plan(question):
    if os.environ.get("ANTHROPIC_API_KEY"):
        raise RuntimeError("wire real_plan here")   # never reached offline
    return [f"{question} - detail {i}" for i in range(1, 4)]  # deterministic

print(plan("why did churn rise?"))   # 3 sub-questions, no key needed

Knowledge check check yourself