Multi-Agent Research Crew

The brief advanced

You've watched one agent try to hold the whole thing in its head — plan the questions, chase the sources, second-guess itself, and write the summary — and the answer came out plausible but thin, with a citation that turned out to point at nothing. That is the failure this project answers. A single agent is a soloist trying to play every instrument at once; a crew is an ensemble where each player owns one part.

Think about how a good newsroom handles a hard story. An editor breaks the question into angles, several reporters chase different sources at the same time, a fact-checker refuses to let an unverified claim through, and a writer stitches the survivors into a piece. Nobody does all four jobs, and that division is exactly why the output is stronger than one exhausted person could produce. A multi-agent research crew is that newsroom rendered as code: a planner, parallel researchers, a critic, and a writer, coordinated by an orchestrator.

The reason to specialize isn't that specialists are smarter — the same model backs every role. It's that a narrow role gets a narrow prompt, and a narrow prompt is easier to steer, easier to test, and harder to derail. A researcher told only "gather and cite evidence" won't wander off into writing conclusions; a critic told only "verify claims against their sources" won't get talked into agreeing. The common mistake is to reach for a crew because it sounds impressive — it isn't free. Every extra agent is more calls, more latency, and one more place errors can enter. Reach for it only when the task genuinely splits into distinct jobs and an independent check measurably raises quality; otherwise a single looping agent is the cheaper, more reliable choice, and the rest of this page keeps pushing you to prove the crew earns its keep.

"One agent doing everything is a jack of all trades — we want a team of specialists." Complex research benefits from division of labor: someone plans the questions, several people dig into sources in parallel, a skeptic pokes holes, and a writer synthesizes. A multi-agent crew mirrors that — specialized roles produce higher-quality, more thorough output than a single generalist agent on genuinely hard tasks.

1 · Discovery — do you actually need a crew? advanced

Here is the trap that swallows most multi-agent projects: they get built before anyone asks whether the task needed a team at all. The crew runs, produces a nice-looking report, and only weeks later does someone notice it cost five times what a single agent would have — for output that was no better. By then the architecture is load-bearing and nobody wants to rip it out.

So this section is a gate, not a formality. Before you write a line of orchestration code, you answer one question honestly: what does splitting this into roles buy me that one agent looping can't? The useful test is to imagine handing the task to a small human team. If you can point at genuinely different jobs — someone who plans, someone who digs, someone who checks, someone who writes — and especially if you can point at a job whose whole value is being independent (a fact-checker who didn't do the research and therefore isn't invested in it), then the roles are real and a crew earns its cost. If instead you're just describing one person's linear workflow, you've found a single agent wearing a costume.

The signal that matters most is the second row of the table: quality benefits from an independent critic. That independence is the entire reason a crew can beat a soloist — a separate agent reviewing claims it has no stake in catches errors the author would defend. The mistake to avoid is treating parallelism as the justification. Running three researchers at once saves wall-clock, which is nice, but it doesn't make the answer more correct; only the critic does that. Read the table below as a scorecard — the more three-star rows you can honestly tick, the more a crew is the right call, and the last row is the humbling reminder that "one agent is fine" is a perfectly good, cheaper answer.

2 · Architecture advanced

The first instinct when you wire up several agents is to let them all talk to each other — a group chat where everyone chimes in. Do that and you get exactly what a leaderless meeting produces: agents restating each other, arguing in circles, and no clear moment where the work is done. The shape below exists to prevent that. It is deliberately not a mesh.

Picture an assembly line with one foreman. Work enters at the top, moves through stations in a fixed order, and comes out the bottom — and critically, there is a single point of control deciding what happens next, not a committee. The orchestrator is that foreman: it's the only box that talks to everyone, so coordination lives in one place instead of being smeared across every pairwise conversation. The researchers are interchangeable stations that can run side by side; the critic is the quality-inspection station that work must pass before it reaches the end; the writer is the final station that assembles what survived.

The load-bearing detail — the one that makes this an architecture and not just a diagram — is the direction of flow. Findings move down through the critic before they reach the writer, which means the writer physically cannot synthesize from unverified material; it only ever sees what the critic passed. That's a structural safety property, not a rule you have to remember to enforce. The common mistake here is to let the writer read the raw research "for context" — the moment you do that, an unverified claim can slip into the report and the critic becomes decorative. Keep the pipe one-directional and the guarantee holds. Everything the code on this page builds is this exact top-to-bottom sequential flow.

An orchestrator plans the work and coordinates specialized agents: several researchers investigate different sources (often in parallel), a critic independently verifies claims and flags weak evidence, and a writer synthesizes the verified findings into a cited report. Each agent has a narrow role and prompt — specialization is what raises quality over a single generalist.

3 · Risk & safety model advanced

The seductive lie about multi-agent systems is that more agents means more safety — more eyes, more checks, more robustness. The opposite is the honest default. Every agent you add is another thing that can be wrong, another API bill, and another turn in a conversation that might never end. A crew multiplies capability, but it multiplies failure surface right alongside it.

The worst failure is quiet, and it's worth feeling. In a chain of agents, one researcher's mistake doesn't just sit there — it gets laundered. The writer receives it, phrases it confidently, cites a source next to it, and now a fabrication looks exactly like a fact. This is error compounding: bad input becomes confident bad output, and the polish the crew adds is what makes it dangerous. A lone agent that's unsure at least sounds unsure; a crew can make a wrong answer sound peer-reviewed.

So the controls in this table aren't garnish — they're the price of admission. The critic exists specifically to break the compounding chain by refusing to pass a claim its source doesn't support. The hard step and token caps exist because two agents can happily talk past each other forever, and "forever" on a metered API is a real invoice. Clear termination conditions exist because an orchestrator that ends the round is safer than open-ended chat that hopes to converge. Read each row as a specific failure someone has actually shipped, paired with the one control that contains it. The mistake to avoid above all: shipping a crew without caps and without a verifying critic, then being surprised when it loops, hallucinates, and bills you for the privilege.

4 · Orchestration patterns advanced

Watch someone discover multi-agent frameworks and you can almost predict the first build: a hierarchical manager delegating to a debating swarm, because it's the most impressive thing the docs demo. It also almost never works on the first try — the manager mis-delegates, the debaters won't agree, and you spend a week debugging an orchestration you never needed. The patterns below are a menu, and the whole skill is ordering the cheapest dish that feeds you.

The four patterns are really answers to one question: how do agents hand work to each other? Sequential is a relay race — a fixed baton pass, plan then research then critique then write. Hierarchical is a manager who decides at runtime who does what, useful when you can't know the sub-tasks in advance. Parallel is several runners on separate lanes whose results merge at the finish, which saves wall-clock when sub-questions are truly independent. Debate is two agents arguing to surface an error neither would catch alone, worth its cost only when accuracy is high-stakes.

Here's the point the whole course keeps making, applied to orchestration: start sequential, and earn every step up from there. A plain plan→research→critique→write pipeline is easy to reason about, easy to test, and easy to put a leash on — and for most research tasks it's genuinely all you need. Parallelism and hierarchy are optimizations; add them only once you can measure them helping, not because the architecture looks smarter with them. The classic mistake is picking the fanciest pattern up front and inheriting all its failure modes for a task a relay race would have handled. A well-run sequential crew beats a chaotic hierarchical one almost every time. The code sketch just below shows that sequential shape — four roles handed to a Crew with a hard max_steps leash.

crew.py (shape — CrewAI-style)planner   = Agent(role="Planner",   goal="break the question into sub-queries")
researcher= Agent(role="Researcher",goal="gather & cite evidence", tools=[search])
critic    = Agent(role="Critic",    goal="verify claims, flag weak evidence")
writer    = Agent(role="Writer",    goal="synthesize a cited report")

crew = Crew(agents=[planner, researcher, critic, writer],
            process="sequential", max_steps=20)   # the leash

5 · Agent / tool surface advanced

The tempting shortcut is to give every agent every tool — search, retrieval, export, the lot — so any of them can do anything. It feels flexible. It's also how a critic ends up "fixing" a claim by going and searching for support instead of judging what it was handed, and how a writer quietly publishes a report nobody approved. When every agent can do everything, no agent's role means anything.

Think of it like keys on a building's access cards. You don't hand every employee a master key; the researcher's card opens the archive, the writer's card opens the drafting room, and only a manager's card, plus a signature, opens the door marked "publish." Least privilege isn't bureaucracy — it's what lets you reason about what each agent can do wrong. The tool surface is where role specialization stops being a prompt and becomes an enforced boundary.

The pattern in the table is deliberate. The researcher is read-only: it can look things up but not change the world, so the worst it can do is retrieve badly. The critic gets no research tools at all — its job is to judge the claim against the source it was given, and handing it a search tool would let it rationalize a weak claim instead of failing it, quietly destroying the independence that made it valuable. The writer produces a draft and nothing more. And the one genuinely dangerous action — writing or publishing the report to a doc or ticket — is the single amber row, gated behind a human approval, because an autonomous system that can publish is an autonomous system that can publish something wrong. The rule to carry away: give each role the narrowest tools that let it do its job, and put a human in front of any action that touches the outside world.

6 · Evaluation expert

A multi-agent crew will always feel better than a single agent — it's more elaborate, it produces more structured output, and you built it, so you want it to win. That feeling is worth nothing. The uncomfortable question this section forces is the only one that matters: measured against a plain single agent, does the crew actually produce better answers, and is "better" worth what it costs?

The evaluations split into two honest halves. The first is does the machinery work: is the report accurate and fully cited, and — the load-bearing one — does the critic actually catch errors? A critic you never tested is a critic you should assume is broken, because a critic that rubber-stamps everything is invisible until the day it lets a fabrication through. The second half is is it worth it: the uplift-vs-single-agent number and the cost-per-task number, side by side. This pairing is the whole justification for the project. A crew that beats one agent by two percent while costing five times as much has failed its evaluation no matter how sophisticated it looks.

The single most important test is the one the tip below names, and it's the one people skip because it feels like sabotaging your own system: plant a known-false claim in the researchers' findings and confirm the critic drops it. This is how you prove the crew's central safety property empirically instead of hoping. Testing the happy path — feeding it good data and seeing a good report — tells you almost nothing, because a broken critic passes the happy path too. You have to feed it a lie and watch it get caught. Build in evidence of decline: if a prompt change makes the planted-error test start passing lies, your evals just earned their keep.

7 · Phased rollout expert

The failure mode for a working crew isn't that it breaks — it's that it works well enough in a demo that someone points it at the real world and lets it publish, all at once. A crew that autonomously writes reports into your docs or tickets on day one is a crew that will autonomously write a wrong report into your docs the first time the critic slips. Rollout is how you buy the trust to loosen the leash, one notch at a time.

The right way to grow it is the same ladder the whole course uses: start where a human is unambiguously in the loop, then earn each step up with evidence. Phase 1 is the crew running end-to-end but with a person reading the output before it's used — this is where you establish baseline quality and cost with the stakes low. Phase 2 is where you strengthen the critic and add parallelism, and crucially where you measure the uplift over a single agent, because that number is the permission slip for the extra machinery. Phase 3 is scaling carefully — hierarchical delegation, budgets, monitoring, a gated export — and only where the evals from §6 prove it pays.

The reason to climb slowly rather than jump is that each phase de-risks the next: you don't add autonomy to a crew whose quality you haven't measured, and you don't add parallelism you can't show helps. Read the red rung as the non-negotiable floor beneath all of this — never run without step and token caps, and never publish a report whose citations the critic hasn't verified. Those two are the lines that don't move regardless of how much trust the earlier phases earned. The common mistake is skipping straight to Phase 3 because the crew "seemed fine," which is precisely the confidence that ships the first laundered hallucination.

Skills & course map expert

How to use this page expert

Steps in order. terminal = run it; file = create it with the exact contents.

Step 1 · Folder + venv expert

terminalmkdir -p research-crew/tests
cd research-crew
python3 -m venv .venv
source .venv/bin/activate     # Windows: .venv\Scripts\Activate.ps1
pip install pytest "anthropic>=0.40"
pip freeze > requirements.txt

(.venv) ... Successfully installed anthropic-0.69.0 pytest-8.3.4

Step 2 · The agents (roles as functions) expert

Create agents.py. Each role is a function that takes a task and a model_call (injectable), so tests can script exact behaviour. The default mock_call makes it run with no key.

agents.py"""Role-specialized agents. Each is one job; model_call is injectable."""
from dataclasses import dataclass


@dataclass
class Finding:
    claim: str
    source: str        # the text the claim is supposed to come from


def mock_call(role: str, task):
    """Deterministic stand-in for an LLM, keyed by role. No API key."""
    if role == "planner":
        return ["What is our churn?", "What caused it?"]
    if role == "researcher":
        # returns a supported finding for each sub-query
        return [Finding(claim=f"Answer to: {task}",
                        source=f"Answer to: {task} (source text)")]
    if role == "writer":
        return "REPORT: " + "; ".join(f.claim for f in task)
    raise ValueError(f"unknown role {role}")


def plan(question, call=mock_call):
    return call("planner", question)


def research(subquery, call=mock_call):
    return call("researcher", subquery)


def write(verified_findings, call=mock_call):
    return call("writer", verified_findings)

Step 3 · The critic + the orchestrator expert

Create crew.py. The critic verifies each finding (claim must be supported by its source); the orchestrator runs plan → research → critique → write under a step cap, and the writer only ever sees verified findings.

crew.py"""Sequential crew with a step cap and a verifying critic."""
from agents import plan, research, write, mock_call, Finding

MAX_STEPS = 12


def critique(findings) -> list:
    """Keep only findings whose claim is supported by its source text.
    This is the crew's defence against compounding errors."""
    verified = []
    for f in findings:
        # 'supported' = the claim's words appear in its cited source
        if f.claim.lower() in f.source.lower():
            verified.append(f)
    return verified


def run_crew(question, call=mock_call):
    steps = 0
    subqueries = plan(question, call); steps += 1
    findings = []
    for q in subqueries:
        if steps >= MAX_STEPS:
            break                        # the leash across all agents
        findings += research(q, call); steps += 1
    verified = critique(findings); steps += 1   # the gate
    report = write(verified, call); steps += 1
    return {"report": report, "verified": verified,
            "dropped": len(findings) - len(verified), "steps": steps}


if __name__ == "__main__":
    r = run_crew("Why did churn rise?")
    print("REPORT:", r["report"])
    print("verified:", len(r["verified"]), "dropped:", r["dropped"],
          "steps:", r["steps"])

Step 4 · Run it, then test it (no key) expert

terminalpython crew.py

REPORT: REPORT: Answer to: What is our churn?; Answer to: What caused it?
verified: 2 dropped: 0 steps: 5

Now create the tests, including one where a researcher lies (claim not in its source) — the critic must drop it.

tests/test_crew.py"""Offline crew tests with scripted agents — no key."""
from agents import Finding
from crew import run_crew, critique, MAX_STEPS


def test_planner_produces_subqueries():
    r = run_crew("why did churn rise?")
    assert r["steps"] > 0 and r["report"].startswith("REPORT:")


def test_supported_claims_pass_critique():
    good = [Finding(claim="net 30", source="payment is net 30 days")]
    assert len(critique(good)) == 1


def test_critic_drops_planted_false_claim():
    # claim is NOT in its source -> must be dropped
    liar = [Finding(claim="revenue tripled", source="payment is net 30 days")]
    assert len(critique(liar)) == 0


def test_writer_only_sees_verified():
    mixed = [Finding(claim="net 30", source="net 30 terms"),
             Finding(claim="fake", source="unrelated text")]
    assert len(critique(mixed)) == 1


def test_step_cap_not_exceeded():
    r = run_crew("why did churn rise?")
    assert r["steps"] <= MAX_STEPS


def test_report_contains_only_verified_claims():
    # build a crew where one researcher lies via a scripted call
    def call(role, task):
        if role == "planner": return ["q1"]
        if role == "researcher":
            return [Finding(claim="true fact", source="a true fact appears here"),
                    Finding(claim="lie", source="nothing relevant")]
        if role == "writer": return "; ".join(f.claim for f in task)
    r = run_crew("q", call=call)
    assert "lie" not in r["report"] and r["dropped"] == 1

terminalpython -m pytest tests/ -v

tests/test_crew.py::test_planner_produces_subqueries PASSED
tests/test_crew.py::test_supported_claims_pass_critique PASSED
tests/test_crew.py::test_critic_drops_planted_false_claim PASSED
tests/test_crew.py::test_writer_only_sees_verified PASSED
tests/test_crew.py::test_step_cap_not_exceeded PASSED
tests/test_crew.py::test_report_contains_only_verified_claims PASSED

6 passed in 0.04s

Step 5 · Go live with Claude or OpenAI / CrewAI (optional) expert

Replace mock_call with a real model call — the orchestration is unchanged. A minimal real caller:

The real model_call — in Claude or OpenAI

The ROLES system prompts and the orchestrator are identical for both vendors — only the one real_call differs. In Claude you call client.messages.create(system=…, messages=[…]) and read msg.content[0].text; in OpenAI the same call is client.responses.create(instructions=…, input=…) and you read resp.output_text. Toggle the tab to switch SDK — run_crew(question, call=real_call) is unchanged either way.

real_call.pyimport anthropic
client = anthropic.Anthropic()
ROLES = {
  "planner": "Break the question into 2-4 sub-queries. One per line.",
  "researcher": "Answer with facts and quote the source text you used.",
  "writer": "Write a short cited report from the findings.",
}

def real_call(role, task):
    msg = client.messages.create(model="claude-opus-4-8", max_tokens=600,
        system=ROLES[role],
        messages=[{"role":"user","content": str(task)}])
    return msg.content[0].text
# then: run_crew(question, call=real_call)  (after setting ANTHROPIC_API_KEY)

real_call.pyfrom openai import OpenAI
client = OpenAI()
ROLES = {
  "planner": "Break the question into 2-4 sub-queries. One per line.",
  "researcher": "Answer with facts and quote the source text you used.",
  "writer": "Write a short cited report from the findings.",
}

def real_call(role, task):
    resp = client.responses.create(model="gpt-5.5", max_output_tokens=600,
        instructions=ROLES[role],
        input=str(task))
    return resp.output_text
# then: run_crew(question, call=real_call)  (after setting OPENAI_API_KEY)

Troubleshooting — every error you might hit expert

from dataclasses import dataclass

@dataclass
class Finding:
    claim: str
    source: str

def mock_call(role, task):
    if role == "planner":
        return ["What is our churn?", "What caused it?"]
    raise ValueError(role)

def plan(question, call=mock_call):
    return call("planner", question)

print(plan("why did churn rise?"))   # ['What is our churn?', 'What caused it?']

from dataclasses import dataclass
@dataclass
class Finding:
    claim: str
    source: str

def critique(findings):
    return [f for f in findings if f.claim.lower() in f.source.lower()]

good = Finding("net 30", "payment is net 30 days")
liar = Finding("revenue tripled", "payment is net 30 days")
kept = critique([good, liar])
print(len(kept), kept[0].claim)   # 1 net 30  -- liar dropped

from dataclasses import dataclass
@dataclass
class Finding:
    claim: str
    source: str
MAX_STEPS = 12

def mock_call(role, task):
    if role == "planner": return ["q1", "q2"]
    if role == "researcher":
        return [Finding(f"Answer to {task}", f"Answer to {task} (src)")]
    if role == "writer":
        return "REPORT: " + "; ".join(f.claim for f in task)

def critique(fs): return [f for f in fs if f.claim.lower() in f.source.lower()]

def run_crew(question, call=mock_call):
    steps = 0
    subs = call("planner", question); steps += 1
    findings = []
    for q in subs:
        if steps >= MAX_STEPS: break        # the leash across all agents
        findings += call("researcher", q); steps += 1
    verified = critique(findings); steps += 1
    report = call("writer", verified); steps += 1
    return {"report": report, "dropped": len(findings)-len(verified),
            "steps": steps}

print(run_crew("why did churn rise?"))

from dataclasses import dataclass
@dataclass
class Finding:
    claim: str
    source: str

def critique(fs): return [f for f in fs if f.claim.lower() in f.source.lower()]

def run_crew(question, call):
    subs = call("planner", question)
    findings = []
    for q in subs: findings += call("researcher", q)
    verified = critique(findings)                 # gate BEFORE writer
    return call("writer", verified)

def lying_call(role, task):
    if role == "planner": return ["q"]
    if role == "researcher":
        return [Finding("revenue tripled", "payment is net 30 days")]  # lie
    if role == "writer":
        return "REPORT: " + "; ".join(f.claim for f in task)

report = run_crew("q", lying_call)
print(report)                          # REPORT:  (empty -- lie dropped)
assert "revenue tripled" not in report
print("invariant holds")

class Budget:
    def __init__(self, total): self.left = total
    def charge(self, n):
        if self.left < n: return False
        self.left -= n; return True

def crew_with_budget(rounds, budget, cost=100):
    verified, stalls = [], 0
    for r in rounds:                      # r = list of (claim, source)
        if not budget.charge(cost):
            return {"verified": verified, "stopped": "budget"}
        before = len(verified)
        verified += [c for (c, s) in r if c.lower() in s.lower()]
        if len(verified) == before:
            stalls += 1
            if stalls >= 2:
                return {"verified": verified, "stopped": "no_progress"}
        else:
            stalls = 0
    return {"verified": verified, "stopped": "done"}

good = [("net 30", "net 30 terms")]
bad  = [("x", "unrelated")]
print(crew_with_budget([good, bad, bad], Budget(1000)))  # stopped no_progress
print(crew_with_budget([good]*20, Budget(300)))          # stopped budget

# --- real model call (needs creds / API key) ---
# import anthropic
# client = anthropic.Anthropic()
# ROLES = {"planner": "Break into 2-4 sub-queries, one per line.",
#          "researcher": "Answer with facts; quote the source text.",
#          "writer": "Write a short cited report from the findings."}
# def real_call(role, task):
#     m = client.messages.create(model="claude-opus-4-8", max_tokens=600,
#         system=ROLES[role], messages=[{"role":"user","content":str(task)}])
#     return m.content[0].text
# run_crew(question, call=real_call)   # after export ANTHROPIC_API_KEY=...
# OpenAI equivalent: from openai import OpenAI; client = OpenAI()
#   resp = client.responses.create(model="gpt-5.5", max_output_tokens=600,
#       instructions=ROLES[role], input=str(task)); return resp.output_text  # OPENAI_API_KEY

# --- offline: conflict reconciliation (runnable) ---
TIER = {"filing": 3, "blog": 1}     # trusted -> untrusted

def reconcile(findings):
    # findings: list of dict(topic, value, source_tier)
    best, conflicts = {}, []
    for f in findings:
        cur = best.get(f["topic"])
        if cur and cur["value"] != f["value"]:
            conflicts.append((f["topic"], cur["value"], f["value"]))
            if f["source_tier"] <= cur["source_tier"]:
                continue                      # keep the more-trusted incumbent
        best[f["topic"]] = f
    return {"kept": {t: v["value"] for t, v in best.items()},
            "conflicts": conflicts}

res = reconcile([
    {"topic": "revenue", "value": "$10M", "source_tier": TIER["filing"]},
    {"topic": "revenue", "value": "$30M", "source_tier": TIER["blog"]}])
print(res["kept"])       # {'revenue': '$10M'}  -- filing beats blog
print(res["conflicts"])  # [('revenue', '$10M', '$30M')]  -- flagged

research-crew/
├─ .venv/
├─ requirements.txt
├─ agents.py          (roles + Finding + mock_call)
├─ crew.py            (critic + sequential orchestrator + cap)
└─ tests/
   └─ test_crew.py    (6 offline tests incl. planted lie)

Knowledge check check yourself