Agents vs workflows
Most problems people reach for an agent to solve are better solved by a workflow — a fixed path of LLM calls you compose and control. This is the core of Anthropic's published "Building Effective Agents" guidance: start with the simplest thing that works, add autonomy only when the task genuinely needs it. This lesson turns that guidance into a decision you can make, with a runnable sketch of each of the five workflow patterns and the one true agent pattern.
Learning objectives
- State Anthropic's workflow-vs-agent distinction and why workflow-first is the default.
- Recognize each of the five workflow patterns in the wild and know when each applies.
- Model each pattern's control flow in plain Python — prompt chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer.
- Explain the cost / latency / reliability reasons autonomy is expensive.
- Decide — as a lead — when a true autonomous agent is justified, and defend it.
1 · The word "agent" hides a decision essential
Picture the kickoff meeting you've probably sat in. Someone with budget says "let's build an AI agent that handles our support tickets," everyone nods, and a few months later you have a slow, expensive system that loops unpredictably, is painful to debug, and does a job a handful of scripted LLM calls would have done cheaper and more reliably. The failure didn't happen in the code — it happened in that first sentence, when a single word ("agent") smuggled in an architecture decision nobody actually made on purpose.
Here's the analogy that unlocks it: think of a train on rails versus a self-driving car. The train is powerful precisely because it can't improvise — it follows a fixed track, so it's predictable, cheap to run, and you always know where it will be. The self-driving car can go anywhere, but you pay for that freedom in complexity, cost, and the constant worry about what it'll do at the next intersection. Most business problems are train problems wearing a car costume. The engineering skill this whole lesson teaches is telling them apart before you lay the track — or worse, before you buy a fleet of cars.
So here is the precise distinction, the one Anthropic's Building Effective Agents guidance turns on: it's about who owns the control flow — who decides what happens next. In a workflow, you own it: you write the path in code and the LLM fills in each step (draft this, classify that, summarize the other). In an agent, the model owns it: at runtime it chooses its own next action, calls tools in a loop, and decides when it's finished. Both are "agentic systems" — both put an LLM to work on cognition — but only the second is an "agent." The common mistake is treating "agentic" and "agent" as the same word and reflexively defaulting to the car; the disciplined move is to name the control-flow owner out loud, because that one choice cascades into every cost, latency, and reliability property you will then have to live with. Everything below is really about pushing that decision toward the simplest thing that still works.
"We should build an agent for this" is where a lot of LLM projects go wrong. In Anthropic's published engineering writing (the Building Effective Agents post), the useful split is between workflows — systems where you code the path and the LLM fills in the steps — and agents — systems where the LLM decides the path at runtime, directing its own tool use in a loop. Both are "agentic systems"; only one is an "agent." The decision between them is the first and most consequential design choice you make.
Per Anthropic's current guidance (verify in their docs, as wording evolves), the headline advice is deliberately unglamorous: find the simplest solution possible, and only increase complexity when it demonstrably improves outcomes. A single well-prompted call beats a workflow when it suffices; a workflow beats an agent when the steps are knowable. You climb that ladder only as far as the problem forces you.
Compare the two halves left and right of the dashed divider — they are the two things the single word "agent" gets used for.
- On the left (green), You write the path: step 1 → step 2 → step 3 are wired in your code, and the LLM only fills in the contents of each fixed box. That is a workflow.
- On the right (amber), the Model owns the flow: it picks each next action, runs an act → observe → decide step, and the curved arrow loops it back to choose again. That is an agent.
- The payoff is the divider itself: both put an LLM to work, but only the right side hands the model the steering wheel — and with it every cost, latency, and reliability property in §3.
In short: Name the control-flow owner out loud. You own it → workflow; the model owns it → agent.
2 · The decision, in one picture essential
When people get this decision wrong, it's almost always because they started from the technology instead of the task. "Should we use an agent, or LangGraph, or a workflow engine?" is the wrong opening question — it's like walking into a hardware store and asking "should I buy the drill or the saw?" before you've said whether you're hanging a picture or building a deck. The tool falls out of the job, not the other way around. So the picture below deliberately starts with the task and pushes technology to the very last box.
There's exactly one question that does all the work, and it's worth internalizing as a reflex: can I write down the steps before I run? Think of the difference between a recipe and a detective. A recipe is fully knowable up front — crack eggs, whisk, pour, bake — so you write it once and it runs the same every time; that's a workflow. A detective can't script the case in advance, because each clue changes what to investigate next; the path only reveals itself in motion. That's an agent. Most tasks that feel like detective work turn out to be recipes once you actually try to enumerate them — which is exactly why the "start simple" instinct pays off. When you truly cannot list the steps ahead of time, and the input genuinely dictates a different path each run, you've found the narrow case that earns an agent. Read the diagram with that single fork in mind: everything to the left is "what do I need," the middle is the one real decision, and everything to the right is a consequence of it.
The fork is about how well you can specify the task. If you can write down the steps up-front, you want a workflow: predictable, cheap, testable. If the task is open-ended — the steps depend on what the model discovers as it goes — you may need an agent. Everything else is choosing which workflow pattern fits.
This one picture is the whole lesson. Follow the arrows left to right; the middle box is the only decision you actually make.
- Task — start with what you need done, before naming any technology.
- Steps knowable? — the fork. Ask: can I write down the steps before I run? That single question decides everything downstream.
- Workflow (green) — if yes, you code the path and let the model fill in each step. Predictable, cheap, and testable one step at a time.
- Agent (amber) — if no, the path truly depends on what the model discovers, so the model drives the loop. More power, but more cost and less predictability.
In short: If you can enumerate the steps, it's a workflow — reach for the patterns in §4–§8. Only the genuinely un-scriptable case earns the agent in §9.
Read it left-to-right. The middle node is the only real question: can you enumerate the steps before you run? If yes, pick a workflow pattern (§4–§8) — you keep control, the system is debuggable, and each step is independently testable. If no — the path truly varies with the input and can't be scripted — that's the narrow case where an agent (§9) earns its cost.
3 · Why autonomy is expensive intermediate
"Workflow first" can sound like timid engineering — until you get the bill. The first time a team ships an agent to production and watches the token spend climb without a ceiling, the abstract advice suddenly becomes concrete. The reason isn't that agents are wasteful by nature; it's that autonomy has a specific, compounding cost structure, and if you can't see the shape of that structure you can't reason about when it's worth paying.
Here's the mental picture. A workflow is a taxi with a fixed route: three stops, a known fare, and you can predict to the dollar what the trip costs before you leave. An agent is a taxi with the meter running while the driver figures out the route as they go — and, crucially, the meter speeds up the longer the trip runs. That second part is the piece beginners miss: it's not just that an agent makes an unknown number of calls. It's that every turn re-sends the whole growing conversation (the observations so far, the tool results, the reasoning), so a call on turn 8 is processing far more tokens than the call on turn 1. Unknown length times growing cost-per-step is why agent spend curves upward rather than climbing in a straight line.
That single fact ripples into four properties, which the table below lays out side by side. Cost is bounded for a workflow (you can count the calls) and unbounded for an agent (it loops until done or capped). Latency is predictable and often parallelizable in a workflow, but a chain of serial round-trips in an agent — each turn waits for the last. Reliability favors the workflow because every step is testable in isolation, whereas an agent's errors compound: a wrong turn on step 3 poisons every step after it. And debuggability collapses in an agent because the path varies per run — the exact failure you saw yesterday may be unreproducible today. None of this makes agents bad; it makes them expensive, and expensive things need to earn their place. Read the code sketch after the table as a model of the shape, not a benchmark — the numbers are illustrative, but the upward curve is real.
Why is "workflow first" the default rather than a stylistic preference? Because an autonomous agent pays real, compounding costs a fixed workflow avoids. A workflow makes a known number of calls on a known path; an agent loops an unknown number of times, each turn re-sending a growing context.
Each bar is the cost of one LLM call; taller means more tokens. Compare the two sides across the dashed divider.
- Left (green): three equal bars sitting on the baseline — a workflow's fixed calls. They're the same height and you can count them before you run, so the total is bounded and known.
- Right (amber → red): the same act-observe loop, but each bar is taller than the last because every turn re-sends the whole growing conversation. The reddening tall bars are the turns that quietly dominate the bill.
- The payoff is the shape: unknown number of calls (the "? calls" label) times a per-call cost that climbs each turn is why agent spend curves upward instead of adding in a straight line.
In short: Workflow cost = a few equal bars you can count; agent cost = a loop of ever-taller bars. That upward curve is why the guidance is "reach for the loop last."
| Axis | Workflow | Autonomous agent |
|---|---|---|
| Cost | bounded — you know the call count | unbounded — loops until done or capped |
| Latency | predictable, often parallelizable | serial round-trips, grows with steps |
| Reliability | each step testable in isolation | errors compound across the loop |
| Debuggability | fixed path — inspect any step | path varies per run — hard to reproduce |
| When it wins | steps are knowable | task is genuinely open-ended |
cost_model.pydef workflow_cost(steps, cost_per_call):
"""A workflow makes a FIXED number of calls."""
return steps * cost_per_call
def agent_cost(avg_loops, cost_per_call, context_growth=1.0):
"""An agent loops an unknown number of times; each turn re-sends a
growing context, so later calls cost more."""
total = 0.0
for turn in range(avg_loops):
total += cost_per_call * (context_growth ** turn)
return total
wf = workflow_cost(steps=3, cost_per_call=0.01)
ag = agent_cost(avg_loops=8, cost_per_call=0.01, context_growth=1.2)
print(f"workflow (3 fixed calls): ${wf:.3f}")
print(f"agent (8 growing loops): ${ag:.3f}")
print(f"agent is ~{ag/wf:.1f}x the workflow here")
workflow (3 fixed calls): $0.030
agent (8 growing loops): $0.165
agent is ~5.5x the workflow here
4 · Pattern 1 — Prompt chaining intermediate
You've felt the failure this pattern fixes. You write one big prompt asking the model to "read this document, pull out the action items, prioritize them, and draft an email" — and the output is a muddle: it half-does each part, drops one entirely, and you can't tell which instruction it fumbled. Cramming several tasks into a single call is the LLM equivalent of asking someone to juggle while reciting poetry: possible, but everything gets a little worse. Prompt chaining is the fix, and it's the simplest step up from a single call.
The analogy is an assembly line. Instead of one worker building the whole car, each station does one job well and passes the result to the next: outline, then draft, then polish. Each step is smaller, so the model does it more reliably; and because the output of step one becomes the input of step two, the sequence composes into the full task. The precise definition: prompt chaining decomposes a task into a fixed, ordered series of LLM calls where each call's output feeds the next. "Fixed and ordered" is the whole point — you wrote the sequence in code, so it's a workflow, not an agent deciding the order at runtime.
The feature that turns a chain from a toy into something production-grade is the gate — a cheap check between steps that bails out early if a step produced junk. Think of it as quality control on the assembly line: if the outline came back empty, there's no point spending money drafting from it, so you stop and flag the failure. The common mistake here is the opposite of over-engineering — it's forgetting the gate entirely, letting a bad step-one output silently corrupt everything downstream until you're debugging a polished essay built on a broken outline. Add the gate, and each link in the chain becomes independently checkable, which is exactly the reliability property a single mega-prompt can never give you.
When to use: the task splits cleanly into fixed, ordered subtasks, each easier and more reliable than doing it all in one shot — and you can check the work between steps. Anthropic's canonical example: draft, then translate; or outline, then write. Each step's output is the next step's input.
Spot it in the wild: any "first do X, then do Y with the result" pipeline — generate-then-format, extract-then-summarize. If you find yourself asking one prompt to do three things, that's usually a chain wanting to be split — with an optional gate between steps that bails early if a step fails a check.
Follow the solid arrows left to right — that is the happy path of the chain.
- The blue boxes (outline, draft, polish, final) are the ordered steps; each arrow means "this step's output becomes the next step's input" — the hand-off that makes it a chain.
- The amber diamond-style box in the middle is the gate: a cheap check that runs between steps. On pass the chain continues to draft.
- The downward arrow to the red ABORT box is the payoff: on fail the chain stops early, so you never spend money drafting and polishing junk.
In short: Split one big ask into ordered steps that feed each other, and put a gate between them to fail fast — that's what makes a chain more reliable than a single mega-prompt.
chaining.pydef step(name, text):
"""Offline stand-in for an LLM call: deterministic, no network."""
return f"[{name}] {text}"
def gate_ok(text):
"""A cheap check between steps (e.g. 'did the outline have >=1 section?')."""
return "outline" in text
def chain(topic):
outline = step("outline", topic)
if not gate_ok(outline): # gate: stop early if step 1 failed
return "ABORT: outline failed its check"
draft = step("draft", outline) # each step feeds the next
polished = step("polish", draft)
return polished
print(chain("a blog post about caching"))
[polish] [draft] [outline] a blog post about caching
Prompt chaining splits one big ask into fixed, ordered steps, where each step's output feeds the next — and an optional gate between steps can stop early if a step fails a cheap check. This models that control flow with a fake step() so it runs offline.
step(name, text)is a stand-in for an LLM call — deterministic, no network — so you can see the shape of the pattern without an API key.outline = step("outline", topic)is step one. Its result becomes the input to step two — that hand-off is what makes it a chain.if not gate_ok(outline):is the gate — a cheap check between steps. If step one's output fails, wereturnan abort instead of wasting the later calls. Gates are what make chains reliable.- The remaining
draftthenpolishsteps each consume the previous output, so the final string nests all three markers.
What the output means: The gate passes (the outline contains "outline"), so all three steps run and you get [polish] [draft] [outline] a blog post about caching — the steps wrapped in order.
Try this: Change gate_ok to look for a word the outline won't contain and re-run — you'll get the ABORT path, proving the gate can stop a chain early.
5 · Pattern 2 — Routing intermediate
Here's the trap chaining doesn't solve. Suppose your support inbox gets billing questions, bug reports, and "what are your hours?" all mixed together. Write one prompt to handle all three and you're forced to stuff every instruction — refund policy, debugging steps, store hours — into one enormous system prompt, and the model does each job a little worse for the distraction. Worse, you're paying for a big, capable model on the trivial "what are your hours?" query when a tiny cheap one would nail it. Different kinds of input want different handling, and one prompt can't gracefully be all things at once.
The analogy is a hospital triage nurse. The nurse doesn't treat anyone — their entire job is a fast, cheap classification: chest pains go to cardiology, a broken wrist to orthopedics, a cold to the general clinic. The expensive specialists only ever see the cases meant for them, so each stays focused and efficient. Routing is that nurse: a cheap up-front classify step decides the category, then dispatches to a specialized handler built for exactly that category. The precise definition: routing classifies an input, then sends it down one of several distinct, specialized paths.
Two wins fall out of this, and naming them helps you spot when routing is the right reach. First, separation of concerns: each handler only ever sees its own kind of input, so its prompt stays small, focused, and testable — you can improve the billing path without touching the technical one. Second, a cost lever: since most real-world traffic is easy, you can route the bulk to a small cheap model and reserve the big expensive one for the genuinely hard cases. The common mistake is skipping the classifier and building one god-prompt with a pile of if-this-then-that instructions inside it — that's routing done badly, buried in a single call where you can't measure or tune it. Pull the classification out into its own explicit step and the whole thing becomes a debuggable dispatch table, as the lab shows.
When to use: inputs fall into distinct categories that are each handled better by a specialized prompt (or a different-sized model), and you can classify the input reliably. Anthropic's example: route customer queries to the right specialized handler; send easy cases to a small cheap model and hard ones to a big one.
Spot it in the wild: a classify-then-dispatch shape. The win is separation of concerns — each handler stays simple because it only sees its own kind of input — plus a cost lever: most traffic is easy and can go to a cheaper model.
Read left to right, then pick one of the three branches — a query only ever takes a single path.
- The blue query box is the incoming input; the arrow carries it into the amber classify box, the one cheap step whose only job is to name the category (billing / technical / general).
- The three arrows fanning out are the routes — but they're exclusive: the classifier's answer selects exactly one. Here that's the "dispatch table" from the lab.
- The three green handler boxes are the payoff: each is a small, focused prompt (or even a different-sized model) for its own category, so you can improve or cost-tune one path without touching the others.
In short: A cheap classifier up front lets each specialized handler stay simple — and lets easy traffic go to a cheaper model.
routing.pydef classify(query):
"""Offline classifier — a real one would be a cheap model call."""
q = query.lower()
if "refund" in q or "charge" in q: return "billing"
if "error" in q or "crash" in q: return "technical"
return "general"
def handle_billing(q): return f"BILLING desk: {q}"
def handle_technical(q): return f"TECH desk: {q}"
def handle_general(q): return f"GENERAL desk: {q}"
ROUTES = {"billing": handle_billing, "technical": handle_technical,
"general": handle_general}
def route(query):
kind = classify(query) # cheap step decides the path
return ROUTES[kind](query) # specialized handler does the work
for q in ["I want a refund", "the app crashed", "what are your hours"]:
print(route(q))
BILLING desk: I want a refund
TECH desk: the app crashed
GENERAL desk: what are your hours
Routing is classify-then-dispatch: a cheap step decides which category an input belongs to, then a specialized handler does the work. Each handler stays simple because it only ever sees its own kind of input.
classify(query)is the cheap routing step — here plain keyword checks, but in production a small, fast model. It returns a category name, not an answer.handle_billing,handle_technical,handle_generalare the specialized handlers — one per category, each free to use its own prompt or even a different-sized model.ROUTESmaps each category name to its handler, sokind = classify(query)thenROUTES[kind](query)is the entire dispatch: decide the path, then walk it.- The loop runs three different queries and each lands at the right desk — separation of concerns, plus the cost lever of sending easy traffic to a cheaper model.
What the output means: Three lines, each query routed to its matching desk: refund → BILLING, crash → TECH, hours → GENERAL.
Try this: Add a fourth category (say "sales") with its own keyword, handler, and ROUTES entry — the dispatch code doesn't change, only the routing table.
6 · Pattern 3 — Parallelization (sectioning & voting) advanced
Two very different problems, one shape. First problem: you need to summarize 200 documents, and doing them one after another means your user stares at a spinner for two minutes while calls that have nothing to do with each other wait politely in line. Second problem: you're asking the model a genuinely hard judgment call — "is this comment abusive?" — and a single answer feels like a coin flip you can't fully trust. Both are solved by running LLM calls at the same time rather than in sequence, and Anthropic splits this into two flavors worth keeping straight.
Sectioning is the first: break a task into pieces that don't depend on each other and run them concurrently. The analogy is a newspaper — you don't wait for the sports section to be written before starting the business section; different reporters write different sections simultaneously, and an editor staples them together at the end. Summarize each of the 200 documents in parallel, then join the results. The payoff is latency: the whole job takes about as long as the single slowest piece, not the sum of all of them.
Voting is the second, and it feels backwards until you see it: run the same task several times and aggregate the answers — majority vote, or best-of. The analogy is a panel of judges rather than a lone referee; any single judge can have an off moment, but the majority is far more stable. Here you deliberately spend more calls to buy confidence on a call that's too important to get wrong once. The key insight that ties both flavors together — and the design decision you can't skip — is the aggregation step: fan work out, then decide how you fan it back in. Concatenate for sectioning, take the majority for voting. The common mistake is treating the two as interchangeable: sectioning divides different work to save time; voting repeats the same work to raise reliability. Reach for the one whose payoff — speed or confidence — matches what your problem actually lacks.
When to use: the work can be split into independent pieces that run at once. Anthropic describes two flavors. Sectioning: break a task into subtasks that don't depend on each other and run them in parallel (e.g. one call per document, or one guardrail check running alongside the main answer). Voting: run the same task several times and aggregate — majority vote or best-of — to raise reliability on a hard judgment.
Spot it in the wild: a fan-out/fan-in shape. Sectioning cuts latency (pieces run concurrently); voting trades extra cost for confidence. Both need an aggregation step — the design choice is how you combine the partial results.
Both halves have the same fan-out / fan-in silhouette — the difference is what fans out and how you combine it.
- Left (sectioning): one task splits into different independent pieces (doc 1/2/3) that run at once, then the green join box concatenates them. The pieces differ, so the payoff is latency — the job takes about as long as the slowest piece.
- Right (voting): the same task runs several times; the amber "safe" and red "flag" boxes are independent verdicts. You don't join them — you take the majority, so the payoff is confidence.
- The element to notice on both sides is the fan-in: sectioning joins, voting picks the majority. That aggregation choice is the real design decision.
In short: Sectioning divides different work to save time; voting repeats the same work to raise reliability. Pick the one whose payoff matches what your problem lacks.
parallel.pyimport hashlib
from collections import Counter
def step(worker, item):
"""Offline stand-in for a per-piece model call."""
return f"{worker}:{item}"
def sectioning(docs):
"""Independent subtasks -> run each, then combine (fan-out / fan-in)."""
partials = [step("summ", d) for d in docs] # would be concurrent
return " | ".join(partials) # aggregation step
def one_run(task, i):
"""Deterministic stand-in for one model verdict (stable across runs)."""
h = int(hashlib.md5(f"{task}:{i}".encode()).hexdigest(), 16)
return "flag" if h % 4 == 0 else "safe"
def voting(task, runs=5):
"""Same task many times -> majority vote for reliability."""
votes = [one_run(task, i) for i in range(runs)]
winner, n = Counter(votes).most_common(1)[0]
return f"verdict={winner} ({n}/{runs} agreed)"
print(sectioning(["doc1", "doc2", "doc3"]))
print(voting("is this comment abusive?"))
summ:doc1 | summ:doc2 | summ:doc3
verdict=safe (4/5 agreed)
Parallelization has two flavors. Sectioning splits work into independent pieces that run at once and then combines them. Voting runs the same task several times and takes the majority — trading extra cost for confidence on a hard call.
sectioning(docs)runs onestep()per document (these would run concurrently in real code) and then aggregates with" | ".join— the fan-out / fan-in shape.one_run(task, i)is a deterministic stand-in for one model verdict — it hashes the task so the demo prints the same result every time (a real run would be a live model call).voting(task, runs=5)collects five verdicts and usesCounter(...).most_common(1)to pick the majority — the aggregation step for voting.- The design choice in both flavors is how you combine the partial results: concatenate for sectioning, majority-vote (or best-of) for voting.
What the output means: The sectioning line joins the three per-doc summaries; the voting line reports verdict=safe (4/5 agreed) — four of five runs agreed on "safe".
Try this: Raise runs to 9 and see how the agreement count changes. More runs cost more but make the majority more stable — that's the cost/confidence trade.
7 · Pattern 4 — Orchestrator-workers advanced
Sectioning had a quiet assumption baked in — and this is the pattern for when it breaks. When you summarize 200 documents, you already knew there were 200 pieces and what each piece was; the split lived in your code. But consider "refactor the auth module": you genuinely cannot know up front how many files it touches, or which ones, until something intelligent looks at the codebase. You can't write the fan-out in advance because the fan-out depends on the input. That's the gap orchestrator-workers fills.
The analogy is a general contractor on a renovation. The homeowner says "update the kitchen"; the contractor walks the site and decides what the job actually decomposes into — call a plumber for the sink, an electrician for the outlets, a tiler for the floor — dispatches each specialist, then assembles the finished result. The homeowner never listed those subtasks; the contractor figured them out on arrival. The precise definition: a central orchestrator LLM decomposes the task at runtime into a variable number of subtasks, dispatches a directed worker call for each, and synthesizes the results into one answer.
Now the subtle part, because this is exactly where people start calling things "agents" that aren't. Compared to sectioning, the difference is who decides the split: in sectioning you fixed it in code, here the model decides it at runtime. That dynamic decomposition is genuinely a step toward agency — the model is making a real planning decision. But it is still a workflow, and the reason is precise: the workers are bounded, directed calls, not a free-running loop. The orchestrator says "summarize file X" and gets exactly that back; there's no worker deciding on its own to go investigate three more files. The common mistake is seeing "the model decides" and concluding "it's an agent" — no. The model deciding the decomposition while you still control that each worker is a single directed call is the bridge between the two worlds, and keeping the workers bounded is what keeps you on the workflow side of it.
When to use: you can't enumerate the subtasks up-front — the number and shape of the pieces depend on the input — but a central LLM can decompose the problem, and the pieces are still done by directed worker calls. Anthropic's example: a coding change that touches an unknown set of files, where an orchestrator decides which files and dispatches a worker per file, then synthesizes the results.
Spot it in the wild vs. parallelization: in sectioning you fix the split in code; in orchestrator-workers the model decides the split at runtime. It's the bridge toward agency — dynamic decomposition — but the workers are still bounded, directed calls, not a free-running loop. That's what keeps it a workflow.
Read left to right: task in, one answer out — but the middle fan-out is decided at runtime, not in your code.
- The amber orchestrator box is the difference from sectioning: it looks at the task and chooses how many subtasks there are ("rename / imports / tests" for a refactor). A different input would produce a different count.
- The three blue worker boxes are each a single, bounded, directed call — the orchestrator says "do this one thing" and gets exactly that back. No worker runs its own loop.
- The green synth box fans the results back in. The caption text below is the payoff: the model owns the decomposition, yet bounded workers keep this on the workflow side of the line.
In short: Sectioning fixes the split in code; orchestrator-workers lets the model pick it at runtime — but keeping the workers bounded is what stops it becoming an agent.
orchestrator.pydef step(role, payload):
"""Offline stand-in for a model call."""
return f"[{role}] {payload}"
def orchestrator(task):
"""A central step decides the subtasks at RUNTIME (count varies)."""
if "refactor" in task:
subtasks = ["rename symbols", "update imports", "fix tests"]
else:
subtasks = ["do it"]
return subtasks
def worker(subtask):
return step("worker", subtask)
def run(task):
subtasks = orchestrator(task) # model picks the pieces
results = [worker(s) for s in subtasks] # a directed call each
return step("synthesize", " + ".join(results)) # combine
print(run("refactor the auth module"))
[synthesize] [worker] rename symbols + [worker] update imports + [worker] fix tests
Orchestrator-workers is the pattern for when you can't list the subtasks up front — their number and shape depend on the input. A central step decides the split at runtime; each piece is still a directed worker call, then a synthesize step combines them.
orchestrator(task)is the central decision: at runtime it chooses the subtasks. A "refactor" produces three; anything else produces one. That dynamic count is what separates this from fixed sectioning.worker(subtask)handles one piece — a bounded, directed call, not a free-running loop. The workers stay simple; the orchestrator holds the plan.subtasks = orchestrator(task)then a worker per subtask thenstep("synthesize", ...)is the full flow: decide the pieces, do each, combine.- It's the bridge toward agency — the model decides the decomposition — but because the workers are bounded calls with no self-directed loop, it stays a workflow.
What the output means: The orchestrator picks three subtasks for the refactor, a worker runs each, and the synthesize step joins them into one result string.
Try this: Pass a non-refactor task ("add a comment") and watch the orchestrator choose the single-subtask path — the number of workers changed with the input, at runtime.
8 · Pattern 5 — Evaluator-optimizer professional
Some outputs are never right on the first try, and you know it. A literary translation, a tricky piece of copy, a deep research answer — the first draft is a starting point, not a finish. A human writer knows this instinctively: they write, step back, critique their own work ("this paragraph is vague, the ending is weak"), and revise. The mistake is expecting a single LLM call to produce polished work when even skilled humans iterate. Evaluator-optimizer bottles that write-critique-revise cycle into a controlled loop.
The analogy is a writer with a demanding editor. The writer produces a draft; the editor reads it against clear standards and hands back specific, actionable notes ("add a concrete example here, tighten the intro"); the writer revises; the cycle repeats until the editor signs off. Two distinct roles, real feedback flowing between them. The precise definition: one LLM call generates, a second evaluates against explicit criteria and returns feedback, the generator revises with that feedback, and you repeat until it passes or you hit a cap. It only works when the criteria are clear enough that a critic can articulate why a draft falls short — vague "make it better" feedback gets you nowhere.
And here is the point that this section exists to hammer home, because it's the single most common confusion in this whole lesson: this has a loop, but it is still a workflow, not an agent. The distinction is who owns the loop. In evaluator-optimizer you own it — you wrote two fixed roles, you defined the scoring function, and above all you set a hard round cap so the loop is guaranteed to terminate whether or not the output ever passes. The model never decides to keep going; your for loop does. Contrast that with the agent in §9, where the model itself decides each next action and when it's done. The rule to carry away: a bounded, self-terminating loop with roles you defined is a workflow; a model-directed loop that stops only when the model says "done" is an agent. Keep the cap, and you keep control.
When to use: there are clear evaluation criteria and iteration measurably improves the output — and, crucially, a critic can articulate why a draft falls short. Anthropic's examples: literary translation, or deep search where a second pass catches what the first missed. One call generates; another evaluates against the criteria and hands back feedback; the generator revises; repeat until it passes or you hit a cap.
Spot it in the wild: a generate → critique → revise loop with a bounded number of rounds. It looks agent-ish, but the control flow is fixed: two roles, a scoring function, and a hard iteration cap. Don't confuse it with a full agent — the loop is your loop, not the model's.
Follow the arrows around the loop, and watch the two ways out.
- The blue generate box and amber evaluate box are the two fixed roles: one writes a draft, the other scores it against explicit criteria.
- The amber curved arrow is the revise cycle: on fail, evaluate hands feedback back to generate for another round. The green "pass" arrow is the other exit — a good-enough draft is accepted.
- The red "hard round cap" box above is the payoff element: it bounds the loop from outside, so it always terminates whether or not a draft ever passes. That cap is what keeps the loop yours, not the model's.
In short: It has a loop, but you own it — two fixed roles, a scoring function, and a hard cap. That's a workflow; a model-directed loop with no cap would be an agent.
evaluator.pydef generate(prompt, feedback=""):
"""Offline stand-in — appends any feedback to simulate improvement."""
base = len(prompt)
return {"text": prompt + feedback, "quality": base + len(feedback)}
def evaluate(draft, target=30):
"""Score against clear criteria; return pass + actionable feedback."""
if draft["quality"] >= target:
return True, ""
return False, " [add detail]"
def optimize(prompt, max_rounds=5):
feedback = ""
for r in range(max_rounds): # HARD cap -> always terminates
draft = generate(prompt, feedback)
ok, feedback = evaluate(draft)
if ok:
return f"passed on round {r+1}: {draft['text']!r}"
return "hit round cap without passing"
print(optimize("translate this line"))
passed on round 2: 'translate this line [add detail]'
Evaluator-optimizer is a generate → critique → revise loop against clear criteria — but it's your loop, with a hard round cap, so it always terminates. That cap is exactly what keeps it a workflow and not an agent.
generate(prompt, feedback)produces a draft; here it simply appends any feedback so its "quality" score rises each round — a stand-in for a model improving on a critique.evaluate(draft, target=30)scores against clear criteria and returns(passed, feedback)— actionable feedback the next generate can act on.for r in range(max_rounds):is the loop, andrangegives it a hard cap — aftermax_roundsit stops whether or not it passed. No unbounded looping.- Each round feeds the previous round's feedback back in, so quality climbs until it clears the target and we
returnthe passing draft.
What the output means: Round 1 scores 18 (below 30); round 2 adds feedback to reach 31 and passes, so it prints passed on round 2 with the revised text.
Try this: Lower target to 10 and it passes on round 1; raise it above what feedback can reach and it hits the round cap — the cap is the safety rail.
9 · The autonomous agent — and when it's justified professional
We climbed the whole ladder to get here, and that was the point. Every pattern so far kept you in charge of the path — even the ones with loops and runtime decisions. The autonomous agent is the moment you finally hand the steering wheel to the model, and it belongs at the top of the ladder precisely because you should reach it last, after the cheaper patterns have failed to fit. This is the detective from §2 made real: a system that plans, acts, observes what happened, and decides its own next move, turn after turn, until it judges the job done.
The defining mechanic is the act-observe loop owned by the model. It picks an action (call a tool, read a file, run a command), sees the result, and feeds that observation back into its own next decision — no path you wrote, no fixed number of steps. That's genuine power: it can handle tasks you truly cannot script, like "investigate this alert" where each finding changes what to look at next. But it's also why the agent inherits every cost from §3 at full strength — unbounded loops, compounding errors, a path that differs each run — and why its only guaranteed brake is a step cap you impose from outside.
So the real skill isn't building the loop (Chapter 4 did that); it's the discipline to run the gate check before you build it. Anthropic's published "should this be an agent?" test has four gates, and the mental move is to treat any single "no" as a veto that sends you back down to compose workflows instead. Complexity — is the task genuinely un-scriptable, or does it just feel that way until you try to enumerate it? Value — does the autonomy actually pay for its higher cost and latency, or are you buying a Ferrari to fetch groceries? Model capability — is the model reliably good at this specific task, or will it confidently wander off? Cost of error — are mistakes cheap and recoverable, or does a wrong action delete data with no human in the loop? The common mistake is running this check after falling in love with the agent idea and rationalizing the answers; run it honestly first, and most "agent" requests resolve into a couple of workflow patterns you can actually test.
When to use: the task is open-ended, you can't predict the number or order of steps, and the model needs to act on feedback from a real environment (tools, files, shell) over many turns. Anthropic's framing: agents are for problems where you can't hard-code the path, and where the value justifies higher cost, higher latency, and lower predictability. The model plans, acts, observes the result, and decides the next action — in a loop — until it judges the task done.
Anthropic's published "should this be an agent?" check has four gates: complexity (is it genuinely hard to script?), value (does the autonomy pay for its cost?), model capability (is the model actually good at this task?), and cost of error (are mistakes recoverable, or gated by a human?). If any gate is "no," compose the workflow patterns above instead.
Trace the amber cycle — unlike the workflows above, no arrow here was drawn by you; the model chooses each hop.
- The amber "model decides" box is the driver: it picks the next action. The blue "tool / env" box runs that action against the real world (a tool, a file, a shell).
- The amber curved arrow is the loop: the model observes the result and decides again — turn after turn — with no fixed step count. Only when it judges the job finished does the green "answer" box exit on "done".
- The red "step cap" box is the payoff to notice: it's the only guaranteed brake. A workflow terminates by construction; this loop terminates only when the model stops or the cap trips.
In short: The model owns this loop, so it inherits every §3 cost at full strength — reach for it last, and gate irreversible actions behind your step cap and human approval.
agent.pydef model_decides(observations):
"""Offline stand-in for the LLM choosing its OWN next action.
Returns ('act', tool) to keep going, or ('done', answer) to stop."""
if len(observations) < 3: # pretend it needs 3 lookups
return ("act", f"lookup_{len(observations)}")
return ("done", f"answer from {observations}")
def agent(goal, max_steps=10):
"""The MODEL owns the control flow — path varies per run."""
observations = []
for step in range(max_steps): # cap = the only safety rail
action, payload = model_decides(observations)
if action == "done":
return f"[{step} steps] {payload}"
observations.append(payload) # act -> observe -> loop
return "stopped: hit step cap"
print(agent("investigate the alert"))
[3 steps] answer from ['lookup_0', 'lookup_1', 'lookup_2']
max_steps cap trips. That's the reliability cost of autonomy, and why Anthropic's guidance is to reach for it last and gate irreversible actions (Chapter 4's safety rails).10 · Recognizing the patterns — a field guide tech-lead
The requests never arrive labeled. No stakeholder walks up and says "I'd like a routing workflow with a voting step." They say "can we get AI to just handle the returns queue?" or "build me something that reads the contracts and tells me what's risky." The whole value you add as a lead is translation — hearing the vague ask and naming the concrete pattern hiding inside it, then pushing the design toward the simplest rung on the ladder that still clears the bar. Doing that well is worth more than knowing any single framework.
The trick is to listen for the verbal tell, because natural language leaks the control flow. "First do X, then Y" is someone describing an ordered pipeline — that's prompt chaining. "Handle each type differently" is categories wanting specialized handlers — routing. "Do all of these" or "be more sure about this" is independent work or a confidence problem — parallelization. "Break it up, but it depends on the input" is runtime decomposition — orchestrator-workers. "Keep improving it until it's good" is iterate-against-criteria — evaluator-optimizer. Only "just figure it out and handle it," with a path nobody can enumerate, points toward an actual agent — and even then you run the four-gate check from §9 before committing. The table below is that translation reflex written down; internalize it and you'll classify most requests in the meeting they're raised.
One warning that separates senior judgment from enthusiasm: the tech-lead move is not to reach for the most powerful pattern that could work. It's to reach for the simplest pattern that meets the bar, and to say "no" — clearly and early — to premature autonomy. A set of composed workflows you can test, reproduce, and cost out beats an impressive agent you can't debug at 2am. And remember these patterns nest: a mature system might route an input, chain the handling of it, run a guardrail check in parallel alongside, and escalate only the genuinely open-ended sliver to an agent. You are assembling composable pieces, not declaring the whole thing "an agent" from the outset — that composition, not the autonomy, is where the engineering maturity shows.
A lead's real skill here is naming the pattern behind a vague request and pushing the design down the complexity ladder. Most "build an agent" asks decompose into one or two workflow patterns that are cheaper, faster, and testable. Use this as the translation table:
Read it bottom to top, like climbing steps — and note that you want to climb as little as possible.
- The rungs stack the patterns from cheapest at the bottom (single call) to most powerful at the top (autonomous agent), matching §3–§9.
- The colors encode the workflow/agent line: green and blue rungs are workflows (you own control flow); the top amber rung is the agent (the model owns it).
- The amber up-arrow on the right is the payoff warning: cost and latency rise the higher you climb. So the green prompt at the bottom — take the lowest rung that meets the bar — is the actual decision rule.
In short: Don't reach for the most powerful pattern that could work; reach for the lowest rung that clears the bar, and say no to premature autonomy.
| You hear… | It's probably… | Because |
|---|---|---|
| "first do X then Y" | prompt chaining | fixed ordered steps with a check between |
| "handle each type differently" | routing | distinct categories, specialized handlers |
| "do all of these at once" / "be more sure" | parallelization | independent pieces, or voting for confidence |
| "break it up — but it depends on the input" | orchestrator-workers | the model decides the split at runtime |
| "keep improving it until it's good" | evaluator-optimizer | clear criteria + bounded iteration |
| "figure it out and just handle it" | agent (maybe) | open-ended — run the four-gate check first |
The tech-lead move is not to pick the most powerful pattern; it's to pick the simplest pattern that meets the bar, and to say no to premature autonomy. A composed set of workflows you can test beats an impressive agent you can't reproduce.
11 · Check yourself tech-lead
A teammate says: "We need an agent that summarizes each incoming document and then flags any that mention a competitor." Which pattern(s) does this actually call for, and is it an agent?
Show answer
Evaluator-optimizer has a loop, and an autonomous agent has a loop. Per Anthropic's framing, what actually distinguishes them?
Show answer
Exercise CS1.1 — Push a request down the ladder
Context: Every engineer has a "we should build an agent" idea in flight. The discipline is running the four-gate check on it honestly and pushing it down the complexity ladder before writing any code.
Your task: Take a real "build an agent" idea from your own work: run Anthropic's four-gate check, redesign it as a composition of the five workflow patterns if any gate is "no," and state the cheapest pattern that meets the bar.
Requirements:
- Answer each of the four gates — complexity, value, model capability, cost of error — for your idea
- If any gate is "no," redesign it as a composition of the five workflow patterns
- Sketch the control flow using the lesson's
step()stand-ins - State the cheapest pattern that still meets the bar
- If you keep the agent, justify which specific gate forced it
💡 Hint: A single "no" on any gate is enough to push the design back to composed workflows — find it before you commit.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: "We should build an agent for this" is where a lot of LLM projects go wrong. Per Anthropic's public "Building effective agents" guidance, autonomy is a cost you pay only when the path can't be predefined.
Your task: For each task, decide workflow (fixed code path) or agent (model directs its own steps): (a) translate a support email to French, (b) "fix this failing test in our repo."
Requirements:
- Classify (a) as a workflow — one call, fixed path, known inputs/outputs
- Explain that paying for autonomy on (a) only adds latency, cost, and failure surface
- Classify (b) as an agent — the steps (read code, run tests, edit, re-run) can't be predefined
- Note the agent case needs the model to decide based on tool results as it goes
- Ground the rule in Anthropic's public guidance: prefer the simplest thing that works
💡 Hint: Ask a single question of each task: can I write the steps down before I run?
Show solution
- (a) Translate an email: Workflow — one LLM call, fixed path, known inputs/outputs. No decisions to delegate, so paying for autonomy would only add latency, cost, and failure surface.
- (b) Fix a failing test: Agent — the number and nature of steps (read code, run tests, edit, re-run) can’t be predefined; the model must decide based on tool results.
Per Anthropic’s public “Building effective agents” guidance: prefer the simplest thing that works, and only add agentic autonomy when the task genuinely requires open-ended, model-directed steps.
Context: Most "build an agent" asks are really one or two named workflow patterns in disguise. Naming the pattern is what lets you reach for the cheapest thing that meets the bar.
Your task: Match each to one of the lesson's five patterns: (a) classify a ticket then send billing vs technical to different prompts; (b) draft a translation, then critique-and-improve it in a loop.
Requirements:
- Identify (a) as routing — a classifier directs input to one of several specialized prompts
- Explain why routing beats one mega-prompt trying to handle every type
- Identify (b) as evaluator-optimizer — one call generates, another evaluates, loop until it passes or caps
- State when evaluator-optimizer fits: clear criteria and iteration measurably helps
- Note that both are workflows — the control flow is fixed even though an LLM sits in each box
💡 Hint: The presence of an LLM in a step doesn't make the system an agent; who owns the control flow does.
Show solution
- (a) Classify then branch: Routing — a classifier directs the input to one of several specialized downstream prompts. Cheaper and more accurate than one mega-prompt trying to do all types.
- (b) Draft then critique-and-improve: Evaluator-optimizer — one call generates, another evaluates against criteria, and you loop until it passes or you hit a cap. Use it when quality has clear criteria and iteration measurably helps.
Both are workflows (fixed control flow), not agents — the path is predefined even though an LLM sits in each box.
Context: Some tasks can't have their subtasks enumerated up front — the number and shape depend on the input. Orchestrator-workers is the pattern that decomposes at runtime while keeping the workers bounded.
Your task: Design a "research a company" feature as orchestrator-workers: show what the orchestrator decides, what workers do, and the one guardrail that keeps cost bounded.
Requirements:
- The orchestrator decides the set of sub-tasks dynamically at runtime (e.g. funding, product, news)
- Workers each own one bounded sub-task and return results — not a free-running loop
- A synthesis step combines worker outputs and drops empties
- Explain why the dynamic split makes this orchestrator-workers, not plain parallelization
- Name the guardrail: cap the number of workers (and per-worker tool calls / token budget) the orchestrator may spawn
💡 Hint: The workers stay simple; the orchestrator holds the plan — and a cap on the fan-out is the core discipline.
Show solution
Orchestrator (LLM): given a company name, decides which sub-tasks are needed,
e.g. [funding history, product line, recent news, headcount].
| spawns (in parallel)
v
Worker 1: search + summarize funding
Worker 2: search + summarize product line
Worker 3: search + summarize recent news
| results returned
v
Orchestrator: synthesizes workers' outputs into one brief, drops empties.
Orchestrator decides the set of sub-tasks dynamically (that’s why it’s not plain parallelization); workers each own one bounded sub-task. Guardrail: cap the number of workers (and per-worker tool calls / token budget) the orchestrator may spawn, so a vague query can’t fan out into runaway cost. Bounding the loop is the core discipline of agentic systems.
Context: A PM wants a fully autonomous agent that "reads any user request and does whatever's needed across our internal tools." Full autonomy over internal tools is the most expensive, highest-blast-radius option.
Your task: Write the design review: when full autonomy is justified, and the cheaper alternative you'd propose first.
Requirements:
- State the stance: full autonomy is expensive, hard to test, and has a large blast radius
- Justify it only when the task space is genuinely open-ended, value per task is high, and you have observability + reversibility
- Propose the cheaper first design: routing + a small set of narrow workflows covering the top request types
- Instrument what falls through to "other" to find where an agent is actually warranted
- Gate any destructive tool call behind human approval even in the agent case
- Tie it to Anthropic's public guidance: add complexity only when simpler compositions demonstrably fall short
💡 Hint: Cover the 80% with cheap workflows first, then reserve autonomy for the instrumented long tail.
Show solution
Design-review stance: full autonomy over internal tools is the most expensive and highest-risk option — unpredictable cost, hard to test, and a large blast radius if a tool call goes wrong. Justify it only when: the task space is genuinely open-ended, the value per task is high, and you have observability + reversibility (dry-runs, human approval on risky actions, audit logs).
Cheaper first proposal: start with routing + a small set of narrow workflows covering the top few request types (they’re usually 80% of volume). Instrument what falls through to “other.” Only the genuinely unpredictable remainder justifies an agent, and even then, gate destructive tool calls behind human approval. This mirrors Anthropic’s public guidance: add complexity only when simpler compositions demonstrably fall short.
Context: A team needs a repeatable way to decide workflow-vs-agent for any new feature, not a one-off judgment call. The right artifact is a checklist biased toward simplicity that anyone can paste into a design doc.
Your task: Turn the lesson into a reusable checklist that decides workflow vs agent and which pattern, read top-to-bottom, first match wins.
Requirements:
- Lead with the decisive question: can you write the steps as fixed code today? If yes, workflow — stop
- Cover single-call (one transform), routing (known categories), and parallelization (independent known sub-tasks)
- Include evaluator-optimizer (quality improves with critique loops) with an iteration cap
- Include orchestrator-workers (the set of steps is data-dependent)
- End with agent only when the path is truly unpredictable AND value-per-task is high — with budget caps, approvals, audit
- Instruct the reader to take the first match; every rung down adds cost, latency, and failure surface
💡 Hint: Order the questions so the cheapest answer is reached first and autonomy is the last resort, not the default.
Show solution
| Question | If yes → |
|---|---|
| Can you write the steps as fixed code today? | Workflow. Stop here. |
| Is it one transform (translate/classify/extract)? | Single call. |
| Do inputs split into known categories? | Routing. |
| Are sub-tasks independent & known? | Parallelization. |
| Does quality improve with critique loops? | Evaluator-optimizer (with an iteration cap). |
| Is the set of steps itself data-dependent? | Orchestrator-workers. |
| Is the path truly unpredictable AND value-per-task high? | Agent — with budget caps, approvals, audit. |
Read top to bottom and take the first match. The checklist is biased toward simplicity on purpose: every rung down adds cost, latency, and failure surface.
Context: A representative team wants to "automate our invoice processing." The lead's job is to walk from that vague ask to a right-sized design — and to resist over-building where autonomy merely tempts.
Your task: Walk from the vague "automate invoice processing" ask to a right-sized design, showing where you resist over-building.
Requirements:
- Decompose the ask: extract fields, validate against a PO, flag exceptions, post approved ones
- Design extraction as a schema-constrained single call or small chain — predictable, so a workflow
- Use deterministic code for validation (numbers match the PO), not an LLM — cheaper and exact
- Route exceptions to a human queue with the model's explanation attached
- Resist the agent temptation (chasing missing PO numbers across systems) — cover known systems with a routed lookup first
- Conclude v1 is mostly workflow + one human-in-the-loop queue and no autonomous agent; add one only if the tail proves unpredictable
💡 Hint: Name each sub-step's cheapest sufficient pattern, and make the human queue the home for the messy remainder.
Show solution
- Decompose the vague ask. “Invoice processing” = extract fields → validate against a PO → flag exceptions → post approved ones. Three of four steps have fixed paths.
- Extraction: schema-constrained single call (or a small chain) — predictable, so a workflow.
- Validation: deterministic code (numbers match the PO), not an LLM — cheaper and exact.
- Exceptions: route mismatches to a human queue with the model’s explanation attached.
- Where autonomy tempts you: “let an agent chase down missing PO numbers across systems.” Resist first — most gaps hit a handful of known systems, so a small routed lookup covers them. Reserve an agent for the long tail, gated by approval before it acts.
Outcome: mostly workflow, one human-in-the-loop queue, and no autonomous agent in v1. Add the agent only if the exception tail proves genuinely unpredictable and worth the cost.
✓ Checkpoint — you can move on when you can…
- State the workflow-vs-agent distinction and why "workflow first" is Anthropic's default.
- Name all five workflow patterns and when each applies.
- Model each pattern's control flow in plain Python.
- Explain the cost / latency / reliability reasons autonomy is expensive.
- Run the four-gate check and decide — and defend — when a true agent is justified.