Debug a broken agent

Every lab below is a self-contained script you can run with python3. The agent loop is modelled with a scripted fake model — a tiny class that returns pre-set tool calls — so the failure is deterministic and reproducible without an API key. Read each broken run, form a hypothesis, then check it against the fix.

0 · The diagnosis decision tree advanced

You've almost certainly done this: the agent misbehaves, so you open the system prompt and start rewording it. You add "always finish your task", you add "use the right tool", you add "be careful with destructive actions" — and you re-run, and it fails a slightly different way, and you reword again. An hour later the prompt is a wall of pleading and the bug is still there. The reason this feels like whack-a-mole is that most agent failures are not prompt failures. They're failures in the scaffolding around the model — the loop, the tool schemas, the message wiring, the safety gate — and no amount of prompt prose fixes a broken loop.

Think of an agent the way a mechanic thinks about a car that won't start. The mechanic doesn't randomly swap parts; they follow a fixed sequence — fuel, spark, compression — because that order isolates the fault fastest. An agent has the same layered anatomy, and the failures have the same layered causes. There are really only four places a request can break on its way to a good answer, plus one place it can do harm: it can spin without ending (the loop), it can pick the wrong action (the tool schemas), it can fail to see what its own tools returned (the message wiring), it can outgrow the window (the context budget), or it can run something it never should have (the safety gate).

The discipline this section teaches is to stop guessing and instead walk those layers in one fixed order every single time, ruling each out with a single cheap check before moving on. A symptom maps to a layer: "it hangs" is the loop, "it did the wrong thing" is routing or the gate, "it acts like the tool never ran" is wiring, "it dies on long runs" is the budget. The common mistake — the one that costs the hour — is jumping straight to the layer you suspect instead of the layer the symptom points to, and skipping the check that would have cleared an earlier layer in seconds. The tree below is that checklist. Read it not as trivia but as the debugging order you'll internalize: left to right, first match wins, name the layer, then open that section.

Don't debug an agent by re-reading the prompt and hoping. Walk the failure down a fixed decision tree — the same order every time — until you can name the broken layer:

Read it left to right as a checklist. Looping? → check for a step cap and a no-progress guard (§1). Right tool but wrong choice? → the tool descriptions are vague or overlapping (§2). Calls a tool then acts like it never ran? → the result isn't being fed back with the right id (§3). Dies mid-run? → the transcript grew past the budget (§4). Did something destructive? → there was no gate between intent and action (§5). One symptom, one layer, one fix.

1 · Symptom: it never stops advanced

Here's the moment you'll remember: you launch the agent, glance away, and come back to a terminal still scrolling tool calls — or worse, a cloud bill that quietly climbed while it looped all night. An agent that never stops is the single most common way a first agent goes wrong, and it's frightening precisely because nothing looks broken. Each individual step is valid: the model asks for a tool, the tool runs, the result comes back. The bug is that the sequence has no ending, and a plain while loop will happily obey that forever.

To see why, remember what an agent loop actually is: it's a normal program loop where the "next instruction" is chosen by a model instead of written by you. In an ordinary loop you control the exit condition — while not done, and you know exactly when done flips. In an agent loop, whether the model ever emits a final answer instead of another tool call is outside your control. So you must supply the termination guarantee yourself, from the outside. That is the whole lesson of this section, and it splits into two distinct guards that people constantly conflate.

• A step cap is the seatbelt: for _ in range(max_steps) instead of while True. It guarantees the loop ends, full stop — even if the model behaves perfectly and just needs many steps, and even if it's hopelessly stuck. It bounds your worst-case cost. But a cap alone is a blunt instrument: an agent that spins uselessly still burns every one of those steps before it quits.
• A no-progress check is the smart guard: it watches for the model doing the same thing again — the identical tool with the identical arguments — because a repeated call on an unchanged world produces no new information. When the signature repeats, you're not making progress, you're spinning, and you should stop now, not at the cap.

The common mistake is shipping only the cap and calling it fixed. The cap makes the process terminate, so the hang disappears from your demo — but the underlying stuck loop is still there, now hidden behind "it eventually returned after 8 calls." A cap bounds the damage; the progress check bounds the waste, and it turns a silent budget-burn into an early, explainable stop you can log and alert on. You want both. Watch the scripted model below get deliberately stuck on one tool, and note how each guard changes the outcome.

The agent runs, calls a tool, calls it again, and again — the process pins a CPU and never returns. Two root causes, almost always: the loop has no hard step cap, and it has no no-progress check, so a model that keeps asking for the same thing spins forever. Here a scripted model is deliberately stuck on one tool:

loop_fix.py# A fake model: returns SCRIPTED tool calls, so the loop is deterministic.
class FakeModel:
    """Scripted stand-in for a real LLM. .respond() returns either a tool call
    or a final {'text':...}. If the script runs out it repeats the LAST step
    forever -- modelling a model that gets stuck."""
    def __init__(self, script):
        self.script, self.i = list(script), 0
    def respond(self, messages):
        step = self.script[min(self.i, len(self.script) - 1)]
        self.i += 1
        return step

def get_pods(namespace="staging"):
    return [{"name": "checkout-api", "status": "CrashLoopBackOff"}]
TOOLS = {"get_pods": get_pods}

STUCK = [{"tool": "get_pods", "input": {}}]   # model never answers, just re-asks

# BROKEN: `while True`, no cap, no progress check
def run_broken(model, budget=20):
    messages, calls = [{"role": "user", "content": "what's wrong?"}], 0
    while True:                       # no termination guarantee
        step = model.respond(messages)
        if "text" in step:
            return step["text"], calls
        calls += 1
        out = TOOLS[step["tool"]](**step["input"])
        messages.append({"role": "tool", "content": str(out)})
        if calls >= budget:           # only a test-harness safety net
            return "!! runaway: never terminated on its own", calls

# FIXED: hard step cap + no-progress detector
def run_fixed(model, max_steps=8):
    messages, seen = [{"role": "user", "content": "what's wrong?"}], set()
    for step_no in range(max_steps):          # (1) hard cap -- always terminates
        step = model.respond(messages)
        if "text" in step:
            return step["text"], step_no
        sig = (step["tool"], str(step["input"]))
        if sig in seen:                        # (2) repeated call = no progress
            return f"stopped: repeated {step['tool']} with no new info", step_no
        seen.add(sig)
        out = TOOLS[step["tool"]](**step["input"])
        messages.append({"role": "tool", "content": str(out)})
    return "stopped: hit step cap", max_steps

print("BROKEN:", run_broken(FakeModel(STUCK)))
print("FIXED :", run_fixed(FakeModel(STUCK)))

BROKEN: ('!! runaway: never terminated on its own', 20)
FIXED : ('stopped: repeated get_pods with no new info', 1)

The broken loop is a bare while True — it only stops because the test harness trips a runaway budget at 20 calls; on its own it would never terminate. The fixed loop adds two guards: a for _ in range(max_steps) hard cap that always terminates, and a seen set that detects the model re-issuing an identical call with no new information and stops at step 1. A cap alone bounds the damage; the progress check catches the loop early.

2 · Symptom: it calls the wrong tool advanced

You ask the agent to fetch the error logs, and it lists pods instead. Your instinct — everyone's instinct — is that the model is being dumb, so you march to the system prompt and add "when the user asks for logs, use the get_logs tool." It helps for one query, then breaks on the next phrasing. What's really happening is subtler and more fixable: the model didn't ignore your instruction, it followed the tool descriptions, and those descriptions were lying to it.

The key mental shift is this: a tool description is not documentation, it is instruction. When the model decides which tool to call, the tool names and descriptions are the primary signal it routes on — far more than a line buried in a long system prompt. The model is, in effect, matching the user's request against each tool's advertised job and picking the best fit. If two tools describe overlapping jobs, or one tool's description leaks a keyword that belongs to another ("get pod info including recent logs"), then a request for logs genuinely looks like a match for the pods tool. The routing is working correctly on bad inputs.

Picture a hardware store where two aisles are both signed "tools and supplies." A customer looking for paint wanders into the wrong one — not because they can't read, but because the signs don't distinguish. The fix isn't to lecture the customer; it's to rewrite the signs so each names a distinct thing and, ideally, when not to go there. Good tool descriptions do exactly that: each names one clear job and the trigger for reaching for it ("Use to read the actual error"), and no two read as substitutes. The strongest versions even state the boundary — "never call this to check status" — so a destructive tool can't be selected for a read.

The common mistake, then, is debugging routing at the prompt when the bug lives in the schema. Before you touch the system prompt, diff the tool descriptions: hunt for overlapping verbs ("get info about…", "get data for…"), leaked keywords, and vague scope. In the lab below nothing about the loop or the model changes between the broken and fixed runs — only the description text does, and that alone flips the routing from wrong to right.

The loop is fine; the agent just chooses badly — asked for logs, it lists pods instead. The reflex is to rewrite the system prompt. The real fix is usually one layer down: the model routes on the tool descriptions, and vague or overlapping descriptions make two tools look interchangeable. We model routing the way a model does it — scoring the request against each tool's description:

route_fix.pyimport re
def words(s):
    return set(re.findall(r"[a-z]+", s.lower()))

def route(request, tools):
    """tools: {name: description}. The model picks the tool whose DESCRIPTION
    best overlaps the request -- so the description IS the routing logic."""
    req = words(request)
    scored = sorted(tools.items(),
                    key=lambda kv: (len(req & words(kv[1])), kv[0]),
                    reverse=True)
    return scored[0][0]

REQUEST = "show me the recent error logs for the checkout pod"

# BROKEN: vague, overlapping descriptions; get_pods even leaks the word 'logs'
VAGUE = {
    "get_pods": "get pod info including recent logs and status",
    "get_logs": "get data for a pod",
}
print("BROKEN routes to:", route(REQUEST, VAGUE))   # wrong: get_pods steals 'logs'

# FIXED: sharp, disjoint descriptions that say WHEN to use each tool
SHARP = {
    "get_pods": "List pods and their status. Use to see WHICH pods are unhealthy.",
    "get_logs": "Fetch recent error log lines for one pod. Use to read the actual error.",
}
print("FIXED routes to :", route(REQUEST, SHARP))

BROKEN routes to: get_pods
FIXED routes to : get_logs

In the broken registry both descriptions are mushy and get_pods even leaks the word "logs" — so a request for logs routes to the pods tool. Nothing about the loop or the model changed in the fix: only the descriptions did. Sharp, disjoint descriptions that say when to use each tool ("Use to read the actual error") send the request to the right place. The tool description is part of the model's instructions, not documentation.

3 · Symptom: it ignores the tool result expert

This one drives engineers to the edge, because your own eyes contradict the agent. You watch the tool run. You see it print the perfect answer to your logs. And the model's very next turn asks for that exact same tool again — or shrugs and answers as if it learned nothing. It looks like the model is stubbornly ignoring reality, and that framing sends people down the wrong road: they add prompt lines like "pay attention to the tool result." That never works, because the model isn't ignoring the result. It never received it.

To debug this you have to hold one fact firmly: the model has no memory and no access to your program's variables — it can only see the transcript you send it. Your Python function returned a value, your logs printed it, but none of that reaches the model unless you explicitly append that result into the messages list before the next call. If the append is missing, the model's view of the world is unchanged from before the tool ran, so of course it re-issues the same call. From the model's side, the tool never happened.

There's a second, sneakier version. In tool-use protocols each tool call the model emits carries a unique id (a tool_use_id), and the result you feed back must be tagged with that same id so the model can match answer to question. Think of it like a coat check: you hand over your coat and get ticket #42; when you come back, a coat tagged #999 is useless to you even though it's a real coat on the rack. If the result is appended with the wrong id, the model scans the transcript for a result matching the request it made, finds none, and re-asks. Both bugs — a missing append and a mismatched id — present identically as "it ignores the tool result," which is why you should always check the wiring before you blame the model.

The durable fix is three small rules that together keep the loop honest: append the assistant's turn to the transcript verbatim, pair every tool result with the exact tool_use_id of the call it answers, and return all of a turn's results in one user message. Get those right and the "ignoring" symptom vanishes — as the lab shows, a one-line id fix takes the agent from giving up after five wasted steps to naming the root cause on step one.

The agent calls the tool, the tool clearly returns the answer — and the model asks for the same tool again, or answers blind. The model can only use what's in the transcript, so this is almost always a wiring bug: the result was never appended, or it was appended with the wrong tool_use_id, so the model can't match the result to the request it made. Here the result is fed back with a mismatched id:

result_fix.pyclass FakeModel:
    """Requests get_logs, then looks for a tool_result carrying the SAME id it
    used. If it can't find it, it re-requests -- the 'ignoring' symptom."""
    def __init__(self):
        self.pending_id = None
    def respond(self, messages):
        if self.pending_id is None:
            self.pending_id = "call_1"
            return {"tool": "get_logs", "input": {"pod": "checkout"}, "id": "call_1"}
        for m in messages:
            if m.get("role") == "tool" and m.get("tool_use_id") == self.pending_id:
                return {"text": f"Root cause found in logs: {m['content']}"}
        return {"tool": "get_logs", "input": {"pod": "checkout"}, "id": "call_1"}

def get_logs(pod):
    return "ERROR: password authentication failed for user 'checkout'"

def run(model, wire_result_correctly, max_steps=5):
    messages = [{"role": "user", "content": "why is checkout down?"}]
    for step_no in range(max_steps):
        step = model.respond(messages)
        if "text" in step:
            return step["text"], step_no
        result = get_logs(**step["input"])
        if wire_result_correctly:
            # correct: pair the result with the SAME id the model used
            messages.append({"role": "tool", "tool_use_id": step["id"], "content": result})
        else:
            # bug: mismatched id -> the model can never find its own result
            messages.append({"role": "tool", "tool_use_id": "wrong_id", "content": result})
    return "!! gave up: never saw its own tool result", max_steps

print("BROKEN:", run(FakeModel(), wire_result_correctly=False))
print("FIXED :", run(FakeModel(), wire_result_correctly=True))

BROKEN: ('!! gave up: never saw its own tool result', 5)
FIXED : ("Root cause found in logs: ERROR: password authentication failed for user 'checkout'", 1)

The scripted model requests get_logs with id call_1, then looks for a tool_result carrying that id. In the broken run the loop tags the result wrong_id, the model never finds its answer, and it re-asks until it gives up. The fix is one line: append the result with tool_use_id = step["id"] — the same id the model used. Now it finds the logs on the next pass and reports the root cause. The three loop rules are: append the assistant turn verbatim, pair every result with its tool_use_id, and return all results in one user turn.

4 · Symptom: it runs out of context / budget expert

This is the bug that passes every test you write and then detonates in production. Your short demos work. Your unit tests, which run two or three steps, are green. Then a real incident comes in, the agent works it for twenty steps, and on step twenty it dies with a context-length error — or the cost graph spikes because that final request re-sent nineteen steps of stale logs. The cruel part is the timing: it fails on the long run, which is exactly the run that mattered most.

The cause is structural, and once you see it you can't unsee it. Every pass of the loop appends two things to the transcript — the model's turn and the tool's result — and nothing is ever removed. So the transcript grows monotonically. Since you re-send the entire transcript on every call (that's how the model gets its "memory"), the request on step 20 carries steps 1 through 19 along with it, most of which are long-dead log dumps the model no longer needs. You're paying to re-transmit history, and eventually that history overflows the fixed context window.

The reframe that fixes it: the transcript is a buffer you manage, not a log you let grow. A log is append-only by definition; a working buffer has a budget and you decide what stays in it. The standard technique is a rolling window with a summary — keep the first message (the task, which anchors everything) and the last few turns verbatim (the recent state the model is actively reasoning over), and collapse everything in the middle into a short summary. That keeps the two things that matter — the goal and the present — at full fidelity while the stale middle is compressed to a line.

• Keep the head: the original task/system message. Drop it and the agent forgets what it's doing.
• Keep the tail: the last few turns, verbatim — this is the live working set the model needs to make its next move.
• Summarize the middle: in production you'd summarize with a cheap model; the lab uses a stub string, but the shape is identical — the middle shrinks to one line no matter how long the run.

The common mistake is treating unbounded history as a style nit to clean up "later." It isn't a nit; it's a latent bug that hides through every short test and surfaces only under load. Bound the transcript from day one — a window plus a running summary — and, as the lab shows, a run that overflowed at step 4 finishes all 12 steps with its peak comfortably under budget.

The agent works for a few steps, then a long run dies with a context-length error (or the per-request cost balloons). The cause is structural: every pass appends the model turn and the tool result, so the transcript grows without bound. On step 20 you're re-sending 19 steps of stale logs. The fix is to bound the history — keep the task and the recent turns, summarize the middle:

budget_fix.pyWINDOW = 200   # pretend token budget for the whole request

def est_tokens(messages):
    return sum(len(str(m["content"])) for m in messages) // 4  # ~1 tok / 4 chars

def big_tool_result(step):
    return f"log page {step}: " + "x" * 120

# BROKEN: append everything, forever
def run_broken(steps=12):
    messages = [{"role": "user", "content": "diagnose the incident " * 3}]
    for s in range(steps):
        messages.append({"role": "assistant", "content": f"calling get_logs (step {s})"})
        messages.append({"role": "tool", "content": big_tool_result(s)})
        if est_tokens(messages) > WINDOW:
            return f"!! context overflow at step {s}: {est_tokens(messages)} > {WINDOW} tok"
    return "finished"

print("BROKEN:", run_broken())

# FIXED: keep the task + recent turns, summarize the middle
def trim(messages, keep_recent=4):
    if len(messages) <= keep_recent + 1:
        return messages
    head, middle, tail = messages[:1], messages[1:-keep_recent], messages[-keep_recent:]
    summary = {"role": "assistant",
               "content": f"[summary of {len(middle)} earlier turns omitted]"}
    return head + [summary] + tail

def run_fixed(steps=12):
    messages, peak = [{"role": "user", "content": "diagnose the incident " * 3}], 0
    for s in range(steps):
        messages.append({"role": "assistant", "content": f"calling get_logs (step {s})"})
        messages.append({"role": "tool", "content": big_tool_result(s)})
        messages = trim(messages)                 # bound the history every pass
        peak = max(peak, est_tokens(messages))
    return f"finished all {steps} steps; peak {peak} tok stayed under {WINDOW}"

print("FIXED :", run_fixed())

BROKEN: !! context overflow at step 4: 212 > 200 tok
FIXED : finished all 12 steps; peak 105 tok stayed under 200

The broken loop overflows a 200-token pretend window at step 4 because nothing is ever removed. The fix keeps the first (task) message and the last few turns verbatim and collapses everything in between into a one-line summary, so the transcript stays bounded — peak 105 tokens across all 12 steps. Real systems summarize with a cheap model instead of a stub string, but the shape is identical: the transcript is a buffer you manage, not a log you let grow.

5 · Symptom: it takes an unsafe action tech-lead

Every failure so far has cost you time or money. This one can cost you a production system. A confused agent, reaching for anything that might help, calls the destructive tool — delete_pod, cancel_order, send_email to 400 customers — and the loop, dutifully, just runs it. There is no undo. This is the failure that keeps agents out of production, and it's the one that separates a demo from a system you'd trust near real infrastructure.

The seductive wrong answer is to handle it in the prompt: "you are not allowed to delete anything without permission." Internalize why that is not a control at all. A prompt instruction is a request to a probabilistic system — the model complies most of the time, which is exactly the trap, because "most of the time" is not a safety property. A model that's confused, jailbroken, or simply misreading an ambiguous instruction will ask for the destructive tool anyway, and a prompt cannot stop it because the prompt is advice, not a barrier. Asking the model to police itself is like asking a driver to promise not to speed instead of installing a speed limiter.

The only real control is a gate in code — a hard check that sits between "the model wants this tool" and "the tool actually runs," outside the model's influence entirely. And the gate shouldn't be a single yes/no; it should be a small decision table keyed on two things:

• The tool's risk class — is it read-only (safe to run anytime), reversible (a restart you can undo), or irreversible (a delete you can't)? Classify every tool at design time.
• The agent's autonomy rung — how much freedom this agent is granted right now: merely observe, allowed to act reversibly, or fully autonomous. Risk × rung yields a verdict: allow, ask (require human confirmation), or block.

The design invariant to burn in: irreversible actions are blocked at every rung by default. That single rule, enforced in a table you can test without the model, is what makes an agent safe near real systems. Note too that a blocked verdict isn't a dead end — it's structured data the loop can hand back to the model ("that action was refused"), so the agent can adapt and propose a safe alternative (open a PR instead of deleting) rather than failing silently. The lab below runs the unsafe delete with no gate, then shows the same request refused the instant a gate is in place.

The most dangerous failure: the agent, when confused, reaches for a destructive tool and the loop just runs it. "I told the model in the prompt not to delete things" is not a control — a hijacked or simply mistaken model will ask anyway. The only real control is a gate in code between "model wants tool" and "tool runs", keyed on the tool's risk class and the agent's autonomy rung:

gate_fix.pyRISK = {"get_pods": "read_only", "restart": "reversible", "delete_pod": "irreversible"}

# scripted model: investigates once, then (wrongly) tries to delete the pod
SCRIPT = [
    {"tool": "get_pods", "input": {}},
    {"tool": "delete_pod", "input": {"name": "checkout-api"}},
    {"text": "done"},
]

def run_pod_tool(tool, args):
    return f"EXECUTED {tool}({args})"

# BROKEN: no gate -- runs every requested tool
def run_broken():
    performed = []
    for step in SCRIPT:
        if "text" in step:
            break
        run_pod_tool(step["tool"], step["input"])       # runs unconditionally
        performed.append(step["tool"])
    return performed

print("BROKEN performed:", run_broken())   # includes the irreversible delete!

# FIXED: gate(risk, rung) -> allow / ask / block
MATRIX = {   # rung -> {risk: verdict}
    "observe": {"read_only": "allow", "reversible": "block", "irreversible": "block"},
    "act":     {"read_only": "allow", "reversible": "ask",   "irreversible": "block"},
}
def gate(tool, rung):
    return MATRIX[rung][RISK[tool]]

def run_fixed(rung="observe", approve=lambda t: False):
    audit = []
    for step in SCRIPT:
        if "text" in step:
            break
        verdict = gate(step["tool"], rung)
        allowed = verdict == "allow" or (verdict == "ask" and approve(step["tool"]))
        if allowed:
            run_pod_tool(step["tool"], step["input"])
        audit.append(f"{'OK ' if allowed else 'NO '} {step['tool']} [{RISK[step['tool']]}] -> {verdict}")
    return audit

print("FIXED audit:")
for line in run_fixed():
    print("  ", line)

BROKEN performed: ['get_pods', 'delete_pod']
FIXED audit:
   OK  get_pods [read_only] -> allow
   NO  delete_pod [irreversible] -> block

The broken loop has no gate, so when the scripted model asks to delete_pod it runs — an irreversible action, no questions asked. The fixed loop routes every call through gate(tool, rung), a pure lookup of (risk × rung) → allow / ask / block. At the OBSERVE rung read-only is allowed and the irreversible delete is blocked, recorded in the audit trail as NO. Note the blocked verdict is data the loop can hand back to the model, so it can adapt (propose a PR) instead of failing silently.

6 · Grade your agent debugging tech-lead

Here's the difference that gets people promoted, and it shows up in how they close a bug. A competent engineer fixes the agent in front of them: they add the step cap, sharpen the description, pair the id, and the ticket closes. A lead does that and asks a second question — "how do I make sure this class of bug can never come back?" — because they've learned that the same failure recurs in the next agent, and the one after that, unless the fix is turned into a constraint the codebase enforces.

That second move is what turns each section of this challenge from a one-off repair into a durable invariant. A step cap is a fix; a cap plus a no-progress detector, with a test asserting a stuck model stops within two steps, is an invariant. A rewritten description is a fix; making all descriptions disjoint and adding a test that asserts request→tool routing is an invariant. A corrected tool_use_id is a fix; routing every tool result through one code path so ids can't drift again is an invariant. The pattern is always the same — close the bug class, not just the ticket — and the deepest version of it is a regression test aimed at the deterministic scaffolding (the cap, the gate, the id pairing), because that logic runs with no API key and the model can't argue its way past a mechanical check. Score your own diagnosis against the bar below, and read the right-hand column as the habit to build toward.

Score your diagnosis of any broken agent against this bar. "Meets" is a correct fix; "above bar" is what a lead does — they fix the instance and close the class of bug so it can't recur.