AI EngineeringZero to ProductionHome·About·Contact
Part V · Build Lab B

The Tool Registry & the Agent Loop

Now you build the working brain: a registry of tools (each tagged with a risk class) and the agentic loop that lets Claude call them. By the end you'll have a real read-only diagnostician that inspects the mock cluster and explains what's wrong — your first end-to-end run.

⏱️ ~60 min🔑 API key needed from Step 4✅ test cases included
⚙️ To run this for realThe code here is complete and correct as written. To actually execute it you'll need:
  • an Anthropic API key (ANTHROPIC_API_KEY) + pip install anthropic
Reading and learning works without any of this — run when you're ready. IDs/ARNs in examples are placeholders; swap in your own.

Learning objectives

  • Model a tool as data: schema + risk class + implementation, in one registry.
  • Reuse the Chapter 4 agent loop, now driven by the registry.
  • Run a real diagnosis against the mock cluster and read the audit trail.
  • Test the loop's behavior deterministically.

Where this fits intermediate

In Lab 8a everything ran offline. This lab makes the first API calls — the loop sends the mock cluster's data to Claude, which reasons about it and calls read-only tools to investigate. Still 100% safe: only read-only tools are wired up here; the safety gate and write actions come in Lab 8c.

Step 1 · Model a tool as data intermediate

A tool bundles three things: the schema Claude sees, the risk class (from Lab 8a), and the Python function that runs it. Keeping them together in one object is what lets the safety gate later look up any tool's risk in one place.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Lab 8b · Step 1
agent/tools.pyfrom dataclasses import dataclass
from typing import Callable
from .schemas import RiskClass
from mock import cluster, actions

@dataclass
class Tool:
    name: str
    description: str
    input_schema: dict
    risk: RiskClass                      # ← the safety-critical field
    run: Callable

    def spec(self) -> dict:
        # the shape Claude's tools=[...] parameter wants
        return {"name": self.name, "description": self.description,
                "input_schema": self.input_schema}

TOOLS: dict[str, Tool] = {}
def register(tool): TOOLS[tool.name] = tool

def _obj(props, required):
    return {"type":"object", "properties": props, "required": required}
▶ How this works

Before an AI model can use a tool, we have to describe that tool in a way both our program and the model understand. This block defines a single container — a Tool — that packs everything about one tool into one object: what it's called, what it does, what inputs it takes, how dangerous it is, and the actual code that runs it. A @dataclass is just Python's shortcut for "a simple record with named fields".

  1. The five lines under class Tool: are the fields every tool must have: name (short id like "kubectl_logs"), description (plain-English text the model reads to decide when to use it), input_schema (what arguments it accepts), risk (how dangerous — the field the safety gate will check later), and run (the real Python function to execute).
  2. risk: RiskClass is called out with a comment as the safety-critical field. Keeping the risk level glued to the tool means later code can look up "how dangerous is this tool?" in one place — you can never accidentally run a tool without knowing its risk.
  3. def spec(self) returns a small dictionary with just name, description, and input_schema. That is exactly the shape the Anthropic API's tools=[...] parameter expects — so spec() is how we hand a tool to the model without leaking our internal risk and run fields.
  4. TOOLS: dict[str, Tool] = {} is an empty registry — a phone book that maps each tool's name to its Tool object. register(tool) simply adds a tool to that book. _obj(...) is a tiny helper that builds the standard "object with these properties" JSON schema, so we don't repeat it for every tool.

What the output means: Nothing prints — this file only defines the shape of a tool and an empty registry. It's the foundation the next steps fill in.

Try this: Ask yourself: which field does the model actually see, and which stays private to your program? (Hint: only the three fields returned by spec() go to the model; risk and run never leave your machine.)

Step 2 · Register the read-only tools intermediate

Give the agent eyes before hands. Each read-only tool wraps a mock-cluster accessor and is tagged READ_ONLY. Note how the description tells the model when to use it — that's what makes the agent investigate in a sensible order.

Lab 8b · Step 2
agent/tools.py (continued)register(Tool(
    "kubectl_get_pods",
    "List pods with status and restart counts. Use FIRST when "
    "diagnosing — it shows what's unhealthy.",
    _obj({"namespace": {"type":"string"}}, []),
    RiskClass.READ_ONLY,
    lambda namespace="staging": cluster.get_pods(namespace),
))
register(Tool(
    "kubectl_logs",
    "Fetch recent log lines for a pod. Use to find the actual error.",
    _obj({"pod": {"type":"string"}, "namespace": {"type":"string"}}, ["pod"]),
    RiskClass.READ_ONLY,
    lambda pod, namespace="staging": cluster.get_logs(pod, namespace),
))
# also: kubectl_get_events, recent_deploys — same pattern
▶ How this works

Here we actually create and register two real tools. Each register(Tool(...)) call fills in the five fields from Step 1 for one tool and drops it into the TOOLS registry. These are read-only tools — they only look at the mock cluster, they never change anything — so they are completely safe to let the model call.

  1. The first tool is named "kubectl_get_pods". Read its description: "List pods with status and restart counts. Use FIRST when diagnosing…". That sentence is written for the model to read — the words "Use FIRST" nudge the model to check pod health before anything else. A good description is how you steer an agent's behaviour.
  2. _obj({"namespace": {"type":"string"}}, []) is the input schema: it says "this tool accepts one optional argument called namespace that is a string". The empty list [] means no argument is required.
  3. RiskClass.READ_ONLY tags the tool as safe. The lambda on the next line is the actual code that runs when the model calls the tool — here it just forwards the request to cluster.get_pods(namespace) and defaults namespace to "staging".
  4. The second tool, "kubectl_logs", follows the identical recipe but requires a pod argument (note ["pod"] in the schema). The trailing comment says the other tools use the same pattern — once you understand one register(Tool(...)) block, you understand them all.

What the output means: Again nothing prints; each call quietly adds one tool to the TOOLS registry. After this file runs, TOOLS holds several named, described, risk-tagged tools ready for the model to choose from.

Try this: Reword kubectl_get_pods's description to be vague (e.g. just "lists pods") and imagine the effect: with weaker wording the model is less likely to call it first. The description is the instruction manual you give the model.

Reuse from Chapter 3Later you'll add a search_docs tool that calls your Chapter 3 retriever — turning RAG into an agent tool. In Lab 8c we ground the agent a simpler way (runbook injected into the system prompt), but the tool approach works too. Same building blocks, composed differently.
🖥️ Add monitoring tools too — all read-onlyThe starter kit also registers monitoring tools against a mock Grafana/Kibana/CloudWatch stack (mock/monitoring.py): firing_alerts, get_metric, list_dashboards, get_dashboard, search_logs. They follow the exact same register(Tool(...)) pattern and are all RiskClass.READ_ONLY — so the agent can triage a "something's wrong in prod" page (start from alerts → confirm with metrics → search logs) while staying at the safe OBSERVE rung. Try: python cli.py "something is wrong with checkout in staging — investigate".

Step 3 · The agent loop advanced

This is the Chapter 4 loop, driven by the registry. For now it runs every requested tool directly (all are read-only, so it's safe). In Lab 8c you'll insert the policy gate between "model wants tool" and "tool runs".

Lab 8b · Step 3
agent/engine.py (read-only version)from anthropic import Anthropic
from agent.tools import TOOLS
client = Anthropic()
MAX_STEPS = 8

def run(incident: str):
    system = ("You are a careful SRE. Diagnose the incident using the "
              "read-only tools. Cite the evidence you find.")
    messages = [{"role":"user", "content": incident}]
    specs = [t.spec() for t in TOOLS.values()]

    for _ in range(MAX_STEPS):                # hard cap — always terminates
        resp = client.messages.create(model="claude-opus-4-8",
                 max_tokens=1500, system=system, tools=specs, messages=messages)
        if resp.stop_reason == "end_turn":
            return next((b.text for b in resp.content if b.type=="text"), "")
        messages.append({"role":"assistant", "content": resp.content})
        results = []
        for b in resp.content:
            if b.type == "tool_use":
                out = TOOLS[b.name].run(**b.input)     # Lab 8c gates this
                results.append({"type":"tool_result",
                                "tool_use_id": b.id, "content": str(out)})
        messages.append({"role":"user", "content": results})
    return "Stopped: hit step limit."
▶ How this works

This is the agent loop — the beating heart of the whole capstone. An "agent" is just a loop that: (1) asks the model what to do, (2) if the model asks to use a tool, runs that tool, (3) feeds the tool's result back to the model, and (4) repeats until the model says it's finished. The model never touches your systems directly — your loop runs the tools on its behalf.

  1. MAX_STEPS = 8 and for _ in range(MAX_STEPS): set a hard cap of 8 passes. The comment says always terminates — this guarantees the loop can never run forever, even if the model keeps asking for more tools. A step cap is a basic safety net for every agent.
  2. messages is the running conversation. It starts with the user's incident. Each pass calls client.messages.create(...), sending the system instructions, the tools=specs list (from Step 2's registry), and the whole conversation so far. The model reads it all and replies.
  3. if resp.stop_reason == "end_turn": means the model is done — it has no more tools to call and just wants to answer. We pull out its text reply and return it, ending the loop.
  4. Otherwise the model asked to use one or more tools. We first append the model's turn to messages verbatim, then loop over resp.content: for each block where b.type == "tool_use", we run the matching tool with TOOLS[b.name].run(**b.input) and collect the result.
  5. Each result is packaged as a tool_result carrying the same b.id (its tool_use_id) so the model knows which request it answers. All results go back in one user message (messages.append(...)), and the loop repeats. If we ever exhaust all 8 steps, the final return reports it hit the step limit.

What the output means: When run, this returns Claude's final diagnosis as plain text — after it has silently called whatever read-only tools it needed along the way. If the model gets stuck looping, you'd instead see "Stopped: hit step limit."

Try this: Trace one full lap in your head: user asks → model says "call kubectl_get_pods" → loop runs it → result appended → model reads result and either answers (end_turn) or asks for another tool. That call-run-feedback cycle is what makes it an agent.

The three loop rules from Chapter 4 still applyAppend the assistant turn verbatim before results · every tool_result carries its tool_use_id · all results in one user message. The finished engine.py in the starter kit already includes the Lab 8c additions (gate + audit) — this excerpt is the read-only core.

Step 4 · Run your first diagnosis advanced

Now you need your API key (from the course .env). A quick runner:

Lab 8b · Step 4
terminalcd devops-agent
python cli.py "checkout-api-7d9f in staging keeps restarting"

Expected (the model investigates, then concludes):

🔧 Running at rung: OBSERVE
--------------------------------------------------
The pod checkout-api-7d9f is in CrashLoopBackOff. Its logs show
"password authentication failed for user 'checkout'", and a deploy
18 minutes ago changed the DATABASE_URL secret reference. The likely
root cause is a broken database credential from that change. Suggested
fix: correct the secret and roll the deployment — do not just restart.
--------------------------------------------------
📋 Audit log (every action attempted):
✓ kubectl_get_pods({'namespace': 'staging'}) [read_only]
✓ kubectl_logs({'pod': 'checkout-api-7d9f'}) [read_only]
✓ recent_deploys({'namespace': 'staging'}) [read_only]

That's a real agent: it chose which tools to call, in what order, connected the log error to the recent deploy, and never changed anything.

▶ How this works

Now you actually run the agent from a terminal. This is the first time the code talks to the real model over the internet, so an API key is needed from here on. You type the incident in plain English and let the agent investigate.

  1. cd devops-agent moves into the project folder. python cli.py "checkout-api-7d9f in staging keeps restarting" launches the program and passes your incident description as the input the loop will send to the model.
  2. You did not tell it which tools to run or in what order. The agent decides that itself using the tool descriptions from Step 2 — that is the whole point of an agent.

What the output means: The OBSERVE rung line confirms it stayed at the safe, read-only level. The paragraph is the model's diagnosis: it connected the crash-looping pod, the "password authentication failed" log line, and a recent deploy that changed the database secret — then suggested a fix without touching anything. The audit log below lists every tool it called (all [read_only]), giving you a full trail of what the agent did.

Try this: Read the three audit lines in order — pods, then logs, then deploys. That ordering came straight from the "Use FIRST" wording in the tool descriptions. Change the incident text and watch which tools the agent chooses to call.

Step 5 · Test the loop expert

Two kinds of test. The registry test needs no API key; the diagnosis test does (it runs the model), so keep those separate so CI can run the free ones always.

Lab 8b · Step 5
tests/test_mock_and_tools.py (add)from agent.tools import TOOLS
from agent.schemas import RiskClass

def test_every_tool_has_a_risk_class():          # no API key
    for name, tool in TOOLS.items():
        assert isinstance(tool.risk, RiskClass), f"{name} missing risk"

def test_readonly_tool_runs_and_returns_data():   # no API key
    pods = TOOLS["kubectl_get_pods"].run(namespace="staging")
    assert any(p["status"] == "CrashLoopBackOff" for p in pods)
▶ How this works

Good agents come with tests. The clever move here is splitting tests into ones that need no API key (fast, free, always run) and ones that call the model (slower, cost money). These two tests are the free kind — they check the registry and tool wiring without ever contacting Claude.

  1. test_every_tool_has_a_risk_class loops over every tool in the TOOLS registry and asserts that its risk really is a RiskClass. The message f"{name} missing risk" is what prints if one is missing — so no tool can ever sneak into the registry untagged and later slip past the safety gate.
  2. test_readonly_tool_runs_and_returns_data actually calls the kubectl_get_pods tool and checks the returned pods include one in "CrashLoopBackOff". This proves the registry is correctly wired to the mock cluster — the tool runs and gives back real data.
  3. An assert is a test's way of saying "this must be true". If it is, the test passes silently; if not, the test fails and shows your message — that's how you know something broke.

What the output means: Run with pytest, both tests pass silently (no output means success). Because neither talks to the model, they run in a fraction of a second and cost nothing — safe to run on every commit in CI.

Try this: Temporarily change a tool's risk to a plain string like "safe" instead of a RiskClass, and rerun — the first test will fail and print … missing risk, showing you exactly how the guard catches mistakes.

✅ Test cases for Lab 8b
TestNeeds API key?Proves
every tool has a risk classNoNothing can slip past the gate untagged
read-only tool returns dataNoRegistry ↔ mock wiring works
diagnosis mentions "database" (Lab 8d eval)YesThe loop actually reasons correctly

Troubleshooting expert

⚠️ Common issues & fixes
SymptomCauseFix
Agent answers without calling any toolTool descriptions too vague, or incident too genericMake descriptions say when to use the tool; name the pod in the incident
Infinite-ish loop / hits step limitModel keeps re-calling toolsConfirm MAX_STEPS cap is present; check you append the assistant turn before results
400 tool_use_id ... without tool_resultA requested tool result wasn't returnedReturn one tool_result per tool_use block, all in one user message
KeyError in TOOLS[b.name]Model called a tool name not in the registryOnly pass spec() for registered tools; the model can only call what you send
AuthenticationErrorNo/invalid API keycp ../.env.example ../.env and paste your key; ensure venv active
Answer ignores the recent deployModel didn't call recent_deploysStrengthen that tool's description; or lower the incident ambiguity. Lab 8c's runbook grounding also helps

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Model a tool as a dataclass with a private risk fieldBeginner

Context: A tool the model can call and a tool the gate can police are two different views of the same object. The Tool dataclass keeps the risk and the real function private from the model on purpose.

Your task: Reproduce the Tool dataclass from agent/tools.py, including its spec() method, and explain which fields spec() deliberately omits.

Requirements:

  • Fields for name, description, input schema, risk class, and the callable to run
  • spec() returns only name, description and input schema — the shape the API's tools=[...] expects
  • risk and run are excluded from spec()
  • Explain that the model never learns the risk the gate checks, nor holds the real function
  • Pure definition — no API key

💡 Hint: The model only needs to know a tool exists and how to call it; the risk and the implementation are the harness's business, not the model's.

Show solution
from dataclasses import dataclass
from typing import Callable
from agent.schemas import RiskClass

@dataclass
class Tool:
    name: str
    description: str
    input_schema: dict
    risk: RiskClass                      # ← the safety-critical field
    run: Callable

    def spec(self) -> dict:
        # the shape Claude's tools=[...] parameter wants
        return {"name": self.name, "description": self.description,
                "input_schema": self.input_schema}

One Tool bundles everything about a tool: the name/description/input_schema the model sees, plus risk (checked by the gate) and run (the real Python callable). spec() returns only the first three keys — exactly what the Anthropic tools=[...] parameter expects — so risk and run stay private to your program. The model never learns the risk tag it might argue around, nor holds the real function. This is a pure definition; no API key needed.

Exercise 2 · Build the registry and register a read-only toolIntermediate

Context: Tools live in a central registry so the loop, the gate and the tests all see the same set. Registering the first read-only tool wires the registry to the mock cluster.

Your task: Add the TOOLS registry, the register() helper and the _obj() schema helper, then register the kubectl_get_pods tool as the lesson does.

Requirements:

  • TOOLS is a name→Tool dict; register() inserts into it
  • _obj(props, required) builds a JSON-schema object with the given properties and required list
  • Register kubectl_get_pods as RiskClass.READ_ONLY, backed by the mock's get_pods
  • Its description nudges the model to investigate first (e.g. "Use FIRST")
  • An empty required list means no mandatory arguments

💡 Hint: The description is written for the model's benefit — wording like "Use FIRST" shapes the order in which it reaches for tools.

Show solution
from mock import cluster
from agent.schemas import RiskClass
from agent.tools import Tool   # (or same module)

TOOLS: dict[str, Tool] = {}
def register(tool): TOOLS[tool.name] = tool

def _obj(props, required):
    return {"type": "object", "properties": props, "required": required}

register(Tool(
    "kubectl_get_pods",
    "List pods with status and restart counts. Use FIRST when "
    "diagnosing — it shows what's unhealthy.",
    _obj({"namespace": {"type": "string"}}, []),
    RiskClass.READ_ONLY,
    lambda namespace="staging": cluster.get_pods(namespace),
))

TOOLS is a name → Tool phone book, and register() drops one in. _obj({"namespace": {"type":"string"}}, []) builds the JSON Schema: one optional string argument (the empty [] means nothing is required). The description is written for the model — the words "Use FIRST" nudge it to check pod health before anything else, which is how sensible investigation ordering emerges. The tool is tagged RiskClass.READ_ONLY and its lambda forwards to the mock accessor. Runnable pure Python; no API key.

Exercise 3 · Register kubectl_logs with a required argumentAdvanced

Context: Some tools can't run without an argument. Marking pod as required in the schema, then testing the registered tool offline, proves the registry-to-mock wiring end to end.

Your task: Register the kubectl_logs tool making pod a required argument, then write a no-API-key test proving the registered tool runs and returns cluster data.

Requirements:

  • Schema declares pod (and optionally namespace) with pod in the required list
  • Invoke the tool via the registry (TOOLS[name].run(...)), not the mock directly
  • Assert the crash-looping pod's logs come back
  • Assert the known root-cause string is present in those logs
  • Runs offline — no API key

💡 Hint: Calling through TOOLS[...] rather than the mock is the point: it's the registry wiring, not just the mock, that you're proving.

Show solution
register(Tool(
    "kubectl_logs",
    "Fetch recent log lines for a pod. Use to find the actual error.",
    _obj({"pod": {"type": "string"}, "namespace": {"type": "string"}}, ["pod"]),
    RiskClass.READ_ONLY,
    lambda pod, namespace="staging": cluster.get_logs(pod, namespace),
))

# tests/test_mock_and_tools.py (add) — no API key
def test_readonly_tool_runs_and_returns_data():
    pods = TOOLS["kubectl_get_pods"].run(namespace="staging")
    assert any(p["status"] == "CrashLoopBackOff" for p in pods)

def test_logs_tool_finds_error():
    logs = TOOLS["kubectl_logs"].run(pod="checkout-api-7d9f")
    assert "password authentication failed" in logs

The only difference from kubectl_get_pods is ["pod"] in the schema, which makes pod mandatory — the model must supply it. The tests call the tools' run callables directly (no model in the loop), proving the registry ↔ mock wiring works: kubectl_get_pods returns the crash-looping pod, and kubectl_logs surfaces the real DB-auth error. These run in a fraction of a second and need no API key, so CI can run them on every commit.

Exercise 4 · Write the read-only agent loopExpert

Context: This is the manual tool-use loop from Chapter 4, hardened: a bounded loop that sends tool specs, replays the assistant turn verbatim, and returns matched tool results until the model is done.

Your task: Reproduce agent/engine.py's read-only run() loop: the MAX_STEPS cap, the client.messages.create call with the tool specs, and the tool_use→tool_result handling. Use the exact model ID the lesson uses.

Requirements:

  • A MAX_STEPS cap guarantees the loop terminates
  • Call client.messages.create with the system prompt, the tool specs, and the running messages, using model claude-opus-4-8
  • Return the text when stop_reason == "end_turn"
  • Append the assistant turn verbatim, then run each tool_use and return a tool_result carrying the matching tool_use_id
  • Send all tool results in a single user message before looping
  • Needs an API key and the anthropic SDK

💡 Hint: The three contracts from Chapter 4 are load-bearing: assistant turn appended verbatim, every result tagged with its tool_use_id, and all results batched into one user message.

Show solution
from anthropic import Anthropic
from agent.tools import TOOLS
client = Anthropic()
MAX_STEPS = 8

def run(incident: str):
    system = ("You are a careful SRE. Diagnose the incident using the "
              "read-only tools. Cite the evidence you find.")
    messages = [{"role": "user", "content": incident}]
    specs = [t.spec() for t in TOOLS.values()]

    for _ in range(MAX_STEPS):                # hard cap — always terminates
        resp = client.messages.create(model="claude-opus-4-8",
                 max_tokens=1500, system=system, tools=specs, messages=messages)
        if resp.stop_reason == "end_turn":
            return next((b.text for b in resp.content if b.type == "text"), "")
        messages.append({"role": "assistant", "content": resp.content})
        results = []
        for b in resp.content:
            if b.type == "tool_use":
                out = TOOLS[b.name].run(**b.input)     # Lab 8c gates this
                results.append({"type": "tool_result",
                                "tool_use_id": b.id, "content": str(out)})
        messages.append({"role": "user", "content": results})
    return "Stopped: hit step limit."

The loop honours the three Chapter 4 rules: append the assistant turn verbatim before results, give every tool_result the matching tool_use_id, and send all results in one user message. MAX_STEPS = 8 guarantees termination. On stop_reason == "end_turn" it returns the model's text answer; otherwise it runs each requested read-only tool via the registry and feeds results back. This needs an API key to run (ANTHROPIC_API_KEY + pip install anthropic) and uses the lesson's model, claude-opus-4-8.

Exercise 5 · Parse tool_use blocks into tool_result blocks (pure logic)Professional

Context: The loop's inner dispatch — turn tool_use blocks into tool_result blocks — is pure logic you can and should unit-test without ever calling the API.

Your task: Write a pure helper build_tool_results(content_blocks) that mirrors the engine's inner loop, and test it with fake blocks.

Requirements:

  • For each tool_use block, run the registered tool with its input
  • Emit one tool_result dict per tool_use, carrying the correct tool_use_id
  • Ignore non-tool_use blocks (e.g. text)
  • Test with a stand-in block object — no real Anthropic content needed
  • Assert one result per tool_use and that the id and type are correct; no API key

💡 Hint: Standing in a tiny fake block class lets you exercise the dispatch logic in milliseconds, isolated from the model.

Show solution
from agent.tools import TOOLS

class _Block:  # stand-in for an Anthropic content block, for testing
    def __init__(self, type, name=None, input=None, id=None, text=None):
        self.type, self.name, self.input, self.id, self.text = type, name, input, id, text

def build_tool_results(content_blocks):
    results = []
    for b in content_blocks:
        if b.type == "tool_use":
            out = TOOLS[b.name].run(**b.input)
            results.append({"type": "tool_result",
                            "tool_use_id": b.id, "content": str(out)})
    return results

# test — no API key
def test_build_tool_results_one_per_tool_use():
    blocks = [
        _Block("text", text="let me look"),
        _Block("tool_use", name="kubectl_get_pods", input={"namespace": "staging"}, id="tu_1"),
    ]
    out = build_tool_results(blocks)
    assert len(out) == 1
    assert out[0]["tool_use_id"] == "tu_1"
    assert out[0]["type"] == "tool_result"

This isolates the loop's dispatch logic from the network. Text blocks are skipped; each tool_use becomes exactly one tool_result whose tool_use_id echoes the request's b.id — the invariant that avoids the 400 tool_use_id ... without tool_result error. Because it only touches the registry and the mock, it runs with no API key, letting you unit-test the agent's plumbing deterministically.

Exercise 6 · Guarantee no tool enters the registry untaggedIndustry scenario

Context: The Chapter 8c gate keys every decision on tool.risk, so a tool registered without a real risk class is a silent hole in the safety model. A one-line CI test closes it.

Your task: Write the registry guard test proving every tool has a real RiskClass, and explain why this test is a load-bearing safety control even though it needs no API key.

Requirements:

  • Loop the registry and assert each tool's risk is a real RiskClass
  • Fail with a message naming the offending tool
  • Also assert spec() exposes only name/description/input_schema (no risk, no run)
  • Explain that an untagged tool would slip past the gate
  • Runs every commit in CI at zero cost

💡 Hint: It's cheap precisely because it's pure — and cheap-but-load-bearing is exactly the kind of test you want gating every commit.

Show solution
from agent.tools import TOOLS
from agent.schemas import RiskClass

def test_every_tool_has_a_risk_class():          # no API key
    assert TOOLS, "registry is empty — tools not imported"
    for name, tool in TOOLS.items():
        assert isinstance(tool.risk, RiskClass), f"{name} missing risk"

def test_registry_specs_are_model_safe():        # no API key
    for name, tool in TOOLS.items():
        spec = tool.spec()
        assert set(spec) == {"name", "description", "input_schema"}
        assert "risk" not in spec and "run" not in spec   # never leaked to the model

The first test loops the whole registry and asserts each tool.risk really is a RiskClass — if someone registers a tool with risk="safe" (a plain string), it fails with … missing risk. That matters because the Lab 8c gate keys its allow/ask/block decision on this field; an untagged tool could otherwise slip past. The second test confirms spec() never exposes risk or run to the model. Both are instant and free, so CI runs them on every commit as the first line of defence.

✓ Checkpoint — Lab 8b complete when…

  • Your tool registry holds the read-only tools, each with a risk class.
  • The loop runs and the agent calls tools in a sensible order.
  • python cli.py "..." produces a correct diagnosis + an audit trail.
  • The no-API-key registry tests pass.

Knowledge check check yourself

✓ Knowledge check

Each Tool bundles name, description, input_schema, risk, and run, but spec() returns only the first three to the model. Why keep risk and run private to your program?

Show answer
Only name/description/input_schema are what the Anthropic tools=[...] parameter needs, so that's all the model sees. Keeping risk and run on your side means the safety gate can look up any tool's danger level and the loop controls execution — the model never learns the risk tag it might try to argue around, nor holds the real function.
✓ Knowledge check

A tool's description says 'List pods... Use FIRST when diagnosing.' Why does the wording of a tool description materially change agent behavior, and what happens if you make it vague?

Show answer
The description is the instruction manual the model reads to decide when to call a tool, so 'Use FIRST' nudges it to check pod health before anything else — that's how the observed pods -> logs -> deploys ordering emerges. Vague wording (just 'lists pods') makes the model less likely to call it first, so it may investigate in a worse order or skip the tool entirely.
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in