Prompting Claude & model migration

1 · Anthropic-specific prompting, in one picture essential

You already know general prompting from ch02 and pe1 — be clear, give examples, ask for structure. This lesson is narrower and deeper: the handful of techniques that are specific to Claude and that Anthropic documents as first-class. There are four, and they stack:

Read it left to right as a build order. XML tags label the pieces of your prompt so Claude never confuses your data for your instructions. The system prompt sets who Claude is. Prefill (on older models) or structured output (on current ones) pins the shape of the reply. And when the prompt is huge, ordering — long document first, question last — is what keeps recall high. Everything else in this half is detail on those four.

2 · Recipe 1 — structure the prompt with XML tags essential

Claude is trained to pay attention to XML-style tags. When your prompt mixes a document, a few examples, and your instructions, wrapping each part in a tag (<document>, <example>, <instructions>) removes ambiguity: Claude can tell "the thing to act on" from "how to act on it". The tag names are yours to choose — there is no fixed vocabulary — but pick descriptive names and close every tag.

xml_prompt.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call.
# XML tags are just text in the prompt string — no special parameter, no beta header.
import anthropic

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from the environment

CONTRACT = "... the full text of a 40-page contract ..."

prompt = f"""<document>
{CONTRACT}
</document>

<instructions>
Answer the question using ONLY the contract above. Quote the exact clause you
relied on inside <quote></quote> tags, then give your answer in <answer></answer> tags.
If the contract does not address the question, say so — do not guess.
</instructions>

<question>What is the termination notice period?</question>"""

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}],
)
print(next(b.text for b in resp.content if b.type == "text"))

Two things are doing work here. First, the tags give Claude a frame: the contract is data, the instructions are rules, the question is the ask. Second, asking for the answer inside <answer> tags makes the reply easy to parse on your side — you split on the tag instead of guessing where the prose ends.

3 · The system prompt's job essential

Claude takes a system prompt separately from the conversation. Put the stable stuff there — who Claude is, the rules it must always follow, the output contract — and keep the variable stuff (the actual document, this turn's question) in the user message. The split matters for two reasons: it reads more reliably, and it caches better (the frozen system prefix is exactly what prompt caching reuses).

system_role.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call.
import anthropic

client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=512,
    # STABLE: role + hard rules + output contract — same every request, caches well.
    system=(
        "You are a senior contracts analyst. Answer only from the document the user "
        "provides. Be explicit and literal. If something is not stated, say 'not stated' "
        "rather than inferring. Always cite the clause you used."
    ),
    # VARIABLE: this turn's data + question.
    messages=[{"role": "user", "content": "<document>...</document>\n\nWhat is the SLA credit?"}],
)
print(resp.stop_reason, "|", resp.usage.input_tokens, "input tokens")

4 · Recipe 2 — prefilling the assistant turn (and its modern replacement) intermediate

Prefilling means you put the first words of Claude's reply into the conversation yourself, as an assistant message with no answer after it. Claude then continues from your words. The classic use is forcing a format: start the assistant turn with { and Claude keeps writing JSON; start it with a label and Claude fills it in. It skips preambles like "Sure, here is…".

prefill_old.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call.
# PREFILL PATTERN — works on older models (e.g. claude-sonnet-4-5). On Opus 4.6+ /
# Sonnet 4.6 / Fable 5 a trailing assistant turn returns a 400. See the replacement below.
import anthropic

client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-sonnet-4-5",                 # older model: prefill still allowed
    max_tokens=256,
    messages=[
        {"role": "user", "content": "Give the capital and population of France as JSON."},
        {"role": "assistant", "content": "{"},  # PREFILL: Claude continues the JSON
    ],
)
# The reply starts after your "{" — prepend it back to reconstruct the full object.
text = next(b.text for b in resp.content if b.type == "text")
print("{" + text)

On current models you get the same guarantee — and a stronger one — with structured output. Instead of nudging the first token, you hand Claude a JSON schema and the reply is constrained to match it. This is the recommended replacement for every "prefill to force a format" case.

structured_output.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call.
# Replaces format-forcing prefills on current models. output_config.format constrains
# the reply to your JSON schema — no prefill, guaranteed shape.
import anthropic

client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=256,
    messages=[{"role": "user", "content": "Give the capital and population of France."}],
    output_config={
        "format": {
            "type": "json_schema",
            "schema": {
                "type": "object",
                "properties": {
                    "capital": {"type": "string"},
                    "population": {"type": "integer"},
                },
                "required": ["capital", "population"],
                "additionalProperties": False,
            },
        }
    },
)
import json
text = next(b.text for b in resp.content if b.type == "text")
print(json.loads(text))    # {'capital': 'Paris', 'population': ...}

5 · Recipe 3 — long-context ordering intermediate

Claude's models have very large context windows (up to 1M tokens on Opus 4.8), so you can drop a whole book in a prompt. But where you put things changes the answer quality. Anthropic's documented long-context guidance is three rules:

Long-context rules

1. Put the long document first, the question last. Claude attends best to a query that comes after the material it's about. A 50-page doc followed by "now answer X" beats "answer X about the following: [50 pages]".
2. Wrap each document in tags (<document>, or <doc index="1"> when there are several) so Claude can reference them and you can ask it to cite by index.
3. Ask it to quote first, then answer. Instruct Claude to pull the relevant sentences into <quotes> before writing the answer. Grounding the answer in extracted quotes raises accuracy and gives you something to audit.

long_context.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call.
import anthropic

client = anthropic.Anthropic()

DOCS = {1: "... report one ...", 2: "... report two ..."}

# Long material FIRST ...
doc_blocks = "\n".join(f'<doc index="{i}">\n{txt}\n</doc>' for i, txt in DOCS.items())

# ... question and instructions LAST.
prompt = f"""{doc_blocks}

Using only the documents above:
1. Pull the sentences relevant to the question into <quotes></quotes>, tagging each with
   the doc index it came from.
2. Then write your answer in <answer></answer>, citing the doc index.

<question>Which report projects higher Q4 revenue, and by how much?</question>"""

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    messages=[{"role": "user", "content": prompt}],
)
print(next(b.text for b in resp.content if b.type == "text"))

6 · The Console prompt improver & generator advanced

You don't have to write every prompt from a blank page. The Anthropic Console (console.anthropic.com) ships two tools that most people underuse:

Treat the output as a strong first draft, not a finished artifact. The Console applies the exact techniques in this lesson — XML tags, explicit instructions, an answer contract — so its drafts are a good scaffold. But you still own the evals: a generated prompt that reads beautifully can still score worse on your task than the one you had. Generate, then measure.

7 · Why migration is a workflow, not a find-and-replace advanced

Models get deprecated and upgraded. A retired model id starts returning 404; a deprecated one keeps working until a cutoff date, then stops. So you will migrate — the only question is whether it's controlled or a fire drill. Two facts make it more than swapping a string:

What actually changes across a model version

1. The request surface can change. Moving to Opus 4.6+ / Sonnet 4.6 / Fable 5, assistant-turn prefills 400, budget_tokens is replaced by thinking={"type":"adaptive"}, and (Opus 4.7+) sampling params temperature/top_p are rejected. A blind id swap can start erroring.
2. Behavior shifts even when the code is valid. Newer models follow instructions more literally, reach for tools differently, and calibrate verbosity to the task. A prompt tuned for the old model may over- or under-trigger on the new one — so the prompt often needs adjusting, not just the id.

The upshot: a prompt is tuned for a model. When the model changes, you re-validate the prompt against evidence — which is exactly what the loop below is for. This is where your eval set (ch05) earns its keep: it's the evidence.

8 · The safe migration loop professional

The whole workflow is a loop you can draw. Nothing exotic — pin, test, diff, adjust, canary, watch — but doing it in this order is the difference between a boring migration and a Friday incident:

The loop, step by step

1. Pin the model id. Put the id in one config constant (MODEL = "claude-opus-4-8"), never scattered as string literals. You can't migrate safely what you can't find in one place.
2. Keep an eval set. A fixed set of representative inputs with known-good expectations (from ch05). This is the ground truth you'll diff against.
3. Run BOTH old and new against the evals. Same prompts, same inputs, two model ids. Compare pass-rates and eyeball the diffs — where the new model changed matters as much as the aggregate number.
4. Adjust prompts if needed. If the new model regresses, it's usually prompt fit, not a bad model: dial back aggressive tool instructions, add a length instruction, remove prefills. Re-run the evals after each change.
5. Roll out behind a flag (canary). Send a small slice of real traffic to the new model, keep the rest on the old one. A flag means instant rollback with no deploy.
6. Watch, then promote. Monitor the canary — latency, cost, error rate, refusals, and any quality signal you have — then widen the flag to 100% once it holds.

9 · Recipe 4 — the migration gate (this one runs offline) professional

The "Pass?" diamond in the diagram deserves an actual rule, not a vibe. The helper below is pure stdlib and runs with a plain python file.py — no API key, no network. It takes per-case results for the old and new model on the same eval set and returns a decision: go (new is as good or better), adjust (small regression — tune prompts and re-run), or block (real regression — do not ship).

migration_gate.pydef migration_gate(old_results, new_results, adjust_band=0.03):
    """Decide whether a new model is safe to roll out, from eval results.

    old_results / new_results: dict[case_id] -> bool (True = passed the eval).
    Both must cover the SAME case ids. Returns a decision dict.

    Rule (per-case regressions matter, not just the aggregate):
      no case regressed                        -> "go"      (as good or better)
      some regressed, rate held within band    -> "adjust"  (small/hidden regression)
      rate dropped past the band               -> "block"   (real regression)
    A "regression" is a case the old model passed but the new one fails. Note a
    migration can hold its aggregate pass-rate while still regressing cases (it fixed
    as many as it broke) — that is exactly why go requires ZERO regressions.
    """
    ids = sorted(old_results)
    assert ids == sorted(new_results), "eval sets must cover the same case ids"

    n = len(ids)
    old_rate = sum(old_results[i] for i in ids) / n
    new_rate = sum(new_results[i] for i in ids) / n
    regressions = [i for i in ids if old_results[i] and not new_results[i]]

    if not regressions and new_rate >= old_rate:
        decision = "go"
    elif old_rate - new_rate <= adjust_band:
        decision = "adjust"
    else:
        decision = "block"

    return {
        "old_rate": round(old_rate, 3),
        "new_rate": round(new_rate, 3),
        "delta": round(new_rate - old_rate, 3),
        "regressions": regressions,
        "decision": decision,
    }


# Fake eval set: 10 cases, old model (e.g. sonnet-4-5) vs new (opus-4-8).
old = {f"case-{i:02d}": True for i in range(10)}
old["case-07"] = False                       # old already failed this one
new = dict(old)
new["case-03"] = False                       # new regressed one case the old passed
new["case-07"] = True                        # new fixed the one old failed

result = migration_gate(old, new)
for k, v in result.items():
    print(f"{k}: {v}")

old_rate: 0.9
new_rate: 0.9
delta: 0.0
regressions: ['case-03']
decision: adjust

Read the output: overall pass-rate is unchanged (0.9 → 0.9), but the models disagree on which cases — the new one fixed case-07 and broke case-03. The aggregate hid a real regression, which is why the gate reports regressions explicitly and returns adjust: go look at case-03, tune the prompt, re-run. A pass-rate that only went up would return go; a drop past the band would return block.

10 · Tech-lead — owning migrations across a fleet tech-lead

A lead's job is to make model changes routine and reversible across many services, not heroic one-offs. The levers:

Fleet-level migration discipline

1. One pinned id per service, discoverable in one grep. A single MODEL constant (or a shared config key) per service. When a deprecation lands, you want rg 'claude-' to find every call site in seconds — not a scavenger hunt.
2. Evals are a shipping gate, tied to the prompt. The eval set lives with the code and runs in CI. A model or prompt change that drops the gate (the migration_gate returning block) fails the build.
3. The model id is a flag, not a constant edit. Route it through a feature flag / config so you can canary and roll back without a deploy. Deploy-to-rollback is too slow when a migration goes sideways in production.
4. Migration is a checklist, not a memory. Codify the breaking changes (prefill → structured output, budget_tokens → adaptive thinking, drop sampling params) so every service applies the same edits. Anthropic publishes a per-version migration guide — start from it.
5. Watch refusals and stop reasons after cutover. New models can refuse more appropriately (handle stop_reason == "refusal") and hit the context window differently. Add these to the canary dashboard, not just latency and cost.
6. Keep the old id available during the canary. Until the new model is at 100% and holding, the old (non-retired) id is your instant fallback. Don't delete the old path until the new one has soaked.

The mental model: a model version is a dependency, and you manage it like any other — pinned, tested in CI, rolled out behind a flag, monitored, and rolled back cleanly. Prompting well (Half A) makes each service work; the migration loop (Half B) keeps it working as the model underneath it moves.

import anthropic
client = anthropic.Anthropic()

doc = "Our refund window is 30 days from delivery."
prompt = (
    "<document>\n" + doc + "\n</document>\n"
    "<question>How long do customers have to request a refund?</question>"
)
resp = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=128,
    messages=[{"role": "user", "content": prompt}],
)
print(resp.content[0].text)

import anthropic
client = anthropic.Anthropic()

SYSTEM = ("You are a support classifier. Reply with exactly one label: "
          "billing, technical, or other.")   # stable -> cacheable
resp = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=8,
    system=SYSTEM,
    messages=[{"role": "user", "content": "My card was charged twice."}],  # variable
)
print(resp.content[0].text)

import anthropic
client = anthropic.Anthropic()

schema = {"type": "object",
          "properties": {"label": {"type": "string"},
                         "confidence": {"type": "number"}},
          "required": ["label", "confidence"]}
resp = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=128,
    output_config={"format": {"type": "json_schema", "schema": schema}},
    messages=[{"role": "user", "content": "Classify: the app keeps crashing."}],
)
print(resp.content[0].text)   # valid JSON per the schema

import anthropic
client = anthropic.Anthropic()

long_doc = "...(many pages of policy text)..."
prompt = (
    "<document>\n" + long_doc + "\n</document>\n"
    "<instructions>First quote the exact sentence that answers the "
    "question, then answer in one line.</instructions>\n"
    "<question>What is the SLA for critical incidents?</question>"
)
resp = client.messages.create(
    model="claude-sonnet-4-6", max_tokens=256,
    messages=[{"role": "user", "content": prompt}],
)
print(resp.content[0].text)

def migration_gate(old, new, band=0.03):
    assert len(old) == len(new)
    regressed = [i for i, (o, n) in enumerate(zip(old, new)) if o and not n]
    drop = (sum(old) - sum(new)) / len(old)
    if not regressed:
        decision = "go"
    elif drop <= band:
        decision = "adjust"
    else:
        decision = "block"
    return {"decision": decision, "regressed": regressed, "net_drop": round(drop, 4)}

old = [1,1,1,1,1,1,1,1,1,1]
new = [1,1,0,1,1,1,1,1,1,1]     # one regression, 10% drop
print(migration_gate(old, new))  # block (drop 0.1 > band)

def uses_prefill(messages):
    return bool(messages) and messages[-1].get("role") == "assistant"

def gate(old, new, band=0.03):
    reg = [i for i,(o,n) in enumerate(zip(old,new)) if o and not n]
    drop = (sum(old)-sum(new))/len(old)
    return "go" if not reg else ("adjust" if drop<=band else "block")

prompts = {
    "p1": {"messages": [{"role":"user","content":"x"}],
           "old": [1,1,1,1], "new": [1,1,1,1]},
    "p2": {"messages": [{"role":"user","content":"x"},
                        {"role":"assistant","content":"{"}],   # prefill!
           "old": [1,1,1,1], "new": [1,1,1,1]},
}
plan = {}
for name, p in prompts.items():
    if uses_prefill(p["messages"]):
        plan[name] = "refactor: prefill -> output_config.format"
    else:
        d = gate(p["old"], p["new"])
        plan[name] = "canary behind flag" if d == "go" else f"hold ({d})"
print(plan)

Knowledge check check yourself