Prompting & Structured Output

The five-part system prompt advanced

Here's a scene you'll recognize the moment you ship your first LLM feature. The demo worked beautifully — you typed a clever instruction into the playground, the model nailed it, everyone clapped. Then real traffic arrives, the instruction is buried in a wall of pasted-together strings, and the model starts ignoring half of what you asked. Nobody changed the model; you changed the prompt — or rather, you never gave it a structure in the first place. The five-part pattern is the structure that turns a lucky one-off into something that behaves the same on request number one and request number ten thousand.

Think of a system prompt the way you'd think of briefing a new contractor on their first morning. You don't lead with today's ticket. You start with who they are and what they're here to do ("you triage support tickets"), then the house rules ("always reply in this format, never guess"), then what tools are on the bench and when to touch them, then a couple of worked examples of a job done right — and only then do you hand them the actual task in front of them. Reverse that order and the contractor reads the specific task before they know the rules, and the rules land as an afterthought. Models read top-to-bottom the same way: whatever frames the work has to come before the work.

The common mistake is treating the prompt as one undifferentiated blob of text — role, rules, examples, and the user's message all mashed together in whatever order they were written. It feels fine because a strong model often copes anyway. The damage is invisible until it isn't: instructions get contradicted by later text, edits to one concern accidentally break another, and — the part that costs real money — you lose prompt caching. Caching only helps when a long, stable prefix stays byte-for-byte identical across calls; if your fixed instructions are tangled up with the per-request user input, nothing is stable enough to cache and you pay full price every time. Separating the stable parts from the volatile ones is what makes the discipline below pay for itself twice: better instruction-following and a cacheable prefix.

So the pattern is less a rigid template than a discipline of ordering by how often each part changes: role and objective almost never change, constraints rarely, tool rules occasionally, examples now and then — and the retrieved documents plus the user's message change on every single call. Stack them stable-to-dynamic and both goals fall out for free. A production system prompt almost always has these parts, in this order:

Lab 2.1 · Build and iterate on a prompt advanced

You don't write a good prompt; you iterate one. The lab below walks a naive version to a specified one — each round adds structure, tested against the same fixed input so the comparison is fair. The diagram shows that loop.

Lab 2.2 · Few-shot examples advanced

Try to describe, in words, exactly how you'd like a date formatted. "Use the day, then the month, then the year, with dashes… no wait, sometimes the month is a word…" — you'll write a paragraph and still leave gaps. Now instead just show two dates the way you want them: 2026-03-09, 2026-11-21. Done. The reader infers the rule instantly. That gap — between how painful it is to specify a format and how easy it is to demonstrate one — is the entire reason few-shot prompting exists.

Few-shot means you hand the model a handful of worked examples (the "shots") before the real question, and let it infer the pattern rather than reading a description of it. It's the difference between teaching someone to make an omelette by reciting a recipe versus letting them watch you crack, whisk, and fold two of them first. Humans and models both learn a format far faster from a couple of concrete demonstrations than from prose that tries to name every rule. When you catch yourself writing a long, hedge-filled description of the output you want ("mostly lowercase, but capitalize proper nouns, and if there's no category use 'other', and…"), that's the signal to stop describing and start showing.

The mechanical detail that trips people up: the examples are not decoration you paste into the system prompt — they live as actual prior turns in the messages array, alternating user (the example input) and assistant (the ideal answer). You're literally faking a short conversation that already went well, so the model's most natural move is to continue the established pattern. The single most common beginner error is dumping the examples as one blob of text; formatting them as real turns is what makes the model treat them as a demonstration to imitate rather than trivia to read.

A couple of honest cautions before the lab. Examples are not free: every one of them is tokens you pay for and send on every call, so five carefully chosen examples beat twenty lazy ones. And the examples teach — including their mistakes. If every example you show happens to be a high-severity bug, don't be surprised when the model labels a trivial typo as "high." Pick examples that span the variety you actually expect, especially the tricky edge cases you most want handled correctly. When you need a specific format or edge-case handling, show don't tell. Examples live as prior turns in the messages array.

fewshot.pymessages = [
    {"role":"user",      "content":"Server crashed on null input to /login"},
    {"role":"assistant", "content":"bug | high | auth"},
    {"role":"user",      "content":"Please add dark mode"},
    {"role":"assistant", "content":"feature | low | ui"},
    {"role":"user",      "content":"Billing charged me twice this month"},  # real input
]
# The model continues the pattern: "bug | high | billing"

Few-shot is powerful but has a cost: those example tokens are sent every call. For strict formats, structured output (below) is usually cleaner and cheaper than many examples.

Effort & adaptive thinking expert

Imagine paying a senior consultant their full hourly rate to answer "is the office open on Monday?" They'd give you a perfect, thoroughly-reasoned answer — and you'd have wildly overpaid for a yes/no. Now imagine handing that same consultant a five-minute glance at a critical database migration. You'd get a fast, shallow answer to a question that deserved an afternoon of thought. Both are the same mistake in opposite directions: the amount of thinking didn't match the difficulty of the task. Effort and adaptive thinking are the dials that let you match the two, and getting this right is one of the highest-leverage cost skills in the whole course.

Here's what "thinking" actually is, plainly: modern models can do a stretch of hidden reasoning before they write their visible answer — scratch work, like you jotting on paper before committing to a solution. That scratch work genuinely improves answers on hard problems, but it costs tokens and time, and on an easy problem it's pure waste. So you're not choosing whether the model is "smart"; you're choosing how much scratch work it's allowed to do, which is really a choice about spend and latency versus depth.

The two dials do slightly different jobs, and confusing them is the common trip-up. Adaptive thinking (thinking={"type":"adaptive"}) hands the model the judgement call — it looks at how hard the question is and decides how much to reason on its own, so easy questions stay cheap and hard ones get more effort without you micromanaging each call. Effort (output_config={"effort": ...}) is your ceiling on top of that — a coarse low/medium/high/max dial that says how much you're willing to spend overall. Think of adaptive thinking as the model's own throttle and effort as the speed limit you post; the model reasons freely up to the limit you set.

The instinct to reach for the biggest, most-thinking setting "just to be safe" is the exact instinct to resist. On a classification or routing task, high effort buys you nothing but a bigger bill and a slower response — the answer was never in doubt. The skill is boring and valuable: cheap dials for cheap tasks, expensive dials only where correctness genuinely beats cost. One more thing worth flagging up front — the API for this changed, and old tutorials will lead you astray, which the callout below covers. Two dials trade cost/latency against quality:

Setting the effort dial — in Claude or OpenAI

Both vendors expose the same idea — let the model think, but cap how hard. In Claude you pass output_config={"effort": …} (with thinking={"type":"adaptive"} so the model self-throttles); in OpenAI the dial is reasoning={"effort": …} on a reasoning model, which adapts within that ceiling on its own. Both accept low | medium | high | max. Use the tabs to switch SDK — your choice applies across the whole page.

effort.pyresp = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    thinking={"type": "adaptive"},         # model decides how much to reason
    output_config={"effort": "medium"},  # low | medium | high | max
    messages=[{"role":"user","content":"Plan a database migration strategy."}],
)

effort.pyresp = client.responses.create(
    model="gpt-5.5", max_output_tokens=1024,
    reasoning={"effort": "medium"},       # low | medium | high | max
    input="Plan a database migration strategy.",
)

Why structured output matters expert

Picture the 2am page. Your classifier has been humming along for weeks, its output scraped by a regex that plucks the priority out of the model's reply. Then one night the model, entirely reasonably, answers "Sure! The priority here is high 😊" instead of the "Priority: high" your regex expected — and the parse returns nothing, the ticket routes nowhere, and your phone lights up. Nothing broke on your side. The model just rephrased. That fragility — a system that depends on the model choosing the same words twice — is exactly what structured output exists to eliminate.

The core idea is a shift in who guarantees the shape of the data. With free text, you're hoping the model happens to phrase things the way your parser expects, then writing defensive code to cope when it doesn't — you're guaranteeing the shape after the fact, badly. With structured output you flip it: you declare a schema up front, and the shape becomes a constraint the model must satisfy while generating, not a pattern you fish for afterward. It's the difference between asking a stranger to "write down your details somewhere on this page" and handing them a form with labelled boxes. The form can't be filled in wrong shape; the blank page can be filled a thousand ways.

Why this suddenly matters so much: the instant the model's output stops being read by a human and starts feeding another program — a database insert, an API call, a UI that branches on the value, the next step in an agent — free text becomes a genuine liability rather than a mild annoyance. A human reader shrugs off an emoji or an extra sentence of preamble; a downstream function chokes on it. The regex you'd write to cope is brittle by nature: it encodes assumptions about phrasing that the model was never promised to keep, so it works in your tests (where you saw three phrasings) and fails in production (where it meets the fourth).

The most common mistake here isn't writing a bad regex — it's reaching for a regex at all. Every hour spent hardening a pattern to survive the model's rewordings is an hour spent solving a problem you could have deleted. Constrain the output to a schema and the whole class of "the model said it differently this time" bugs simply stops existing; you get back a typed object you can use directly, with no parsing step to break. The moment another program consumes the model's output — a database, an API, a UI — free text is a liability. You'll write brittle regex, it'll break on the model's next phrasing, and you'll page someone at 2am. Instead, constrain the output to a schema the model must satisfy.

Lab 2.3 · Schema-constrained JSON with Pydantic expert

Here the abstract idea becomes concrete: you declare a Pydantic model, pass it as output_format, and get a real typed object back — no parsing step to break. The diagram traces that one-way path from your class to a validated instance.

extract.pyfrom dotenv import load_dotenv
from anthropic import Anthropic
from pydantic import BaseModel
from typing import Literal
load_dotenv()
client = Anthropic()

class Ticket(BaseModel):
    category: Literal["bug","feature","billing","other"]
    priority: Literal["low","medium","high"]
    summary: str
    needs_human: bool

resp = client.messages.parse(               # .parse() validates for you
    model="claude-opus-4-8", max_tokens=512,
    messages=[{"role":"user",
        "content":"I was charged twice and support hasn't replied in 3 days!"}],
    output_format=Ticket,
)

t = resp.parsed_output                      # a real Ticket instance
print(t.category, t.priority, t.needs_human)
print(type(t))                              # <class 'Ticket'>

billing high True
<class '__main__.Ticket'>

extract.pyfrom dotenv import load_dotenv
from openai import OpenAI
from pydantic import BaseModel
from typing import Literal
load_dotenv()
client = OpenAI()

class Ticket(BaseModel):
    category: Literal["bug","feature","billing","other"]
    priority: Literal["low","medium","high"]
    summary: str
    needs_human: bool

resp = client.responses.parse(              # .parse() validates for you
    model="gpt-5.5", max_output_tokens=512,
    input="I was charged twice and support hasn't replied in 3 days!",
    text_format=Ticket,
)

t = resp.output_parsed                      # a real Ticket instance
print(t.category, t.priority, t.needs_human)
print(type(t))                              # <class 'Ticket'>

billing high True
<class '__main__.Ticket'>

Lab 2.4 · Ship a real classifier expert

Combine everything into a deliverable: a batch text classifier that returns typed results and never crashes on bad output. The diagram shows the flow every item takes — including the fail-safe branch that turns a bad response into data instead of a crash.

classifier.pyfrom pydantic import BaseModel
from typing import Literal
import anthropic
from anthropic import Anthropic
client = Anthropic()

class Sentiment(BaseModel):
    label: Literal["positive","neutral","negative"]
    confidence: float

SYSTEM = (
    "You classify customer feedback sentiment. "
    "Confidence is your certainty from 0.0 to 1.0."
)

def classify(text: str) -> Sentiment | None:
    try:
        r = client.messages.parse(
            model="claude-haiku-4-5",      # small model — this is easy work
            max_tokens=128, system=SYSTEM,
            messages=[{"role":"user","content":text}],
            output_format=Sentiment,
        )
        return r.parsed_output
    except (anthropic.APIError, ValueError):
        return None                          # caller decides how to handle

for fb in ["Love it!", "It's fine I guess", "Worst update ever"]:
    s = classify(fb)
    print(f"{fb:20} -> {s.label} ({s.confidence:.2f})" if s else f"{fb}: FAILED")

Love it!             -> positive (0.98)
It's fine I guess    -> neutral (0.72)
Worst update ever    -> negative (0.99)

classifier.pyfrom pydantic import BaseModel
from typing import Literal
import openai
from openai import OpenAI
client = OpenAI()

class Sentiment(BaseModel):
    label: Literal["positive","neutral","negative"]
    confidence: float

SYSTEM = (
    "You classify customer feedback sentiment. "
    "Confidence is your certainty from 0.0 to 1.0."
)

def classify(text: str) -> Sentiment | None:
    try:
        r = client.responses.parse(
            model="gpt-5.5",              # small/cheap model — this is easy work
            max_output_tokens=128, instructions=SYSTEM,
            input=text,
            text_format=Sentiment,
        )
        return r.output_parsed
    except (openai.APIError, ValueError):
        return None                          # caller decides how to handle

for fb in ["Love it!", "It's fine I guess", "Worst update ever"]:
    s = classify(fb)
    print(f"{fb:20} -> {s.label} ({s.confidence:.2f})" if s else f"{fb}: FAILED")

Love it!             -> positive (0.98)
It's fine I guess    -> neutral (0.72)
Worst update ever    -> negative (0.99)

Common pitfalls expert

Exercises expert

class _Any:
    '''stands in for any undefined demo value; supports call/attr/index/
    iteration and basic arithmetic (as 0.7) so demo snippets run.'''
    def __call__(self, *a, **k): return _Any()
    def __getattr__(self, k): return _Any()
    def __getitem__(self, k): return _Any()
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __contains__(self, o): return True
    def __enter__(self, *a): return _Any()
    def __exit__(self, *a): return False
    def __float__(self): return 0.7
    def __int__(self): return 1
    def __lt__(self, o): return True
    def __gt__(self, o): return False
    def __le__(self, o): return True
    def __ge__(self, o): return False
    def __add__(self, o): return o
    def __radd__(self, o): return o
    def __bool__(self): return True
    def __repr__(self): return 'demo'
    def __str__(self): return 'demo'
def classify(*a, **k):  # demo stub
    return _Any()
fb = _Any()
class _results_t:
    append = 'demo'
    def append(self, *a, **k): return 'demo'
    def __getattr__(self, k): return 'demo'
results = _results_t()
class _review_queue_t:
    append = 'demo'
    def append(self, *a, **k): return 'demo'
    def __getattr__(self, k): return 'demo'
review_queue = _review_queue_t()

s = classify(fb)
if s is None or s.confidence < 0.6:
    review_queue.append(fb)
else:
    results.append(s)

parts = [
    "You are a support-ticket triager. Classify and route incoming tickets.",  # 1 role
    "Respond in JSON only. Never invent a category outside the enum.",          # 2 constraints
    "Call escalate only when the user reports data loss.",                      # 3 tools
    "Example: 'server crash' -> bug | high | auth",                             # 4 few-shot
    "{user_ticket}",                                                            # 5 dynamic (LAST)
]
system = "\n".join(parts)
print(system)
assert system.index("You are") < system.index("{user_ticket}")  # stable before dynamic

messages = [
    {"role":"user",      "content":"Server crashed on null input to /login"},
    {"role":"assistant", "content":"bug | high | auth"},
    {"role":"user",      "content":"Please add dark mode"},
    {"role":"assistant", "content":"feature | low | ui"},
    {"role":"user",      "content":"Billing charged me twice this month"},  # real input
]
roles = [m["role"] for m in messages]
assert roles[0] == "user" and roles[-1] == "user"
assert all(a != b for a, b in zip(roles, roles[1:])), "must alternate"
print("examples:", roles.count("assistant"), "| to classify:", messages[-1]["content"])

RULES = {
    "classify":  ("low",    "simple, fast, cheap"),
    "route":     ("low",    "simple, fast, cheap"),
    "extract":   ("low",    "shallow structure"),
    "summarize": ("medium", "balanced default for app work"),
    "plan":      ("high",   "hard multi-step reasoning"),
    "code":      ("high",   "correctness beats cost"),
}
def choose_effort(task):
    return RULES.get(task, ("medium", "unknown task -> safe balanced default"))

for t in ["classify", "plan", "translate"]:
    lvl, why = choose_effort(t)
    print(f"{t:10} -> effort={lvl:6} ({why})")

from typing import Optional, Literal
from pydantic import BaseModel, ValidationError

class SalesLead(BaseModel):
    name: Optional[str] = None
    email: Optional[str] = None
    company: Optional[str] = None
    interested_products: list[str] = []
    stage: Literal["cold", "warm", "hot"] = "cold"

ok = SalesLead(name="Sam", interested_products=["Pro"], stage="warm")
print(ok.company, ok.stage)          # None warm  -- missing field is fine

try:
    SalesLead(stage="boiling")        # not in the Literal enum
except ValidationError as e:
    print("rejected bad stage:", e.error_count(), "error(s)")

# --- real classifier (needs ANTHROPIC_API_KEY) ---
# from pydantic import BaseModel
# from typing import Literal
# import anthropic; from anthropic import Anthropic
# client = Anthropic()
# class Sentiment(BaseModel):
#     label: Literal["positive","neutral","negative"]
#     confidence: float
# def classify(text):
#     try:
#         r = client.messages.parse(model="claude-haiku-4-5", max_tokens=128,
#             messages=[{"role":"user","content":text}], output_format=Sentiment)
#         return r.parsed_output
#     except (anthropic.APIError, ValueError):
#         return None

# --- runnable routing logic with a stub classifier ---
class Sentiment:
    def __init__(self, label, confidence):
        self.label = label; self.confidence = confidence

def classify(text):                      # stub standing in for the SDK call
    table = {"Love it!": Sentiment("positive", 0.98),
             "It's okay I guess": Sentiment("neutral", 0.42),
             "\x00": None}             # simulate a parse failure
    return table.get(text, Sentiment("neutral", 0.5))

results, review_queue = [], []
def classify_and_route(text):
    s = classify(text)
    if s is None or s.confidence < 0.6:
        review_queue.append(text)
    else:
        results.append((text, s.label, s.confidence))

for fb in ["Love it!", "It's okay I guess", "\x00"]:
    classify_and_route(fb)
print("trusted:", results)
print("to human:", review_queue)

import hashlib

class PromptRegistry:
    def __init__(self):
        self.store = {}   # (id, version) -> text
    def register(self, pid, version, text):
        self.store[(pid, version)] = text
    def get(self, pid, version):
        return self.store[(pid, version)]

def canary_version(pid, request_key, canary_pct):
    """Deterministic bucket so the same user is stable across calls."""
    h = int(hashlib.sha256(f"{pid}:{request_key}".encode()).hexdigest(), 16)
    return "v2" if (h % 100) < canary_pct else "v1"

reg = PromptRegistry()
reg.register("triage", "v1", "You are a triager. Respond JSON only.")
reg.register("triage", "v2", "You are a triager. Respond JSON only. State reasons briefly.")

log = []
for uid in ["u1", "u2", "u3", "u4", "u5", "u6", "u7", "u8"]:
    ver = canary_version("triage", uid, canary_pct=25)   # 25% see v2
    prompt = reg.get("triage", ver)
    log.append({"user": uid, "prompt_id": "triage", "prompt_ver": ver})

from collections import Counter
print("split:", Counter(r["prompt_ver"] for r in log))
print("every log line carries the version:", all("prompt_ver" in r for r in log))

Knowledge check check yourself