Request parameters, in full
A precise reference to every parameter of client.messages.create: what each does, its sane default, when you'd actually change it, and the gotchas — don't set both temperature and top_p, max_tokens must fit the model, forcing a tool changes stop_reason, extended thinking runs at temperature=1. Defaults are good; change them deliberately.
Learning objectives
- Name every parameter of
client.messages.createand say what each one does. - Recite the three required parameters and give sane defaults for the rest.
- Shape
messagesandsystemcorrectly — roles, content blocks, cache_control. - Tune sampling deliberately:
temperaturevstop_p(never both),top_k,stop_sequences. - Force, allow, or forbid tools with
tool_choice, and control parallel calls. - Turn on extended thinking correctly and know the gotchas (thinking needs
temperature=1). - Recognise the operational knobs —
stream,metadata,service_tier,betas— and when to touch them.
1 · One endpoint, many knobs essential
Picture the cockpit of a plane you've never flown. Dozens of switches, and the panic move is to start flipping the ones that look interesting. But a good pilot knows that almost all of those switches sit at a safe default, and touching the wrong one on a calm day is how you make things worse, not better. The Claude API is exactly that cockpit: one big panel of parameters, most of which you should leave alone unless you can say out loud why you're moving them.
Here's the reassuring part, and the reason this reference is worth reading once end to end: it's all one call. People new to LLM APIs expect a maze — a separate endpoint for tools, another for vision, another for streaming, another for "reasoning mode". There isn't. Everything is a keyword argument on client.messages.create(...). Once that clicks, "how do I add tools?" stops being "which API do I switch to?" and becomes "which parameter do I add?" — a much smaller, much friendlier question.
The common mistake this whole lesson is built to prevent is over-configuring: a beginner reads that temperature, top_p, and top_k all exist, sets all three "to be safe," and ends up with a call that's slower, costlier, and often dumber than the three-line version would have been. Every knob you turn is a chance to break something that was working. So the mental model to carry from the first page to the last is simple: the defaults are good — change them deliberately, one at a time, with a reason you could defend.
Almost everything you do with Claude goes through a single call: client.messages.create(...). Tools, vision, streaming, thinking, structured output — none of them are separate APIs. They are all parameters on this one call. Learn the parameters once and the whole surface opens up.
This lesson is a reference. Earlier lessons showed you the parameters you needed in the moment (a model here, a max_tokens there); this one lays every parameter out in a single place, with its default, what it actually changes, and when — if ever — you should touch it. Read it once end to end, then come back to the big table whenever you're unsure.
There are only three required parameters — model, max_tokens, and messages. Everything else has a sensible default, and the single most common mistake is over-configuring: setting knobs you don't understand and making the model worse. The mental model to carry through this lesson: the defaults are good — change them deliberately, one at a time, with a reason.
messages.create. Three inputs are required; the rest are optional knobs you add only with a reason.
Read left to right: inputs on the left flow into the one endpoint in the middle, which returns one response on the right.
- The blue boxes —
model,max_tokens,messages— are the three required inputs. Every call sets all three; they have no defaults. - The amber box stands for the whole pile of optional knobs (temperature, tools, thinking…). Its arrow is dashed because most calls leave it empty — the defaults are already good.
- The green box in the middle is the single endpoint
messages.create; everything funnels through it and comes back as one response. There is no separate "tools API" or "streaming API".
In short: One call, three required inputs, one response — every other feature is just another keyword argument you opt into.
2 · The parameter groups essential
A flat list of fifteen parameters is a memorisation problem. Five labelled drawers is a filing problem — and filing is far easier. The reason this section exists before the big reference table is that a list you have to memorise slides out of your head by next week, but a shape you understand sticks. So instead of asking you to hold fifteen names in your head, we group them into five drawers, and from then on the only question about any parameter is "which drawer does this live in?"
Think of it like the controls in a car, sorted by what they affect. Some controls you must operate to move at all (key, gear, wheel) — those are the required parameters. Some shape where you're going (the destination you punch into the GPS) — that's your prompt. Some tune how the car behaves without changing the destination (sport mode, eco mode) — that's sampling. Some are optional attachments you only fit when the trip needs them (a roof box) — that's tools. And some are about the journey's logistics rather than the drive itself (toll transponder, trip logger) — the operational knobs. Same panel, five very different jobs.
The mistake this grouping heads off is reaching into the wrong drawer. Engineers who don't have this map will try to make a response "more focused" by adding a tool, or try to speed up a call by lowering temperature — both are category errors, like adjusting the GPS to change the engine mode. When a parameter confuses you, name its drawer first: is this about what I'm asking (prompt), how the words are chosen (sampling), whether it can act (tools), or how the request is delivered and billed (operational)? The drawer usually answers the question before you've read the docs.
The parameters fall into five natural groups. Hold this shape in your head and every specific parameter has a home:
This picture is the whole parameter surface of messages.create on one line. Every specific parameter in the lesson lives in exactly one of these five boxes — so when a parameter confuses you, first ask which box it's in.
- Required —
model,max_tokens,messages. The minimum any call needs; there are no defaults for these three. - Prompt —
systemand the content blocks insidemessages. This is where your actual instructions and the conversation live. - Sampling —
temperature,top_p,top_k,stop_sequences. These tune how the model picks each next token, not what you ask. - Tools —
toolsandtool_choice. Present only when you want the model to call functions. - Operational —
stream,thinking,metadata,service_tier,betas. These govern delivery, reasoning, and billing rather than the words of the answer.
In short: The big table in §3 is organised by these same five groups. Read left to right — most calls set the first two boxes, occasionally reach into sampling or tools, and leave the operational box to a shared house wrapper.
Required is the minimum any call needs. Prompt shaping is where your actual instructions and conversation live. Sampling tunes how the model picks the next token. Tools let the model call functions. Operational knobs govern delivery, reasoning, and billing rather than the content of the answer. The reference table in §3 uses these same groups.
3 · The full reference table essential
This is the page you'll come back to at 2am when a call is misbehaving and you can't remember whether the default for temperature is 0 or 1. Don't try to absorb it in one sitting — that's not what a reference is for. Read it once to learn the shape (the five drawers from §2, laid out row by row), then treat it as the map you unfold whenever a parameter surprises you.
The single most useful column here is the last one — "When to touch it." A lot of API docs tell you what a parameter does and stop there, which quietly implies you should be setting all of them. This table takes the opposite stance: for every non-required row it tells you the specific situation that justifies the deviation, and by omission, tells you to leave it alone the rest of the time. If you can't find your situation in that column, the honest answer is usually "don't set this parameter."
Read each row as a small contract: the default is what happens if you say nothing, what it does is the behaviour, and when to touch it is the trigger. Notice how few rows say "Always" — only the three required ones and system. Everything below system is a lever for a special case, and the skill this table teaches is recognising when you're actually in that special case versus when you just think you are.
Scan one row left to right; each box is one column of the reference table in the order you should think about them.
- The blue box is the parameter name — the thing you might set.
- The green box is its default: what the API assumes when you say nothing. Most rows are safe here.
- The amber box is what it does — the behaviour the parameter controls.
- The grey box is the payoff column, when to touch it — the specific trigger that justifies deviating from the default.
- The bottom line is the rule the table quietly teaches: if no trigger in that last column matches your situation, leave the parameter alone.
In short: Read for the trigger, not the definition — "when to touch it" tells you whether this row is even yours to set.
Every parameter of messages.create, its default, what it does, and when you'd actually reach for it. Bookmark this section.
| Parameter | Default | What it does | When to touch it |
|---|---|---|---|
model (required) | — | Which Claude answers (e.g. claude-opus-4-8). | Always. Pick by task: Opus for hard work, Haiku for cheap/fast. |
max_tokens (required) | — | Hard cap on output tokens. The model stops here even mid-sentence. | Always set it. Big enough to finish; must fit the model's output limit. |
messages (required) | — | The conversation: a list of {role, content}. Roles alternate user/assistant. | Always. This is the prompt (plus any history you carry). |
system | None | Top-level instructions / persona. A string, or a list of text blocks (for cache_control). | Almost always — it's where role, rules, and tools context go. |
temperature | 1.0 | Randomness of token choice. 0 ≈ deterministic-ish, 1 = full range. | Lower (0–0.3) for extraction/classification; leave at 1 for creative work. |
top_p | 1.0 | Nucleus sampling: consider only the top tokens summing to p probability. | Rarely. An alternative to temperature — never set both. |
top_k | unset | Only sample from the k most likely tokens. | Rarely — an advanced sampling lever; most workloads never need it. |
stop_sequences | [] | Strings that force the model to stop when generated (the string is not included). | When you need a hard delimiter — end of a section, a custom terminator. |
stream | False | Return tokens incrementally over SSE instead of one final message. | Interactive UIs, and any large-max_tokens call (avoids timeouts). |
tools | None | List of tool definitions (name, description, JSON-schema input) the model may call. | When the model needs to act — call functions, search, run code. |
tool_choice | {"type":"auto"} | Whether the model may / must / must-not use a tool. | To force a specific tool, force some tool, or forbid tools for one turn. |
thinking | off | Extended (adaptive) reasoning before the answer: {"type":"adaptive"}. | Hard multi-step problems. Pair with output_config.effort. |
metadata | None | Opaque request metadata, notably {"user_id": ...} for abuse tracking. | In production, to attach a stable per-end-user id (hashed, not PII). |
service_tier | provider default | Requests a priority/standard/batch service level (verify current values in docs). | Rarely — when you have a priority-tier arrangement and want to steer to it. |
betas / extra_headers | none | Opt into beta features via client.beta.messages + a beta flag string. | Only when a specific beta feature needs it (files, caching betas, etc.). |
output_config | None | Structured-output format and the effort dial for thinking depth. | For guaranteed JSON (format) or to tune reasoning cost (effort). |
4 · Required + prompt shaping — model, max_tokens, messages, system essential
Get these four right and you can ship real features; get them subtly wrong and you'll spend an afternoon debugging a "the model isn't following instructions" bug that was never about the model at all. This is the group you touch on essentially every call, so it's worth slowing down on the two that actually have shape to them — messages and system — rather than treating them as boilerplate.
The mental model for messages is a transcript of a conversation, written down turn by turn. Each turn is one speaker (a role, either "user" or "assistant") and what they said (content). The API is stateless — it has no memory of earlier calls — so this transcript is the whole story every time; if you want the model to "remember" the last three turns, you resend all three. That's the single most common beginner surprise: the model didn't forget, you simply didn't include the history.
Content can be a plain string when all you have is text, but it can also be a list of content blocks — {"type": "text", ...}, {"type": "image", ...}, a tool result — which is how one user turn carries a screenshot and a question together. system is the parameter people most often misuse: it is not another message in the transcript, it's a separate top-level channel for standing instructions — the model's role, rules, and persona. The analogy is a play: system is the stage direction that tells the actor who they are and how to behave; messages is the actual dialogue. Put "you are a terse DevOps assistant" in system, not in a user turn, and it carries the weight of an instruction rather than a passing request. And when that instruction is large and stable, giving system as a list of text blocks (rather than a bare string) lets you attach cache_control and stop paying to re-read the same prefix on every call.
These are the parameters you set on essentially every call. model and max_tokens are scalars; the interesting shape is in messages and system.
messages is a list of turns. Each turn has a role ("user" or "assistant") and content. Content can be a plain string, or — when you need images, documents, or tool results — a list of content blocks ({"type": "text", ...}, {"type": "image", ...}, and so on). system is separate from messages: it's top-level instructions, given as a string, or as a list of text blocks when you want to attach cache_control to cache a large stable prompt.
system; put the alternating user/assistant dialogue in messages. Because the API is stateless, the transcript is resent in full every call.
Read top to bottom: the wide band on top is system; the stacked rows below it are the messages transcript.
- The grey band is
system— the standing instructions (who the model is, the rules). It sits above the conversation, not inside it, and can be a plain string or a list of blocks so you can attachcache_control. - The stacked rows are
messages: each row is one turn with arole(userorassistant) and its content. Notice the roles alternate. - The bottom line is the beginner trap: the API keeps no memory, so this entire list is resent on every call — "remembering" earlier turns just means including them again.
In short: system = the stage direction (who to be); messages = the dialogue (what was said) — and you resend the whole script each time.
required.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call
import anthropic
client = anthropic.Anthropic() # reads ANTHROPIC_API_KEY from the environment
resp = client.messages.create(
model="claude-opus-4-8", # REQUIRED: which model answers
max_tokens=1024, # REQUIRED: hard cap on output tokens
system=[ # top-level instructions (list form → cacheable)
{
"type": "text",
"text": "You are a terse DevOps assistant. Answer in one sentence.",
"cache_control": {"type": "ephemeral"}, # cache this stable prefix
}
],
messages=[ # REQUIRED: the conversation
{"role": "user", "content": "What does 'kubectl get pods' show?"},
# content can also be a LIST of blocks, e.g. text + an image:
# {"role": "user", "content": [
# {"type": "text", "text": "What's in this screenshot?"},
# {"type": "image", "source": {"type": "url", "url": "https://.../shot.png"}},
# ]},
],
)
# content is a LIST of blocks — read only the text ones
print(next((b.text for b in resp.content if b.type == "text"), ""))
This is the smallest useful call — the three required parameters plus a system prompt. It needs pip install anthropic and an ANTHROPIC_API_KEY, and it makes a live request. Everything else in the lesson is an addition to this shape.
client = anthropic.Anthropic()builds the client and reads your key from the environment, so the key never appears in the code.modelandmax_tokensare the two required scalars: which Claude answers, and the hard ceiling on how many tokens it may produce.systemis given here as a list of text blocks (not a plain string) so acache_controlmarker can be attached — that caches the stable prompt prefix. A plain string works too when you don't need caching.messagesis the conversation. Each turn is{role, content}; content is a string, or (see the commented lines) a list of blocks when you add an image or document.
What the output means: The one-sentence answer prints. Because resp.content is a list of blocks, we pull the first text block rather than assuming a string.
Try this: Swap the system list for a plain string (system="...") and re-run — it still works; you just lose the ability to cache that prefix.
max_tokens is the output ceiling, and it can't exceed the model's max output (e.g. 128K on Opus 4.8, 64K on Sonnet/Haiku). Set it too low and the reply is truncated mid-thought with stop_reason == "max_tokens"; set it very high on a non-streaming call and the SDK may refuse it to avoid an HTTP timeout — stream instead.5 · Recipe — temperature vs top_p (and stop_sequences) intermediate
The most common cause of "my extraction pipeline gives a different answer every time I run it" is a single unset parameter — temperature — sitting at its default of 1. Sampling parameters are where a lot of flakiness (and a lot of wasted 400 errors) come from, so they're worth understanding rather than copy-pasting.
Here's what these knobs actually control. At every step the model produces a probability distribution over possible next tokens — "the next word is 78% likely to be X, 12% Y, 3% Z…". temperature decides how boldly it gambles on that distribution. Turn it toward 0 and the model almost always takes the single most likely token — flat, predictable, repeatable. Leave it at 1 (the default) and it samples across the full range, which is what you want for prose and brainstorming but is a menace for a task where you need the same answer twice. The intuition: temperature is the "how surprising am I allowed to be?" dial.
top_p is a second, different way to constrain that same gamble — instead of scaling the whole distribution, it lops off the long tail and only samples from the most-likely tokens that together add up to probability p. Because both knobs shape the same randomness, the golden rule is to set at most one of them. This is the classic beginner trap: setting temperature=0.2 and top_p=0.9 "to be extra careful" doesn't give you finer control, it gives you two levers fighting over the same distribution — and on the Anthropic API it's a frequent source of 400 errors. Anthropic's own guidance is to adjust temperature and leave top_p alone; reach for top_p only if you have a specific reason, and drop temperature when you do. Note also that temperature here ranges 0 to 1 (not the 0–2 you may have seen on other providers). A completely separate knob, stop_sequences, has nothing to do with randomness — it's a list of strings that end generation the instant the model writes one (the marker itself is not returned), and when one fires stop_reason becomes "stop_sequence".
temperature ranges 0 to 1. Near 0 the model almost always picks the most likely token — good for extraction, classification, and anything you want reproducible-ish. At 1 (the default) it samples across the full distribution — good for brainstorming and prose. stop_sequences is unrelated to randomness: it's a list of strings that, when the model generates one, cut the response off immediately (the stop string itself is not returned). When that happens, stop_reason is "stop_sequence" and stop_sequence tells you which one fired.
temperature concentrates the choice on the most-likely token (repeatable); default temperature=1 samples the whole curve (varied). top_p is a second lever on the same curve — set at most one.
Each side shows the model's probability distribution over the next token as a little bar chart; compare the shapes left versus right.
- The left blue bars are
temperature ≈ 0: one tall bar and the rest tiny — the model almost always takes the single most likely token, so output is repeatable. This is what you want for extraction and classification. - The right amber bars are the default
temperature = 1: the bars are flatter and closer in height, so the model samples across the full range — good for prose, a menace when you need the same answer twice. - The bottom line is the golden rule:
temperatureandtop_pare two levers on this same curve, so setting both just makes them fight — pick one (and on the Anthropic API, setting both is a common 400).
In short: Low temperature = one tall spike (repeatable); temperature 1 = a spread-out curve (varied). Never turn both randomness knobs at once.
sampling.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
model="claude-opus-4-8",
max_tokens=256,
temperature=0, # near-deterministic: extraction / classification
# top_p=0.9, # DON'T also set this — pick ONE knob, not both
stop_sequences=["\n\nEND"], # stop the moment the model writes this marker
system="Extract the error code. Output only the code, then write \n\nEND.",
messages=[{"role": "user",
"content": "Log: FATAL 2026-09-06 db timeout code=E5521 retry=3"}],
)
text = next((b.text for b in resp.content if b.type == "text"), "")
print("text:", repr(text))
print("stop_reason:", resp.stop_reason) # "stop_sequence" if the marker fired
print("stop_sequence:", resp.stop_sequence) # which sequence caused the stop
This call is tuned for a deterministic extraction task — pull one error code out of a log line. It shows the two sampling ideas you'll use most: a low temperature, and a stop_sequence that ends the reply at a marker.
temperature=0makes the model almost always pick the single most likely token — exactly what you want for extraction, where creativity is a bug, not a feature.- The commented
top_p=0.9line is a trap on purpose:temperatureandtop_pare two controls for the same thing, so you set one, never both. stop_sequences=["\n\nEND"]tells the model to stop the instant it writes that marker. The marker itself is not included in the returned text.- After the call,
resp.stop_reasonreports why it stopped andresp.stop_sequencesays which marker fired — that's how you confirm the stop sequence did the work rather than a natural end or the token cap.
What the output means: The extracted code prints, then stop_reason ("stop_sequence" if the marker fired) and the specific stop_sequence that caused it.
Try this: Delete the stop_sequences argument and re-run — the model finishes on its own and stop_reason becomes "end_turn" instead.
temperature for almost everything; reach for top_p only if you have a specific reason, and then drop temperature.stop_sequences is an input (strings you supply that end generation). stop_reason is an output field telling you why the model stopped: "end_turn" (finished naturally), "max_tokens" (hit your cap), "stop_sequence" (hit one of yours), "tool_use", or "refusal". Always read stop_reason before trusting the content.6 · Recipe — tools & tool_choice (force / allow / forbid) intermediate
Someone forces a tool, then writes resp.content[0].text to read the answer — and gets an empty string or a crash. It's one of the most common tool-use bugs, and it comes from not internalising a single fact: forcing a tool changes the shape of the response. This section is really about two parameters that work as a pair, and the surprising thing that happens when you turn the second one up.
The mental model: tools is you handing the model a labelled toolbox — each entry a name, a plain-English description, and a JSON schema describing the arguments it takes. Crucially, the model never runs your tool; it decides a tool is needed and hands you the filled-in arguments in a tool_use block, and your code does the actual work. tool_choice is the separate dial for how much say the model gets in whether to reach into that box at all — think of it as a spectrum from "use your judgement" to "you must use this exact tool" to "hands off the toolbox entirely."
That dial has a consequence people miss, and it's the whole reason this is a recipe rather than a one-liner: the moment you force a tool (with {"type":"tool"} or {"type":"any"}), the model stops answering in prose. There is no text block to read — the answer lives inside the tool_use block's arguments, and stop_reason flips to "tool_use" to tell you so. This is a feature, not a quirk: forcing a tool is exactly how you get guaranteed structured extraction — you know the model will emit a dict matching your schema instead of a paragraph you'd have to parse. It's the right move for "pull these fields out of this text" and the wrong move for open-ended chat, where forcing a tool just straitjackets a model that should be free to answer normally. Read stop_reason first, then read from the block it points you to.
tools is a list of tool definitions (each a name, a description, and a JSON-schema for its input). tool_choice controls whether and which tool the model uses. It has four modes:
tool_choice is a spectrum from none to a forced tool; when you force one, stop_reason becomes "tool_use" and the answer lives in a tool_use block, not a text block.
Read the top row left to right as a spectrum of control, then follow the two arrows down to see how the response shape depends on it.
- The top row is the four
tool_choicemodes, from "hands off the toolbox" (none, red) through the default (auto) to "you must use this exact tool" (tool, green).anymeans "must use some tool". - The left arrow shows the un-forced path: the model answers in a text block and
stop_reasonis"end_turn". - The right arrow shows the forced path (
anyortool): there is no prose — the answer arrives as a tool_use block andstop_reasonflips to"tool_use". This is the "aha": forcing is how you get guaranteed structured extraction, but you must read the tool_use block, notcontent[0].text.
In short: tool_choice runs from forbid to force; the instant you force, read the tool_use block because stop_reason is now "tool_use".
| tool_choice | Meaning |
|---|---|
{"type": "auto"} | Default. Model decides whether to use a tool at all. |
{"type": "any"} | Model must use one of the tools (its choice which). |
{"type": "tool", "name": "X"} | Model must use tool X — forced. |
{"type": "none"} | Model may not use any tool this turn. |
Any of these can also carry "disable_parallel_tool_use": true to force at most one tool call per turn (by default the model may request several at once). Here is a forced tool call — useful when you want structured extraction and you know the model should always fill in this one tool:
tool_choice.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call
import anthropic
client = anthropic.Anthropic()
record_incident = {
"name": "record_incident",
"description": "Record a parsed incident from an alert line.",
"input_schema": {
"type": "object",
"properties": {
"service": {"type": "string"},
"severity": {"type": "string", "enum": ["low", "high", "critical"]},
},
"required": ["service", "severity"],
},
}
resp = client.messages.create(
model="claude-opus-4-8",
max_tokens=512,
tools=[record_incident],
# FORCE this exact tool + at most one call this turn:
tool_choice={"type": "tool", "name": "record_incident",
"disable_parallel_tool_use": True},
messages=[{"role": "user",
"content": "CRITICAL checkout-api is down, 500s everywhere"}],
)
# a forced tool means stop_reason == "tool_use" and a tool_use block with the args
tool_use = next((b for b in resp.content if b.type == "tool_use"), None)
print(resp.stop_reason) # "tool_use"
print(tool_use.name, tool_use.input) # record_incident {'service': 'checkout-api', ...}
This forces the model to call one specific tool — a clean way to get guaranteed structured data out of free text. The magic is entirely in tool_choice; the tool definition itself is an ordinary name + description + JSON schema.
tools=[record_incident]makes one tool available. Itsinput_schemadeclares the fields (service,severitywith anenum) the model must fill in.tool_choice={"type": "tool", "name": "record_incident"}forces that exact tool — the model won't answer in prose, it will emit atool_useblock with the parsed arguments."disable_parallel_tool_use": Truecaps it at a single tool call this turn (without it, the model may request several at once).- Because a tool was forced,
resp.stop_reasonis"tool_use"and you read the answer from thetool_useblock's.input— not from a text block.
What the output means: stop_reason is "tool_use", and the tool's .input is a dict like {'service': 'checkout-api', 'severity': 'critical'} — structured, not prose.
Try this: Change tool_choice to {"type": "auto"} and re-run. Now the model decides whether to call the tool — on a clear incident it usually still does, but it may answer in text instead.
{"type":"tool"} or {"type":"any"} the model will emit a tool_use block and stop_reason becomes "tool_use" — it won't answer in prose. Don't force a tool and then look for a text answer; read the tool_use block instead. Forcing is great for guaranteed structured extraction, wrong for open-ended chat.7 · Recipe — extended thinking advanced
Two things trip everyone up here, and both throw a 400 in production if you get them wrong: the old budget_tokens syntax is gone, and you cannot lower the temperature while thinking is on. Extended thinking is a genuinely powerful lever — but it's also the one parameter with rules that changed recently, so muscle memory from a tutorial written a year ago will now fail.
What thinking actually buys you: on a hard, multi-step problem — planning a migration, chasing a subtle bug, a maths proof — the model reasons privately before committing to an answer, the way you'd scribble on a whiteboard before speaking. That scratch-work costs extra tokens and latency, which is exactly why it's the wrong tool for a lookup or a one-word classification and the right one when a wrong answer is expensive. The way you turn it on today is adaptive thinking ({"type": "adaptive"}), where the model itself decides how deep to go, and you steer the overall depth with a separate dial, output_config.effort (low through max).
Now the two gotchas, stated plainly so you never hit them. First, the pre-2026 form {"type": "enabled", "budget_tokens": N} — where you handed the model a fixed thinking budget — is removed on Opus 4.8 / 4.7 and returns a 400; the replacement is adaptive thinking plus effort, and there's no fixed token budget to set anymore. Second, and this is the subtle one: extended thinking always runs at temperature=1. You cannot combine it with a lowered temperature, or with top_p or top_k — those sampling parameters are incompatible with thinking and the request 400s. The reflex to "make thinking more focused by dropping the temperature" is precisely the trap. So the whole recipe collapses to one sentence: turn thinking on, tune depth with effort, and leave every sampling knob untouched.
Extended thinking lets the model reason privately before it answers — a real lever on hard, multi-step problems (planning, tricky debugging, math). On current models you turn it on with adaptive thinking, where the model itself decides how much to think, and you tune the overall depth with output_config.effort.
Two accuracy points that trip people up. First, on Opus 4.8 / 4.7 the old {"type": "enabled", "budget_tokens": N} form is gone — it returns a 400. Use {"type": "adaptive"}. Second, thinking runs at temperature=1 — you cannot combine extended thinking with a lowered temperature (or with top_p/top_k); those sampling params are incompatible with thinking. So the recipe is: turn thinking on, leave sampling alone.
{"type":"adaptive"} and dial depth with output_config.effort. Thinking always runs at temperature=1; passing any sampling knob alongside it is a 400.
Follow the top arrows left to right — that's the order the response comes back in — then note the two annotations above and below.
- The grey box is the
thinkingblock: the model reasons privately, like scribbling on a whiteboard, before it answers. That scratch-work costs extra tokens and latency. - The green box is the final
textanswer the model commits to after thinking. Your loop reads blocks bytypeto separate the two. - The dashed arrow on top is the only depth control you use —
output_config.effort(lowthroughmax) — not temperature. - The red box is the trap: setting
temperature,top_portop_kwhile thinking is on returns a 400, because thinking always runs attemperature=1.
In short: thinking block → text answer; tune depth with effort, and leave every sampling knob untouched or the request 400s.
thinking.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes a real API call
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
model="claude-opus-4-8",
max_tokens=8000,
thinking={"type": "adaptive", "display": "summarized"}, # model decides how much
output_config={"effort": "high"}, # low | medium | high | max — the depth dial
# NOTE: do NOT also set temperature / top_p / top_k here — thinking runs at
# temperature=1 and rejects those sampling params.
messages=[{"role": "user",
"content": "Plan a zero-downtime Postgres major-version upgrade."}],
)
for block in resp.content:
if block.type == "thinking":
print("[thinking]", block.thinking[:80], "...")
elif block.type == "text":
print("[answer]", block.text)
This turns on extended thinking — private reasoning before the answer — for a genuinely hard planning task. Two things are load-bearing here: the adaptive type, and the absence of any sampling parameter.
thinking={"type": "adaptive"}is the current way to enable thinking on Opus 4.8 / 4.7 — the model decides how much to think. The old{"type": "enabled", "budget_tokens": N}form is gone and returns a 400.output_config={"effort": "high"}is the depth dial —lowthroughmax. This is how you control reasoning cost, not temperature.- There is deliberately no
temperature/top_p/top_khere: extended thinking always runs attemperature=1, and combining it with a custom sampling value is a 400. - The reply comes back as blocks — a
thinkingblock (the summarised reasoning) followed by thetextanswer — so the loop checksblock.typefor each.
What the output means: A truncated summary of the model's reasoning prints first (the thinking block), then the actual upgrade plan (the text block).
Try this: Lower effort to "low" and compare the depth of the plan against the token count in resp.usage — that's the quality-vs-cost trade thinking gives you.
output_config.effort, not with temperature. Extended thinking always runs at temperature=1; passing a custom temperature (or top_p/top_k) alongside it is a 400. Reach for thinking on genuinely hard problems — it costs more tokens and latency, so it's overkill for a lookup or a classification.8 · Validate before you send (this one runs offline) professional
A 400 from a bad parameter combination isn't free — it costs you a round trip, and in production it costs you a failed request, a retry, and possibly a 2am page. The frustrating part is that almost every one of these mistakes is knowable before the request ever leaves your machine: you don't need Claude to tell you that you set both temperature and top_p, or that max_tokens is bigger than the model can produce. You already have all the information.
This is the same instinct as a compiler catching a type error before you run the program, or a form validating an email field before it hits the server. The principle is shift the check left — move it as close to where the mistake is made as you can, so it fails cheaply and instantly instead of expensively and remotely. A malformed params dict caught by a unit test costs nothing; the same dict caught as a 400 in front of a user costs a request, latency, and trust.
The common trap this guards against is treating the API as your validator — "I'll just send it and see if it errors." That works exactly once, in a REPL, on a quiet afternoon. It does not scale to a codebase with fifty call sites where a subtle combination (thinking on and a lowered temperature, say) only trips on the one route nobody tested. The helper below is deliberately pure stdlib — no API key, no network — so it drops straight into a test and lints every call site offline. It doesn't try to be clever; it just encodes the handful of rules this lesson taught and returns a plain list of human-readable warnings.
Most parameter mistakes are catchable before you spend a request: both temperature and top_p set, a max_tokens that exceeds the model's output cap, or extended thinking combined with a non-default temperature. The helper below is pure stdlib — no API key, no network — and runs with a plain python file.py. It inspects a params dict and returns a list of warnings, so you can lint your call sites in a test.
Both paths start from the same params dict on the left; the fork shows two ways to discover a bad parameter combination.
- The green top path is "shift left": the dict goes into
validate_params()— pure stdlib, no network — which returns human-readable warnings you fix instantly, before spending anything. - The red bottom path is treating the API as your validator: you send the request, pay for a round trip and tokens, and only then learn it was invalid via a remote 400.
- The payoff is the contrast in the two right-hand boxes: same mistake, but caught cheaply and instantly on top versus slowly and expensively on the bottom. That's why the check belongs in a unit test.
In short: Validate offline and a bad params dict costs nothing; let the API catch it and it costs a request, latency, and maybe a 2am page.
validate_params.py# Catch the common messages.create mistakes locally, before spending a request.
# Model output caps as of this writing (verify current values in the docs):
MODEL_MAX_OUTPUT = {
"claude-opus-4-8": 128_000,
"claude-sonnet-4-6": 64_000,
"claude-haiku-4-5": 64_000,
}
def validate_params(params):
"""Return a list of human-readable warnings for a messages.create params dict.
Pure stdlib; does not call the API."""
warnings = []
model = params.get("model")
if not model:
warnings.append("model is REQUIRED and missing")
if "max_tokens" not in params:
warnings.append("max_tokens is REQUIRED and missing")
if not params.get("messages"):
warnings.append("messages is REQUIRED and empty/missing")
# don't set both temperature and top_p
if "temperature" in params and "top_p" in params:
warnings.append("both temperature and top_p set — pick ONE, not both")
# max_tokens must fit the model's output cap
cap = MODEL_MAX_OUTPUT.get(model)
mt = params.get("max_tokens")
if cap is not None and isinstance(mt, int) and mt > cap:
warnings.append(f"max_tokens={mt} exceeds {model} output cap of {cap}")
# extended thinking requires temperature == 1 (i.e. leave it unset)
thinking = params.get("thinking")
thinking_on = isinstance(thinking, dict) and thinking.get("type") in ("adaptive", "enabled")
if thinking_on and params.get("temperature", 1) != 1:
warnings.append("thinking is on but temperature != 1 — thinking runs at temperature=1")
if thinking_on and ("top_p" in params or "top_k" in params):
warnings.append("thinking is on but top_p/top_k set — incompatible with thinking")
return warnings
bad = {
"model": "claude-opus-4-8",
"max_tokens": 200_000, # over the 128K cap
"messages": [{"role": "user", "content": "hi"}],
"temperature": 0.2, # both temperature ...
"top_p": 0.9, # ... and top_p
"thinking": {"type": "adaptive"}, # thinking on, but temperature != 1
}
for w in validate_params(bad):
print("WARN:", w)
print("ok:", validate_params({
"model": "claude-opus-4-8", "max_tokens": 1024,
"messages": [{"role": "user", "content": "hi"}],
}))
WARN: both temperature and top_p set — pick ONE, not both
WARN: max_tokens=200000 exceeds claude-opus-4-8 output cap of 128000
WARN: thinking is on but temperature != 1 — thinking runs at temperature=1
WARN: thinking is on but top_p/top_k set — incompatible with thinking
ok: []
Unlike the four blocks above, this one is pure Python — no key, no network — so it runs with a plain python validate_params.py. It's a pre-flight linter: hand it a params dict and it lists the mistakes that would otherwise cost you a wasted (or 400'd) request.
- It first checks the three required parameters are present —
model,max_tokens,messages. - Then the classic mutual-exclusion bug: both
temperatureandtop_pset. Pick one. MODEL_MAX_OUTPUTholds each model's output cap, so it can flag amax_tokensthat exceeds what the model can actually produce.- Finally the thinking rules: if thinking is on,
temperaturemust be 1 andtop_p/top_kmust be unset — otherwise the real API 400s. The deliberately-brokenbaddict trips all of these at once.
What the output means: Four WARN: lines for the broken dict (both-sampling, over-cap, thinking+temp, thinking+top_p), then ok: [] for the clean one — an empty warning list means the params are safe to send.
Try this: Drop this into a unit test over your real call sites. Catching a bad params dict offline costs nothing; catching it as a 400 in production costs a request and a page.
validate_params into a unit test catches the whole class of "why did my request 400?" bugs without ever hitting the network. It's the cheapest possible check — no key, no tokens, no latency.9 · Tech-lead — operational knobs & a house policy tech-lead
When you're the lead, the failure mode isn't a bad parameter on one call — it's fifty engineers each guessing at stream, metadata, and betas in fifty slightly different ways. The operational knobs are the group where inconsistency, not incorrectness, is the real cost. Any single call site setting them "wrong" is survivable; a codebase where nobody agrees on them is a slow-motion incident waiting for the day support asks "which user made this call?" and the answer is "we don't log that."
What makes this group different is that these parameters don't change what the model says at all — they govern how the request is delivered, tracked, and billed. That's precisely why they should be a platform decision, not a feature decision. The content parameters (messages, system, tools) belong to whoever is building the feature and vary per request. The operational parameters are cross-cutting: streaming behaviour, per-user tracking, service tier, beta flags. Leave those to individual judgement and you get drift; centralise them and every engineer inherits the right behaviour for free.
The concrete mechanism a good lead reaches for is a thin house wrapper — one function that everyone calls instead of messages.create directly, which fills in the model default, a sane max_tokens, the streaming decision, and the metadata.user_id, and runs the validate_params linter from §8 in that one place. The mistake to avoid is the opposite instinct — baking these into a giant shared config object with thirty toggles, or worse, letting beta flags accumulate uncommented across the codebase until nobody remembers which are still needed. The policy below is the short version: stream the big and interactive calls, tag every production request with a hashed user id, turn thinking on by route, treat service_tier and betas as deliberate opt-ins you verify against current docs, and put the whole lot behind one wrapper.
The last group of parameters doesn't change what the model says — it changes how the request is delivered, tracked, and billed. A lead sets a house policy for these so every call site is consistent:
messages.create directly; it fills the operational defaults (streaming, metadata.user_id) and runs the §8 linter in one place, so consistency is inherited, not policed per call site.
Read left to right: many feature call sites converge on one wrapper, which then makes the real API call.
- The blue boxes on the left are the many places in the codebase that need Claude — dozens of call sites across features.
- The green box is the single house wrapper everyone calls instead of
messages.createdirectly. It fills the cross-cutting operational knobs (the streaming decision,metadata.user_id) and runsvalidate_paramsfrom §8 in that one spot. - The grey box is the real endpoint the wrapper finally calls.
- The payoff is in the bottom lines: content parameters (
messages,system,tools) stay per-feature, but the operational knobs are centralised — so every engineer inherits the right behaviour for free instead of guessing.
In short: Funnel every call through one thin wrapper — feature teams own the content, the platform owns the operational knobs.
Operational parameter policy
streamfor anything interactive or large. Streaming isn't just UX — it's how you avoid HTTP timeouts on bigmax_tokenscalls. Policy: stream any user-facing response and any call withmax_tokensabove ~16K; use.get_final_message()to still get the assembled reply and usage.metadata.user_idon every production call. Attach a stable, hashed per-end-user identifier (never raw PII) so abuse and rate-limit issues can be traced to a tenant. It costs nothing and it's the field support will ask you for.thinking+effortby route, not globally. Turn thinking on for the hard routes (planning, debugging) and leave it off for lookups and classification. Standardise theeffortlevel per route so cost is predictable.service_tieronly if you have an arrangement. It requests a priority / standard / batch service level; the exact accepted values and behaviour evolve, so treat it as opt-in and verify the current options in the docs before wiring it in.betasdeliberately, never by default. Beta features live onclient.beta.messageswith an explicit flag string. Pin the flag, document why you need it, and plan to remove it when the feature goes GA — don't leave stale beta headers scattered across the codebase.- Centralise the defaults. Wrap
messages.createin one thin house function that fillsmodel, sanemax_tokens,metadata, and the streaming decision. Thenvalidate_paramsfrom §8 runs in that one place.
The through-line: the content parameters (messages, system, tools) are per-request and belong to the feature; the operational parameters (stream, metadata, service_tier, betas) are cross-cutting and belong to the platform. Put the second set behind one wrapper and every engineer inherits the house policy for free.
service_tier's accepted values and betas' flag strings change as features ship and graduate. This lesson deliberately doesn't hard-code them: when you reach for either, check the current Anthropic docs for the exact string rather than copying one that may have rotated.Exercise AP8.1 — Read the table, build the call
Context: Choosing parameters is a design decision, not a default-copy. A classification task wants a cheap model, a tiny output, and near-zero randomness — and every extra knob you add should earn its place.
Your task: Without looking at the recipes, write a messages.create call for a classification task (label a support ticket): pick the right model, a small max_tokens, the right temperature, and decide whether you need thinking. Justify each non-required parameter in one sentence and delete any you can't justify.
Requirements:
- Pick a fast, cheap model appropriate to a simple labeling task
- Set a small
max_tokensthat fits just the label - Choose a low/zero
temperaturefor stable, repeatable labels - Decide (and justify) that
thinkingis unnecessary for classification - Write a one-sentence justification per non-required parameter — and drop any that has none
💡 Hint: Start from the three required params and add a knob only when you can name the behavior it buys; classification rarely needs more than a low temperature.
Exercise AP8.2 — Extend the linter
Context: A linter is only as good as its coverage. Two more checks catch config mistakes that silently waste requests: a forced tool with no tools defined, and a huge output with streaming off.
Your task: Extend validate_params with two checks: (1) warn if tool_choice is {"type":"tool"} but tools is empty or missing, and (2) warn if max_tokens is very large (say > 16000) but stream is not True. Run it on a params dict that trips both.
Requirements:
- Add a check for a forced
tool_choicewith no usabletoolslist - Add a check for a large
max_tokens(> 16000) withoutstream=True - Preserve the linter's existing warnings — extend, don't replace
- Both new warnings are appended as clear strings to the returned list
- Demonstrate on a dict that trips both and confirm both warnings print
💡 Hint: Read tool_choice, tools, and stream defensively so a missing key is treated as absent, not an exception.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Every Claude call rests on three required fields; everything else is a knob you opt into. Knowing the minimum keeps you from cargo-culting parameters you don't understand.
Your task: Write a messages.create() call using only the three required parameters — model, max_tokens, and messages — and explain in a comment why system is optional.
Requirements:
- Pass exactly
model,max_tokens, andmessages— nothing else max_tokensis understood as a hard cap on the output lengthmessagesis a list of role/content dicts with a single user turn- A comment states that without
systemthe model falls back to its defaults - Print the reply's text from the first content block
💡 Hint: Reach for a current model id such as claude-sonnet-4-6; resist adding temperature or anything else until a task demands it.
Show solution
system is optional; the three required fields are the minimum to get a reply.
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
model="claude-sonnet-4-6", # required
max_tokens=256, # required: hard cap on output
messages=[{"role": "user", "content": "Define idempotency."}], # required
)
# system is optional: with no role/rules given, the model uses its defaults.
print(resp.content[0].text)
Context: temperature and top_p are two different ways to shape randomness, and Anthropic's guidance is to tune only one. For deterministic extraction you want the temperature floor.
Your task: Show one call that lowers temperature for a deterministic extraction, plus a comment stating the rule about setting temperature and top_p together.
Requirements:
- Set
temperature=0for focused, repeatable output - A comment states you should not also set
top_p— pick one knob - Use
stop_sequencesto cut the output at a natural boundary - The task is a small, single extraction (one fact out of one sentence)
- Print both the text and the response's
stop_reason
💡 Hint: Tuning both sampling knobs at once fights itself — lower temperature alone, and let a stop_sequences entry end the reply cleanly.
Show solution
Tune one sampling knob at a time. Setting both temperature and top_p is discouraged; pick one.
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
model="claude-sonnet-4-6",
max_tokens=64,
temperature=0, # focused, repeatable; do NOT also set top_p
stop_sequences=["\n\n"], # stop at first blank line
messages=[{"role": "user", "content": "Extract the year: Founded in 1994."}],
)
print(resp.content[0].text, resp.stop_reason)
Context: When you force a tool, Claude's answer is no longer prose — the structured arguments arrive in a tool_use block and stop_reason flips to tool_use. Reading content[0].text then gets you nothing.
Your task: Define one tool and force it with tool_choice, then explain in a comment why you must read the tool_use block rather than content[0].text.
Requirements:
- Declare a tool with a
name,description, and aninput_schema - Force it via
tool_choice={"type": "tool", "name": ...} - Recognize that forcing a tool drives
stop_reasontotool_use - Scan
resp.contentfor the block whosetypeistool_useand read itsinput - A comment explains the arguments live in that block, not in a text block
💡 Hint: Never index content[0] blindly — iterate the blocks and branch on block.type so you pick up the arguments wherever they land.
Show solution
Forcing a tool sets stop_reason to "tool_use"; the arguments live in a tool_use block, not in text.
import anthropic
client = anthropic.Anthropic()
tools = [{
"name": "record_temp",
"description": "Record a temperature reading.",
"input_schema": {"type": "object",
"properties": {"celsius": {"type": "number"}},
"required": ["celsius"]},
}]
resp = client.messages.create(
model="claude-sonnet-4-6", max_tokens=256, tools=tools,
tool_choice={"type": "tool", "name": "record_temp"},
messages=[{"role": "user", "content": "It is 21 degrees C."}],
)
# stop_reason == 'tool_use' -> the args are in a tool_use block:
for block in resp.content:
if block.type == "tool_use":
print(block.input) # {'celsius': 21}
Context: Extended thinking lets Claude reason before answering, but it changes the rules: thinking runs at temperature 1 and rejects the sampling knobs. Depth is dialed with output_config.effort, not temperature.
Your task: Enable extended thinking and set the reasoning depth via output_config, and note the two constraints thinking imposes on sampling parameters.
Requirements:
- Turn thinking on with an adaptive
thinkingsetting ({"type": "adaptive"}) - Set reasoning depth through
output_configwith aneffortoflow|medium|high|max - Do not pass
temperature,top_p, ortop_k— a comment notes thinking runs at temperature 1 and rejects them (a 400) - Give
max_tokensreal headroom for a reasoning task - Print only the final
textblocks from the response
💡 Hint: Treat output_config.effort as the single dial for how hard the model thinks, and leave every sampling parameter unset while thinking is on.
Show solution
Thinking uses {"type": "adaptive"}; it runs at temperature 1 and rejects temperature/top_p/top_k. Depth is tuned with output_config.effort.
import anthropic
client = anthropic.Anthropic()
resp = client.messages.create(
model="claude-opus-4-8",
max_tokens=2048,
thinking={"type": "adaptive", "display": "summarized"},
output_config={"effort": "high"}, # low | medium | high | max
# NOTE: with thinking on, do NOT pass temperature/top_p/top_k -> 400.
messages=[{"role": "user", "content": "Prove sqrt(2) is irrational."}],
)
for b in resp.content:
if b.type == "text":
print(b.text)
Context: A bad parameter combination burns a request (and money) before you learn it was invalid. A small offline linter catches the classic mistakes at your boundary instead of at the API.
Your task: Implement validate_params(p, model_cap) that returns warnings for setting both temperature and top_p, for max_tokens above the model cap, and for thinking enabled together with a non-default temperature.
Requirements:
- Warn when both
temperatureandtop_pare present - Warn when
max_tokensexceeds the suppliedmodel_cap - Warn when
thinkingis on buttemperatureis not the required default of 1 - Also flag any missing required param (
model,max_tokens,messages) - Return a list of warning strings; run fully offline against a sample dict
💡 Hint: Read the params dict defensively with .get() so a missing key is just another warning rather than a KeyError.
Show solution
This is the lesson's linter: catch config mistakes before spending a request.
def validate_params(p, model_cap):
warns = []
if "temperature" in p and "top_p" in p:
warns.append("set temperature OR top_p, not both")
if p.get("max_tokens", 0) > model_cap:
warns.append(f"max_tokens {p['max_tokens']} > cap {model_cap}")
if p.get("thinking") and p.get("temperature", 1) != 1:
warns.append("thinking requires temperature=1 (drop it)")
for k in ("model", "max_tokens", "messages"):
if k not in p:
warns.append(f"missing required param: {k}")
return warns
bad = {"model": "claude-opus-4-8", "max_tokens": 200000,
"messages": [], "temperature": 0.3, "top_p": 0.9}
print(validate_params(bad, model_cap=128000))
Context: When a platform fronts every Claude call, per-team discipline doesn't scale — policy has to be enforced in one sanitizer that rewrites requests before they leave the building.
Your task: Enforce a house policy on incoming params: default max_tokens if unset, cap it at the model's limit, strip top_p when temperature is present, and require a metadata.user_id for abuse tracking, returning the sanitized params.
Requirements:
- Look the cap up per-model and clamp
max_tokensto it (defaulting it first when absent) - Drop
top_pwhenevertemperatureis set, so only one sampling knob survives - Reject the request (raise) if
metadata.user_idis missing - Return a copy — do not mutate the caller's dict
- Runs offline; demonstrate it on a params dict that trips every rule
💡 Hint: Copy the dict up front, then apply each rule as an independent transform so adding the next policy later is just one more step.
Show solution
A single sanitizer keeps every team inside the guardrails without hand-review.
CAPS = {"claude-opus-4-8": 128000, "claude-sonnet-4-6": 64000,
"claude-haiku-4-5": 64000}
def house_policy(p):
p = dict(p)
cap = CAPS.get(p["model"], 4096)
p.setdefault("max_tokens", 1024)
p["max_tokens"] = min(p["max_tokens"], cap) # never exceed the cap
if "temperature" in p:
p.pop("top_p", None) # one sampling knob
md = p.setdefault("metadata", {})
if "user_id" not in md:
raise ValueError("metadata.user_id required for abuse tracking")
return p
out = house_policy({"model": "claude-sonnet-4-6", "max_tokens": 999999,
"temperature": 0.2, "top_p": 0.8,
"metadata": {"user_id": "u-42"}})
print(out["max_tokens"], "top_p" in out) # 64000 False
✓ Checkpoint — you can move on when you can…
- Name the three required parameters and give a sane default for every other one.
- Shape
messages(roles + content blocks) andsystem(string or cacheable list) correctly. - Explain why you never set both
temperatureandtop_p. - Distinguish
stop_sequences(input) fromstop_reason(output). - Force, allow, and forbid tools with
tool_choice, and disable parallel calls. - Turn on adaptive thinking correctly and state why you can't also lower the temperature.
- Name the operational knobs (
stream,metadata,service_tier,betas) and a house policy for each.
Knowledge check check yourself
Why should you never set both temperature and top_p, and what do they each control?
Show answer
top_p does nucleus sampling over the top tokens summing to probability p), so setting both is contradictory and is a common source of 400 errors. Pick one sampling lever.What happens if max_tokens is set too low, and why can setting it very high on a non-streaming call also fail?
Show answer
stop_reason == "max_tokens". Set very high on a non-streaming call and the SDK may refuse to avoid an HTTP timeout on a long generation; the fix is to stream (stream=True) so tokens arrive incrementally.