Reasoning models & test-time compute

1 · What a reasoning model is essential

Here's a moment you may have already lived. You ask a chatbot a genuinely hard question — a multi-step word problem, a subtle bug, a proof — and it answers instantly, confidently, and wrong. It didn't stop to think; it just pattern-matched the first plausible-looking words and blurted them out. Then you try a "thinking" model on the same question, wait a few seconds while it churns, and get the right answer. The difference between those two experiences is this lesson.

The analogy that makes it click: imagine a student sitting a maths exam. One student reads each question and writes the answer straight down — fast, but they trip on anything that needs working. The other student scribbles on scratch paper first — lays out the steps, checks the arithmetic, catches their own mistakes — and only then writes the final answer in the box. The scratch paper isn't the answer; the examiner never grades it. But it's why the second student's answer is right. A reasoning model is the second student: the "scratch paper" is a stream of private thinking tokens it generates before the visible reply.

Now the precise definition, which will land because you have the picture. A reasoning model is a model trained to first emit an internal chain of thinking tokens — working the problem out step by step — and only then produce the answer you see. Those thinking tokens are genuine model output: real compute, generated one at a time, spent before the answer begins. You typically don't see them, but they cost you money and they cost you latency. In exchange, on hard problems, the answer is markedly more likely to be correct.

The mistake almost everyone makes first is assuming "reasoning model = strictly better model, always use it." It isn't. On "what's the capital of France?" the thinking tokens buy you nothing — the fast student already knew it — while still charging you for the scratch paper and making the user wait. A reasoning model is not an upgrade you leave on; it is a tool for a specific class of problem (hard, multi-step, correctness-critical). The entire rest of this lesson is about telling those problems apart from the easy majority and spending thinking only where it pays.

A normal model reads your prompt and starts writing the answer immediately — one fast pass. A reasoning model first produces a private chain of thinking tokens — it works the problem out step by step — and only then writes the visible answer. Those thinking tokens are extra compute spent before the answer appears. You usually don't see them, but you pay for them, and they wait for them: more thinking means a slower, pricier, but often more correct reply on hard problems.

The setting that turns this on and up is the thinking / effort control introduced in Ch 02 · Prompting. This lesson is about when to reach for it and how much to spend.

Compare that to the fast path: question → answer, no thinking budget, cheap and low-latency. For an easy question the fast path is the right tool; spending thinking tokens on "what's 2+2" just burns money and time for no gain.

2 · Extended & adaptive thinking essential

Picture two ways a manager could handle "how much time should this take?" Manager A gives every task on the team a fixed 30 minutes — the trivial ones waste 25, the hard ones blow through and fail anyway. Manager B instead says "you may spend up to two hours; use what the task actually needs, no more." Manager B gets the trivial tasks back in minutes and lets the genuinely hard one soak up the time it deserves. That's the difference between a fixed thinking budget and an adaptive one, and it's the whole idea of this section.

Two words to keep straight, because they're easy to blur. Extended thinking is about the ceiling: you allow the model a larger allowance of thinking tokens than the default, so it can reason further before answering. Adaptive thinking is about how much of that allowance the model actually uses: a well-behaved reasoning model spends little on an easy question and a lot on a hard one, all under the same ceiling. You set the ceiling once; the model decides, per request, how far to go — which is exactly Manager B's "up to two hours, use what you need."

Why does this matter enough to name it? Because the alternative — a fixed spend on every call — is the naive setting that quietly wrecks your bill. If you hard-code "always think hard," every "yes/no" question drags the full budget along. Adaptive thinking is the mechanism that lets one setting serve both a one-line lookup and a proof, without you writing branching logic for each.

The one mental model that prevents most cost surprises: thinking tokens are spent before the answer and are billed like output tokens. This trips people up because the thinking is invisible — you see a 2,000-token answer and assume you paid for 2,000 tokens. But if the model burned 6,000 thinking tokens getting there, you're billed for ~8,000 and you waited proportionally longer for the reply to even start. The common mistake is budgeting from the visible answer length alone; always budget from thinking plus answer. The runnable lab below turns that into exact dollars.

Extended thinking means you allow the model a larger budget of thinking tokens so it can reason further before answering. Adaptive thinking means the model spends as much as the problem needs up to that budget — an easy question finishes quickly; a hard one uses more of the allowance. You set the ceiling; the model decides how much of it to use.

The key mental model: thinking tokens are spent before the answer, and they are billed like output tokens. A 2,000-token answer with a 6,000-token thinking budget can cost like an 8,000-token response and take proportionally longer to arrive.

thinking_cost.pydef response_cost(thinking_tokens, answer_tokens, price_per_1k=0.015):
    """Thinking tokens are billed before the answer even starts."""
    total = thinking_tokens + answer_tokens
    return {
        "thinking_tokens": thinking_tokens,
        "answer_tokens": answer_tokens,
        "billed_tokens": total,
        "cost_usd": round(total / 1000 * price_per_1k, 5),
    }

fast = response_cost(thinking_tokens=0,    answer_tokens=2000)
deep = response_cost(thinking_tokens=6000, answer_tokens=2000)
print("fast:", fast)
print("deep:", deep)
print("deep costs", round(deep["cost_usd"] / fast["cost_usd"], 1), "x the fast reply")

fast: {'thinking_tokens': 0, 'answer_tokens': 2000, 'billed_tokens': 2000, 'cost_usd': 0.03}
deep: {'thinking_tokens': 6000, 'answer_tokens': 2000, 'billed_tokens': 8000, 'cost_usd': 0.12}
deep costs 4.0 x the fast reply

3 · Test-time compute — buying accuracy at inference intermediate

For years the only known way to make a model smarter was to make it bigger — more parameters, more training data, more GPUs for months. That's expensive, slow, and something you do once, long before any user shows up. Then a quieter idea proved surprisingly powerful: leave the model the same size, but let it work harder on each individual question at the moment it's asked. Give it room to think, or let it try several times and pick the best. This shift — from spending compute at training time to spending it at inference (test) time — is why "thinking" models suddenly leapt ahead on hard reasoning, and it has a name: test-time compute.

The everyday analogy is a person taking a hard exam. Their raw ability — everything they learned before walking in — is fixed; that's the "training." But whether they rush and hand in after ten minutes, or use the full three hours to check their work, dramatically changes their score on the hard problems. Same brain, more effort in the room, better result. Test-time compute is buying that extra effort-in-the-room for the model, per question, with tokens.

Precisely: test-time compute is any technique that spends more inference-time compute to raise answer quality — the two main forms being more thinking tokens on a single attempt (thinking longer) and several independent attempts you vote over (which you'll meet as self-consistency in §6). On genuinely hard tasks it reliably lifts accuracy. On easy tasks it does almost nothing, because the quick answer was already right — the exam-taker who already knows "2+2" gains nothing from three extra hours.

The trap to internalize now, before you ever turn the dial: the accuracy-versus-compute curve bends over. It shows diminishing returns — the first chunk of extra thinking buys a big jump, and each chunk after that buys less than the one before. This is the single most important shape in the whole lesson, because it's why "just max out thinking on everything" is always the wrong instinct: you pay linearly for compute while accuracy crawls up asymptotically. The goal is never "maximum compute"; it's "just enough compute for this task's difficulty." The lab below makes that curve numeric so you can feel exactly where it flattens.

Test-time compute is the idea behind all of this: instead of only making the model bigger at training time, you spend more compute at inference (test) time — more thinking tokens, or several attempts you vote over. On hard reasoning tasks this reliably lifts accuracy: the model that "thinks longer" gets more of them right. On easy tasks it does almost nothing, because the fast answer was already correct.

The classic curve: accuracy rises with compute but with diminishing returns — each extra unit of thinking helps less than the last. So the goal is never "max everything"; it's "enough compute for this task's difficulty."

test_time_compute.pydef accuracy(base_acc, compute_units):
    """Each extra unit of compute closes half the remaining gap to 1.0.
    Deterministic: no randomness, so the curve is reproducible."""
    acc = base_acc
    for _ in range(compute_units):
        acc = acc + (1.0 - acc) * 0.5
    return round(acc, 4)

print("difficulty: HARD (base 0.40)")
for units in range(0, 6):
    print(f"  compute={units}: accuracy={accuracy(0.40, units)}")
print("gain 0->1 unit:", round(accuracy(0.40, 1) - accuracy(0.40, 0), 3))
print("gain 4->5 unit:", round(accuracy(0.40, 5) - accuracy(0.40, 4), 3))

difficulty: HARD (base 0.40)
  compute=0: accuracy=0.4
  compute=1: accuracy=0.7
  compute=2: accuracy=0.85
  compute=3: accuracy=0.925
  compute=4: accuracy=0.9625
  compute=5: accuracy=0.9812
gain 0->1 unit: 0.3
gain 4->5 unit: 0.019

The first unit of thinking bought +0.30 accuracy; the fifth bought +0.019. That shape is why a sensible policy caps effort rather than always maxing it.

4 · The effort dial: low → medium → high → max advanced

Nobody wants to think in raw token counts. "Should this request get 4,000 thinking tokens or 9,000?" is a question no product manager, and no sane engineer under deadline, wants to answer per prompt. So the interface you actually get is coarser and friendlier: a small set of named settings — commonly low, medium, high, max — that you turn like a dial. Each notch quietly maps to a bigger thinking budget underneath. You're still spending test-time compute from §3; you've just been handed a steering wheel instead of the raw hydraulics.

The analogy is the effort setting on a treadmill, or the power mode on a laptop. You don't specify watts; you pick "eco / balanced / performance," and the machine translates that into the real knobs. The crucial thing to understand is that one notch on this dial moves three things at once, and they don't all move in your favour: turning effort up raises accuracy on hard tasks (good), and raises cost (bad), and raises latency (bad). There is no setting that's better on all three — that's what makes it a genuine tradeoff and not just a "make it good" button.

Because of the diminishing-returns curve you met in §3, these notches are not evenly spaced in value. Going low→medium→high typically climbs the steep part of the curve — real accuracy for reasonable cost. Going high→max climbs the flat tail — you roughly double the spend to recover the last couple of accuracy points. That asymmetry is the whole reason the dial has a "max" that you should reach for rarely: it exists for the handful of problems where being right is worth almost any price, not as your default.

The mistake this section exists to prevent is treating the dial as "higher is safer, so crank it." Defaulting everything to high or max is the most common way a reasoning bill quietly triples: you're paying the flat-tail premium on every request, including the easy ones that didn't need any thinking at all. Read the table and the lab below as a map of where each notch earns its cost — and note the column "reach for it when…", because matching the notch to the task is the entire skill.

In practice you don't hand-set token counts — you turn an effort dial with a few named settings. Higher effort = a bigger thinking budget = more accuracy on hard tasks, but more cost and more latency. Think of it as one knob trading three things at once.

effort_dial.pyEFFORT = {
    # effort: (thinking_tokens, base cost multiplier, latency seconds)
    "low":    (0,     1.0, 0.8),
    "medium": (2000,  3.0, 2.5),
    "high":   (6000,  6.0, 6.0),
    "max":    (12000, 10.0, 12.0),
}

def tradeoff(hard_task_base_acc=0.40):
    rows = []
    for name, (think, cost_mult, latency) in EFFORT.items():
        # more thinking closes the accuracy gap, with diminishing returns
        acc = hard_task_base_acc
        for _ in range(think // 2000):
            acc = acc + (1.0 - acc) * 0.5
        rows.append((name, think, cost_mult, latency, round(acc, 3)))
    return rows

print(f"{'effort':7} {'think':>6} {'cost':>5} {'lat_s':>6} {'acc':>6}")
for name, think, cost, lat, acc in tradeoff():
    print(f"{name:7} {think:6d} {cost:5.1f} {lat:6.1f} {acc:6.3f}")

effort   think  cost  lat_s    acc
low          0   1.0    0.8  0.400
medium    2000   3.0    2.5  0.700
high      6000   6.0    6.0  0.925
max      12000  10.0   12.0  0.991

5 · Routing: fast model vs reasoning by difficulty advanced

Imagine a hospital that sent every single walk-in — sniffles, paper cuts, and heart attacks alike — straight to the top cardiac surgeon. The care would be excellent and the hospital would be bankrupt by Tuesday, with a waiting room full of people who needed a bandage. No emergency department works that way. They put a triage nurse at the door: a fast, cheap first assessment that sends the sprained wrist to a quick-care clinician and reserves the expensive specialists for the cases that actually need them. Routing a reasoning system is exactly this triage, and it is the single biggest cost lever you have.

Here's why it dominates every other optimization. The effort dial in §4 tunes how much you spend once you've already committed to the reasoning path — but routing decides whether you step onto that path at all. And the numbers are lopsided: in almost every real workload the overwhelming majority of traffic is easy (lookups, summaries, chit-chat), and thinking tokens are billed like output. If you send that easy majority through a reasoning model, you pay 5–10x for accuracy those questions never needed. Tuning effort saves cents; routing the 90% easy traffic away from reasoning entirely saves the bulk of the bill.

The mechanism is a difficulty classifier sitting in front of your models — the triage nurse. It doesn't need to be smart or expensive; it needs to be cheap and fast, because it runs on every request before any real work happens. A handful of heuristics often suffices: does the prompt contain hard-signal words like "prove", "derive", "debug", "why"; does it require math; is it multi-step or unusually long. Score those, compare to a threshold, and send anything below it to the fast path. When heuristics aren't enough you can use a small, fast model as the classifier — but even then the classifier must cost a tiny fraction of the reasoning call it's protecting, or it defeats its own purpose.

The mistake to avoid is skipping triage and "just using the good model for everything because it's simpler." It is simpler — and it's how teams discover a five-figure monthly bill for work a cheap model could have handled. A router is a few lines of code that pays for itself on the first hour of traffic. The lab below builds exactly such a classifier so you can watch trivial questions take the cheap path while the genuinely hard ones get escalated.

The single biggest lever is routing: send easy work to the fast, cheap path and only the hard work to the reasoning path. Most real traffic is easy; if you route it all through a reasoning model you pay 5–10x for accuracy you didn't need. A cheap difficulty classifier in front of the models pays for itself immediately.

router.pydef estimate_difficulty(task):
    """Cheap heuristic difficulty score in [0,1] — deterministic, no model call."""
    hard_signals = ("prove", "derive", "multi-step", "optimize", "why", "plan", "debug")
    score = 0.1
    if task.get("needs_math"):        score += 0.4
    if task.get("multi_step"):        score += 0.3
    text = task.get("text", "").lower()
    score += 0.1 * sum(sig in text for sig in hard_signals)
    return min(score, 1.0)

def route(task, threshold=0.5):
    d = estimate_difficulty(task)
    path = "reasoning (high effort)" if d >= threshold else "fast (low effort)"
    return round(d, 2), path

tasks = [
    {"text": "What is the capital of France?"},
    {"text": "Summarize this email in one line."},
    {"text": "Derive the optimal batch size, show the multi-step math.", "needs_math": True, "multi_step": True},
    {"text": "Why does the checkout service crash under load? Debug it.", "multi_step": True},
]
for t in tasks:
    d, path = route(t)
    print(f"diff={d:<4} -> {path:24} | {t['text'][:38]}")

diff=0.1  -> fast (low effort)        | What is the capital of France?
diff=0.1  -> fast (low effort)        | Summarize this email in one line.
diff=1.0  -> reasoning (high effort)  | Derive the optimal batch size, show th
diff=0.6  -> reasoning (high effort)  | Why does the checkout service crash un

6 · Production cost control: budgets & self-consistency professional

The reasoning bill that ruins a Monday looks like this: a traffic spike hits at 9am, every hard request quite reasonably asks for high effort, and by noon you've spent the entire day's budget — so the afternoon's requests either fail outright or you're frantically flipping settings in production while users watch spinners. Free-floating effort has no brakes: each individual decision ("this task is hard, give it high effort") is locally correct, yet collectively they blow the whole allowance before the day is half over. Section 6 is about putting brakes on.

The fix is a budget, and the analogy is a household grocery envelope with a fixed amount of cash for the month. You don't decide each purchase in isolation; you decide it in light of how much is left and how many days remain. If it's the 25th and the envelope is nearly empty, you buy the store brand — not because it's better, but because it's what you can sustain to month-end. A budget-aware effort selector does the same: it picks the highest effort you can still afford for all the requests you expect to serve, and steps the dial down as the money runs low. The essential property is graceful degradation — under pressure it serves everyone at a reduced effort rather than serving the early requests lavishly and then failing the rest.

The second technique in this section spends test-time compute a completely different way. Instead of one long, expensive think, self-consistency asks the model the same question several times independently at moderate effort, then takes the majority vote. The intuition: independent reasoning attempts tend to make different mistakes but tend to agree on the right answer, so the correct answer floats to the top of the tally even when individual samples slip. It's the "ask three doctors and go with two-out-of-three" strategy — and on problems with a single checkable answer, a handful of cheap votes often beats one costly deep think.

The nuance people miss is that these are two flavours of the same test-time-compute budget, not a free extra. Self-consistency trades a bigger single thinking budget for several parallel calls — you're still paying N times, just spread across attempts instead of concentrated in one. Which is cheaper depends on your latency budget (parallel votes can return faster than one enormous think) and on whether the task even has a single answer to vote on (it's useless for open-ended writing). The labs below build both a budget selector that degrades gracefully and a majority-vote function so you can see each mechanism concretely.

At scale you can't let effort float freely — you give it a budget. A budget-aware selector picks the highest effort you can still afford for the remaining requests, degrading gracefully instead of blowing the daily spend by lunchtime.

budget_selector.pyCOST = {"low": 0.01, "medium": 0.03, "high": 0.06, "max": 0.10}  # $ / request

def pick_effort(remaining_budget, remaining_requests, desired):
    """Pick the desired effort if the budget can sustain it for all remaining
    requests, else step down until it fits. Deterministic."""
    order = ["max", "high", "medium", "low"]
    start = order.index(desired)
    for effort in order[start:]:
        if COST[effort] * remaining_requests <= remaining_budget:
            return effort
    return "low"  # cheapest fallback

print(pick_effort(remaining_budget=100.0, remaining_requests=500,  desired="high"))
print(pick_effort(remaining_budget=20.0,  remaining_requests=500,  desired="high"))
print(pick_effort(remaining_budget=3.0,   remaining_requests=500,  desired="high"))

high
medium
low

Another test-time-compute technique is self-consistency: instead of one long think, sample several independent answers at moderate effort and take a majority vote. On problems with a single right answer, voting over a handful of samples often beats one expensive attempt — trading parallel calls for a bigger single thinking budget.

self_consistency.pydef fake_sample(task_seed, i):
    """A DETERMINISTIC stand-in for an independent model sample.
    Varies by index (no random): most samples land on the correct answer,
    a minority drift to wrong ones — like real reasoning noise."""
    correct = "42"
    distractors = ["41", "43", "40"]
    # every 4th sample goes astray, cycling through distractors
    if i % 4 == 3:
        return distractors[(i // 4) % len(distractors)]
    return correct

def self_consistency(task_seed, n_samples=8):
    votes = {}
    for i in range(n_samples):
        ans = fake_sample(task_seed, i)
        votes[ans] = votes.get(ans, 0) + 1
    winner = max(votes, key=lambda k: votes[k])
    return winner, votes

winner, votes = self_consistency("hard-math-q", n_samples=8)
print("votes:", votes)
print("majority answer:", winner)

votes: {'42': 6, '41': 1, '43': 1}
majority answer: 42

7 · Turning it on in code professional

Everything so far has been the mental model; here's where it becomes an API parameter. The good news, and it surprises people who expected something exotic, is that "turn on reasoning" is not a different endpoint, a different model family you have to learn, or a special mode — it's one field on the ordinary message request. All the theory about ceilings and adaptive spend collapses, in code, into a small thinking object you attach to a normal call.

Two parts to that object are worth naming because they connect straight back to earlier sections. First, you enable thinking (type "enabled") — this is the switch that turns the fast student into the scratch-paper student from §1. Second, you set budget_tokens — and this number is the effort dial from §4, just expressed in the raw units the named notches were hiding. "Low/medium/high/max" were always just friendlier labels for "how many thinking tokens do you allow"; here you're setting that ceiling directly, per request.

One structural detail to expect in the response, so it doesn't confuse you: the reply comes back as a list of content blocks, not a single string. Some blocks are the hidden thinking (the scratch paper — billed, useful for logging and debugging, usually not shown to end users) and one is the final text answer (what you actually ship). Separating those two is a product decision you'll make deliberately — surface the reasoning for transparency and eval, hide it for clean UX and to avoid leaking chain-of-thought.

A fair warning about this particular snippet: unlike every lab above it, it does not run offline — it needs the SDK installed, network access, and credentials, and it makes a real, billed call. That's deliberate. The offline labs let you model the tradeoffs for free; this one shows the single line where those tradeoffs meet the real API. Treat it as the shape of the call, not something to fire in a loop while you're learning.

In real code you enable extended thinking on the request. This snippet is illustrative — it needs the SDK and a network call, so it does not run offline like the labs above.

Turn on the thinking / effort dial — in Claude or OpenAI

The effort dial maps across both SDKs. In Claude you attach a thinking object whose budget_tokens is the dial and read hidden thinking blocks off resp.content; in OpenAI you pass reasoning={"effort": "low|medium|high"} on the Responses API and read the answer off resp.output_text. Toggle the tab to switch SDK.

thinking_sdk.py# needs the SDK: pip install anthropic; requires network + credentials.
# Illustrative only — the offline labs above model the same tradeoffs.
from anthropic import Anthropic

client = Anthropic()

resp = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=4000,
    thinking={"type": "enabled", "budget_tokens": 6000},  # the effort dial
    messages=[{"role": "user",
               "content": "Prove that the sum of the first n odd numbers is n^2."}],
)

# The response carries hidden `thinking` blocks (billed) plus the final answer.
for block in resp.content:
    print(block.type)

thinking_sdk.py# needs the SDK: pip install openai; requires network + credentials.
# Illustrative only — the offline labs above model the same tradeoffs.
from openai import OpenAI

client = OpenAI()

resp = client.responses.create(
    model="gpt-5.5",
    max_output_tokens=4000,
    reasoning={"effort": "high"},  # the effort dial: low | medium | high | max
    input="Prove that the sum of the first n odd numbers is n^2.",
)

# Reasoning tokens are billed but hidden; the final answer is on output_text.
print(resp.output_text)

8 · A reasoning strategy for the system tech-lead

Here's the failure mode this section exists to prevent. A team ships a reasoning feature, and effort gets set the way most things get set under deadline: by whoever last touched the code, by feel, in scattered if-statements across the codebase. It works in the demo. Three months later nobody can answer "why is our reasoning bill $40k?" because there's no single place that decides when the system pays for reasoning — the policy lives in a dozen people's heads and a hundred ad-hoc branches. A tech lead's job is to make that policy one explicit, auditable thing instead of emergent folklore.

Think of it the way you'd think about a company's spending-approval policy. A healthy org doesn't let every employee decide expenses by gut; it has tiers — anyone can buy coffee, a manager signs off on a laptop, a VP approves a server rack — and a fallback for when the budget's frozen. Everyone can point to the rule. A reasoning tier is the same: a written policy that composes the pieces you've already built into one coherent decision made the same way every time, by every request, and inspectable when the bill arrives.

Concretely, the tier stacks the earlier sections into layers. A difficulty router (§5) sits in front so the easy majority never touches reasoning at all. Behind it, a default effort per route (§4) — easy routes stay low, hard routes go high — rather than one global setting. A spend budget (§6) caps the total and degrades gracefully under load. And critically, evals: measurements proving the extra compute actually buys accuracy on your tasks, because a benchmark win doesn't guarantee a win on your traffic. The reframe that separates a lead from an engineer here is simple: reasoning is a cost center you manage, not a switch you leave on — you route to it, cap it, and prove it earns its cost. The playbook and decision code below make each of those layers explicit.

A lead doesn't set effort per prompt by feel — they design the whole reasoning tier: a difficulty router in front (fast path for the easy majority), default effort tuned per route, a spend budget with graceful degradation, and evals proving the extra compute actually buys accuracy on your tasks (not just benchmarks). Reasoning is a cost center you manage, not a switch you leave on.

The reasoning-tier playbook

1. Route by difficulty first — most traffic never needs the reasoning path (router.py).
2. Set a default effort per route, not a global one — easy routes stay low, hard routes go high.
3. Cap spend with a budget that degrades effort gracefully under pressure (budget_selector.py).
4. Prove the win on evals — measure accuracy vs cost at each effort on your own tasks; stop where returns flatten.
5. Consider self-consistency for single-answer tasks where parallel votes beat one long think.

This connects to two neighbours. When even high effort on a base model can't reliably hit a narrow behavior, that's the signal to consider the fine-tuning decision (XT4) — reasoning tunes compute at inference, fine-tuning tunes the model itself. And the effort/thinking setting you're dialing here is the one introduced in Ch 02 · Prompting.

def difficulty(prompt):
    hard_signals = ["prove", "step by step", "derive", "plan", "why"]
    score = sum(sig in prompt.lower() for sig in hard_signals)
    return "hard" if score or len(prompt) > 200 else "easy"

def route(prompt):
    return "reasoning-model" if difficulty(prompt) == "hard" else "fast-model"

print(route("What's the capital of France?"))          # fast-model
print(route("Derive the shortest path step by step"))  # reasoning-model

THINKING_BUDGET = {"low": 1000, "medium": 4000, "high": 16000, "max": 32000}
PRICE_PER_1K = 0.015   # illustrative output-token price in dollars

def est_cost(effort, answer_tokens=500):
    tokens = THINKING_BUDGET[effort] + answer_tokens
    return round(tokens / 1000 * PRICE_PER_1K, 4)

for e in THINKING_BUDGET:
    print(f"{e:6} -> ~{THINKING_BUDGET[e]} think tokens, ${est_cost(e)}")
# max costs ~20x low; use the lowest effort that clears your accuracy bar

from collections import Counter

def self_consistency(answers):
    counts = Counter(answers)
    best, n = counts.most_common(1)[0]
    return {"answer": best, "votes": n, "confidence": round(n / len(answers), 2)}

# imagine 5 independent reasoning samples for one arithmetic problem
samples = ["42", "42", "42", "41", "42"]
print(self_consistency(samples))
# {'answer': '42', 'votes': 4, 'confidence': 0.8}

class ReasoningBudget:
    def __init__(self, ceiling=50000):
        self.ceiling = ceiling; self.used = 0
    def effort_for(self, requested):
        order = ["low", "medium", "high", "max"]
        budget = {"low": 1000, "medium": 4000, "high": 16000, "max": 32000}
        # downgrade until it fits the remaining budget
        i = order.index(requested)
        while i > 0 and self.used + budget[order[i]] > self.ceiling:
            i -= 1
        chosen = order[i]; self.used += budget[chosen]
        return chosen

b = ReasoningBudget(ceiling=20000)
print(b.effort_for("max"))    # high  (32k would overflow -> downgraded)
print(b.effort_for("max"))    # low   (little budget left)
print("used:", b.used)

from anthropic import Anthropic

client = Anthropic()

resp = client.messages.create(
    model="claude-sonnet-4-5",
    max_tokens=4000,
    thinking={"type": "enabled", "budget_tokens": 8000},
    messages=[{"role": "user",
               "content": "A train leaves at 3pm going 60mph... solve step by step."}],
)

for block in resp.content:
    if block.type == "thinking":
        print("[reasoning]", block.thinking[:80], "...")
    elif block.type == "text":
        print("[answer]", block.text)

def reasoning_policy(request):
    tier = request["tier"]           # 'free' | 'pro'
    kind = request["kind"]           # 'chat' | 'math' | 'code' | 'plan'
    latency_sla_ms = request["sla_ms"]

    if kind == "chat" or latency_sla_ms < 500:
        return {"model": "fast", "thinking": None}      # cheap, snappy
    if kind in {"math", "plan"}:
        effort = "high" if tier == "pro" else "medium"  # gate cost by tier
        return {"model": "reasoning", "thinking": effort}
    if kind == "code":
        return {"model": "reasoning", "thinking": "medium"}
    return {"model": "fast", "thinking": None}

print(reasoning_policy({"tier": "pro", "kind": "plan", "sla_ms": 5000}))
# {'model': 'reasoning', 'thinking': 'high'}
print(reasoning_policy({"tier": "free", "kind": "chat", "sla_ms": 300}))
# {'model': 'fast', 'thinking': None}

Knowledge check check yourself