Token counting & usage

1 · Why count tokens at all essential

Picture the invoice nobody warned you about. You ship an agent that quietly stuffs a growing chat history and a fat retrieved-context blob into every call. In the demo it's a few cents. A month later finance forwards you a bill with an extra zero, and the worst part is you can't say which feature spent it. That surprise is entirely avoidable — because unlike almost everything else about a language model, the cost of a call is not fuzzy. It's a number you can compute exactly, before and after, if you know where to look.

Here's the mental model that makes the rest of this lesson click: a token is the model's unit of both work and money, so counting tokens is simply reading the meter. Think of it like a taxi with the fare display on the dash — you don't guess what the ride will cost, you watch it tick. count_tokens is the display before you get in (a quote); the usage object is the receipt after you get out. Miss either and you're back to guessing, which is how the surprise-bill story starts.

The mistake almost everyone makes first: reaching for a familiar token estimator from a different ecosystem and trusting it. It feels close enough — until "close enough" is a 20% miscount on the exact number your budget gate depends on, and your "safe" prompt silently blows the window in production. The rule below exists precisely because this trap is so easy to fall into and so quiet when it bites.

Every request to Claude is billed by the token — a chunk of text roughly ¾ of a word. You pay for the tokens you send (input) and the tokens the model writes back (output), at different per-model prices. Two questions follow you around: will this prompt even fit in the model's context window, and what will it cost. Both have exact answers, and this lesson is the recipe book for getting them.

The one rule that saves the most grief: do not estimate Claude tokens with tiktoken — that's OpenAI's tokenizer and it undercounts Claude by 15–20% (much more on code). The only accurate count is from Anthropic's own count_tokens endpoint, which uses the same tokenizer as inference for the model you name.

Read the diagram left to right: you count first (a cheap, output-free call), decide whether the prompt fits your token budget and the model's context window, send the real request only if it does, then read the usage that comes back and log the cost. The rest of the lesson is one recipe per box.

2 · Pre-flight: count tokens before you send essential

The cheapest token is the one you never generate. There's a discipline pilots follow that maps perfectly here: the pre-flight check. You don't taxi to the runway and then discover the fuel gauge is empty — you check on the ground, where a problem costs nothing but a delay. Counting tokens before you send is exactly that check. It runs on the "ground" (no generation, so no output bill), tells you the real size of what you're about to fly, and lets you turn back if the load is wrong.

Why this deserves its own recipe rather than a footnote: the alternative is discovering your prompt is too big after the model has already read it. By then you've paid to process a giant input, or the request errors out at the window limit, or worse, it squeaks under the window but leaves no room for the answer and the model gets cut off mid-sentence. A pre-flight count converts all of those failure modes into a single cheap boolean — "does this fit?" — that you answer before spending anything.

The common mistake here is subtle and worth naming: people count the prompt but forget to count it the same way the real call will be built. If your generation call includes a system prompt and a tools definition, those tokens count too — and a count that omits them is a lie that passes review and fails in prod. The fix is baked into the API's design, which is why the next paragraph stresses passing the identical arguments.

client.messages.count_tokens(...) takes the same arguments you'd pass to messages.create — model, messages, and optionally system and tools — and returns an object whose input_tokens is exactly what that prompt will cost as input. It runs no generation, so it's fast and cheap. Use it to reject or trim an over-budget prompt before paying for a full call.

count_tokens.py# ▶ NEEDS API KEY + NETWORK. Real Anthropic SDK — will not run offline.
# pip install anthropic ; export ANTHROPIC_API_KEY=sk-ant-...
import anthropic

client = anthropic.Anthropic()

messages = [{"role": "user", "content": "Summarize the theory of relativity."}]

# Pre-flight: same args as messages.create, but no generation happens.
count = client.messages.count_tokens(
    model="claude-opus-4-8",
    system="You are a concise physics tutor.",
    messages=messages,
)
print("input tokens:", count.input_tokens)   # e.g. 24

# A pre-flight budget/window check before you spend on a real call:
MODEL_CONTEXT_WINDOW = 1_000_000      # Opus 4.8 context window
MY_INPUT_BUDGET = 50_000              # your own per-request cap
if count.input_tokens > MY_INPUT_BUDGET:
    raise SystemExit("Prompt over budget — trim it before sending.")
if count.input_tokens > MODEL_CONTEXT_WINDOW:
    raise SystemExit("Prompt exceeds the model context window.")

# Only now do you pay for generation:
resp = client.messages.create(
    model="claude-opus-4-8", max_tokens=1024,
    system="You are a concise physics tutor.", messages=messages,
)

input tokens: 24

3 · The usage object on a response essential

If the pre-flight count was the quote, this is the itemized receipt — and you should read it like one. Nobody glances at a restaurant bill and assumes the total; you scan the line items, because that's where the surprise charge hides. The usage object is that itemization for a Claude call: it doesn't just say "you spent tokens," it breaks the spend into buckets that are each priced differently. Skip the line items and you'll mis-attribute cost — and cost you can't attribute is cost you can't cut.

Here's why the breakdown matters and isn't just bookkeeping. Input and output tokens are billed at different rates (output is the expensive side), and prompt-cache traffic splits input further into a "write" tier and a "read" tier at their own multipliers. So two calls that look identical on the surface — same prompt, same answer — can cost wildly different amounts depending on how much was served from cache. The only way to know which call you actually made is to read usage.input_tokens and usage.output_tokens together with the two cache fields, rather than eyeballing the prompt.

The mistake this section is built to prevent has burned careful engineers: assuming input_tokens is "the size of my prompt." It isn't — it's the uncached remainder. A long-lived agent replaying a cached prefix can report a shockingly small input_tokens and you'll conclude your prompt shrank, when really most of it was served cheaply from cache. Read the receipt as a whole, and the numbers stop lying to you. The four fields you care about most:

A subtle trap: input_tokens is the uncached remainder only. The full prompt size is input_tokens + cache_creation_input_tokens + cache_read_input_tokens. If a long-running agent shows a tiny input_tokens, the rest was served from cache — check the sum, not the one field.

read_usage.py# ▶ NEEDS API KEY + NETWORK. Real Anthropic SDK — will not run offline.
import anthropic

client = anthropic.Anthropic()

resp = client.messages.create(
    model="claude-opus-4-8", max_tokens=256,
    messages=[{"role": "user", "content": "Name three prime numbers."}],
)

u = resp.usage
print("input_tokens:                ", u.input_tokens)
print("output_tokens:               ", u.output_tokens)
print("cache_creation_input_tokens: ", u.cache_creation_input_tokens)
print("cache_read_input_tokens:     ", u.cache_read_input_tokens)

# Full prompt size = uncached input + cache write + cache read
full_prompt = (u.input_tokens
               + (u.cache_creation_input_tokens or 0)
               + (u.cache_read_input_tokens or 0))
print("full prompt tokens:          ", full_prompt)

# Log the request id alongside usage so you can trace a call with Anthropic.
print("request id:", resp._request_id)

4 · Computing dollar cost from usage (offline recipe) intermediate

Once you can read the receipt, converting it to dollars is the easy part — which is exactly why it's worth doing well. This is the one step in the whole loop that needs no API key, no network, and no luck: it's a price list times some counts. That makes it the perfect thing to pin down in code you can unit-test, so that the number you quote to your boss or your budget alert is arithmetic you've verified, not a vibe. Get this seam right and every other cost claim in your system inherits its trustworthiness.

The analogy is a grocery receipt: each line is quantity × unit_price, and the total is just the sum. Tokens work the same way, with one twist that trips people up — the "unit price" isn't one number per model, it's a small table of four: input, output, cache-write, and cache-read, and they are not equal. Output typically costs several times more than input, and a cache read is the bargain bin at roughly a tenth of the input rate. Multiply each bucket by the right rate, divide by a million (because prices are quoted per million tokens), and add. That's the entire calculation.

Two mistakes to sidestep. First, do not hard-code prices and forget them — model prices drift, and a stale table quietly mis-quotes every cost you report; keep the table in one place and date it. Second, do not price only input_tokens and output_tokens when caching is in play — the cache-write and cache-read buckets are real money (one above the input rate, one far below it), and dropping them either over- or under-states the bill. The helper below makes both easy to get right by taking the whole usage dict and the price table as inputs.

Cost is pure arithmetic: multiply each token bucket by its per-token price and add them up. Prices are quoted per 1,000,000 tokens, so divide by a million. Cache reads are the wrinkle — they bill at roughly 0.1× the input rate, which is the whole point of caching. The helper below is stdlib-only and RUNS offline so you can unit-test your cost math without touching the network: it takes a fake usage dict plus a price table and returns the dollar cost, handling the cache-read discount.

cost_from_usage.py# (runs offline) — pure stdlib, no SDK, no network. `python cost_from_usage.py`
# Prices are $ per 1,000,000 tokens and DRIFT — always re-check current pricing.
PRICES = {
    # model: (input_per_mtok, output_per_mtok)
    "claude-opus-4-8":   (5.00, 25.00),
    "claude-sonnet-4-6": (3.00, 15.00),
    "claude-haiku-4-5":  (1.00,  5.00),
}

CACHE_READ_MULTIPLIER = 0.1   # cache reads bill at ~0.1x the input rate

def cost_from_usage(model, usage):
    """usage: dict with input_tokens / output_tokens and optional cache fields.
    Returns the dollar cost of one call, rounded to 6 decimals."""
    in_price, out_price = PRICES[model]
    inp   = usage.get("input_tokens", 0)
    out_  = usage.get("output_tokens", 0)
    creat = usage.get("cache_creation_input_tokens", 0)   # billed ~1.25x input
    read  = usage.get("cache_read_input_tokens", 0)       # billed ~0.1x input

    dollars = (
        inp   * in_price
        + out_  * out_price
        + creat * in_price * 1.25
        + read  * in_price * CACHE_READ_MULTIPLIER
    ) / 1_000_000
    return round(dollars, 6)

# A fake usage receipt — no API call needed.
fake_usage = {
    "input_tokens": 1_000,
    "output_tokens": 500,
    "cache_creation_input_tokens": 0,
    "cache_read_input_tokens": 8_000,   # served cheaply from cache
}
cost = cost_from_usage("claude-opus-4-8", fake_usage)
print("call cost: $" + format(cost, ".6f"))
# 1000*5 + 500*25 + 8000*5*0.1 = 5000 + 12500 + 4000 = 21500 microdollars
print("that's", 1_000 + 500 + 8_000, "tokens touched")

call cost: $0.021500
that's 9500 tokens touched

5 · Tracking & logging usage per request intermediate

A single call's cost is trivia; a month of calls with no record is a mystery you pay for. The failure mode is familiar to anyone who has run a service: the bill arrives, it's higher than expected, and there's no way to say which endpoint, which user, or which feature drove it. You end up bisecting production by intuition. The cure is boring and cheap — write down what each call cost as it happens — but almost nobody does it until after the first scary invoice. This recipe is you doing it before.

Think of it as double-entry bookkeeping for your model calls. Every time money leaves (a call is made), a row goes into the ledger: the model, the tokens, the dollar cost, and the request id. Do that consistently and questions that were once archaeology become one-line queries — "cost per feature this week," "which request ids are our most expensive," "did that new prompt actually get cheaper." The value isn't any single row; it's that the ledger exists at all, so the data is there when you need to ask.

The mistake to avoid is scattering this logic — computing cost one way in a script, another way in a notebook, a third way in the service — so your numbers never quite agree. The clean move is a single thin wrapper around messages.create that reads usage once and reuses the same offline cost function you already unit-tested in recipe 4. Tested offline, trusted online: the math that priced your fake receipts is the math pricing your real ones, with no second implementation to drift.

Knowing one call's cost is nice; knowing your spend over time is what keeps a product alive. Wrap every call so it logs usage the moment the response returns, keyed by request id, and accumulate a running total. The recipe below shows the shape — a thin wrapper that records model, tokens, cost, and request id per call.

usage_logger.py# ▶ NEEDS API KEY + NETWORK. Real Anthropic SDK — will not run offline.
# (The cost_from_usage helper it calls IS the offline one from recipe 4.)
import anthropic
from cost_from_usage import cost_from_usage

client = anthropic.Anthropic()
LEDGER = []   # in real life: a DB row or a metrics counter per call

def tracked_create(**kwargs):
    resp = client.messages.create(**kwargs)
    u = resp.usage
    usage_dict = {
        "input_tokens": u.input_tokens,
        "output_tokens": u.output_tokens,
        "cache_creation_input_tokens": u.cache_creation_input_tokens or 0,
        "cache_read_input_tokens": u.cache_read_input_tokens or 0,
    }
    cost = cost_from_usage(kwargs["model"], usage_dict)
    LEDGER.append({
        "request_id": resp._request_id,
        "model": kwargs["model"],
        "usage": usage_dict,
        "cost_usd": cost,
    })
    return resp

tracked_create(
    model="claude-opus-4-8", max_tokens=256,
    messages=[{"role": "user", "content": "Hello"}],
)
print("calls logged:", len(LEDGER))
print("total spend: $" + format(sum(row["cost_usd"] for row in LEDGER), ".6f"))

6 · Budget, rate limits & the headers advanced

Nothing humbles a working prototype like the day it starts getting rejected under load — for a reason that has nothing to do with your prompt. Your calls were fine yesterday; today, at real traffic, they bounce with a 429. You stare at the prompt, it's small, it fits the window, so what gives? The answer is that there are two ceilings in this system, and the one that stopped you isn't the one you were watching. This section is about telling them apart, because conflating them sends you optimizing the wrong thing.

Here's the analogy that keeps it straight. The context window is the size of a single suitcase — how much you can pack into one request (input plus the space the answer needs). Rate limits are the airline's rule for how many suitcases you may check per hour across all your trips. A tiny suitcase (small prompt) that clears the size rule effortlessly can still be refused because you've already checked your hourly quota of bags. One limit is about the shape of a single request; the other is about your throughput over time. Fixing a window problem (trim the prompt) does nothing for a rate problem (slow down or raise the quota).

The beginner mistake, then, is to see a 429 and start hacking at the prompt size. That's treating a rate symptom with a window remedy. The right instinct is to read what the API is telling you: a 429 comes with a retry-after and x-ratelimit-* headers that spell out your remaining quota and how long to wait. The reassuring part — covered below — is that the SDK already retries these for you with backoff, so most of the time the correct action is to do nothing and let it recover; you reach for the raw headers only when you want to pace a bulk job or surface remaining capacity on a dashboard.

Two different ceilings can stop you, and beginners conflate them. The context window is how many tokens fit in one request (input + output). Rate limits are how much you may send per minute/day across all requests — requests-per-minute (RPM), input- and output-tokens-per-minute (ITPM/OTPM), and tokens-per-day (TPD).

When you exceed a rate limit the API returns HTTP 429 with a retry-after header and x-ratelimit-* headers telling you your remaining quota. The SDK already retries 429 and 5xx with exponential backoff (default max_retries=2) — so most of the time you do nothing. When you need the raw numbers, reach for the headers via with_raw_response:

rate_limits.py# ▶ NEEDS API KEY + NETWORK. Real Anthropic SDK — will not run offline.
import anthropic

client = anthropic.Anthropic()

# .with_raw_response exposes the HTTP headers alongside the parsed message.
raw = client.messages.with_raw_response.create(
    model="claude-opus-4-8", max_tokens=64,
    messages=[{"role": "user", "content": "ping"}],
)
h = raw.headers
print("requests remaining:", h.get("anthropic-ratelimit-requests-remaining"))
print("input tokens remaining:", h.get("anthropic-ratelimit-input-tokens-remaining"))
message = raw.parse()   # the Message you'd normally get from .create(...)
print("output tokens:", message.usage.output_tokens)

# The SDK auto-retries 429/5xx, but you can still catch a give-up:
try:
    client.messages.create(
        model="claude-opus-4-8", max_tokens=64,
        messages=[{"role": "user", "content": "ping"}],
    )
except anthropic.RateLimitError as e:
    retry_after = int(e.response.headers.get("retry-after", "60"))
    print(f"Rate limited; retry after {retry_after}s.")

7 · Pagination: list endpoints & the auto-paging iterator professional

This is the bug that passes every test and lies to production. You write a job that lists your batches, sums their costs, and reports the total. In dev you have eight batches, the number's right, you ship. Six months later the account has four hundred batches, your report still says a tidy sum — and it's wrong, because you were only ever counting the first page. Nothing errored. Nothing warned you. The code just quietly stopped telling the truth the moment the data outgrew one page. That silence is what makes pagination worth its own section.

Why do list endpoints page at all? Because a large account could hold thousands of files or batches, and dumping all of them in one response would be slow, memory-hungry, and fragile. So the API hands back a window onto the results — a page — plus a way to ask for the next one. Think of it like search results: page one shows ten hits, and "there are 4,000 results" doesn't mean they're all on the screen in front of you. Reading only what's on screen and calling it "all the results" is precisely the trap.

The good news is the SDK removes the tedium if you let it: iterating the object a .list() call returns transparently fetches page after page until the account is exhausted. The mistake — the one that caused the silent-undercount story above — is reaching for .data, which is only the current page, and treating it as the full set. The rule to internalize is short: iterate to get everything; index .data only when you truly want a single page. Get this reflex right and the "worked in dev, wrong in prod" class of pagination bugs simply never happens to you.

List endpoints — Message Batches, Files — return results a page at a time so a huge account doesn't dump everything at once. The SDK hides this: iterating the return value of a .list() call auto-paginates across every page. The rookie mistake is reading only .data (the first page) and thinking that's all there is.

paginate.py# ▶ NEEDS API KEY + NETWORK. Real Anthropic SDK — will not run offline.
import anthropic

client = anthropic.Anthropic()

# Iterating the list result auto-fetches EVERY page — not just the first.
total_batches = 0
for b in client.messages.batches.list(limit=20):
    total_batches += 1
    print(b.id, b.processing_status)
print("batches seen across all pages:", total_batches)

# Same pattern for the Files API (beta):
for f in client.beta.files.list(betas=["files-api-2025-04-14"]):
    print(f.id, f.filename, f.size_bytes)

# .data is only the CURRENT page — do NOT treat it as the full set:
first_page = client.messages.batches.list(limit=20)
print("this is ONLY page 1:", len(first_page.data), "items")
if first_page.has_next_page():
    print("...more pages exist; iterate the list to get them all")

8 · Tech-lead — owning the token & cost surface tech-lead

Here's the shift that separates a lead from an engineer: the recipes above are things you do; a lead's job is to make them things the system guarantees. An individual remembering to count tokens is good hygiene; a platform where no request can reach billing without passing a budget gate is a contract. The difference matters because hygiene depends on everyone being careful forever, and they won't be — a rushed teammate ships the un-gated call, a new service forgets to log. A contract holds even on the bad day. Your task is to move each of these habits from "please remember" to "can't skip."

Think of it as the difference between telling drivers to be careful and installing guardrails. The four habits from this lesson become four rails: a pre-flight gate so an oversized prompt is refused before it costs anything; a usage-and-cost-and-request-id log on every call so spend is a query rather than a guess; a single dated price table so nobody quotes a stale number; and a hard budget cap plus max_tokens ceiling so one runaway job can't drain the month. None of these is clever on its own — the leadership is in making them non-optional and putting them where the tokens actually are.

The mistake leads make here is optimizing cost everywhere except where it lives. Chasing a 5% saving on a rarely-used route while the expensive path runs Opus on work Haiku could handle is motion, not progress. Attack the tokens: cache the stable prefixes (that 0.1× cache-read line is the biggest lever on repeated calls), route each request to the smallest model that clears your eval bar, and cap generation length so a loop can't run away. And keep the offline cost_from_usage helper dependency-free on purpose, so the same tested math prices both your CI assertions and your live traffic.

A lead turns these recipes into a system contract. Pre-flight every request through count_tokens against a budget you own, not just the model's window. Log usage + cost + request id on every call into one place, so cost per feature is a query, not a guess. Price the workload from a table you keep current — prices drift, so the numbers in recipe 4 are a snapshot, not gospel; re-check them and keep the table in one module. Attack cost where the tokens are: cache stable prefixes (the 0.1× cache-read line), pick the smallest model that passes your evals, and cap max_tokens so a runaway generation can't blow the budget.

The offline cost_from_usage helper is deliberately dependency-free so it can live in your test suite: feed it recorded usage dicts and assert the cost math, then let the real SDK calls feed it live resp.usage in production. Same function, tested offline, trusted online.

import anthropic
client = anthropic.Anthropic()

resp = client.messages.count_tokens(
    model="claude-opus-4-8",
    messages=[{"role": "user", "content": open("CLAUDE.md").read()}],
)
print(resp.input_tokens)   # exact input token count for this model

PRICES = {   # (input $/MTok, output $/MTok)
    "claude-opus-4-8":   (5.0, 25.0),
    "claude-sonnet-4-6": (3.0, 15.0),
    "claude-haiku-4-5":  (1.0,  5.0),
}

def cost(input_tokens, output_tokens, model):
    pin, pout = PRICES[model]
    return input_tokens/1e6*pin + output_tokens/1e6*pout

# e.g. a response with usage.input_tokens=1200, usage.output_tokens=800
print(f"${cost(1200, 800, 'claude-opus-4-8'):.5f}")     # $0.02600
print(f"${cost(1200, 800, 'claude-sonnet-4-6'):.5f}")   # $0.01560

PRICES = {"claude-opus-4-8": (5.0, 25.0)}

def full_cost(u, model):
    pin, pout = PRICES[model]
    pin_t = pin / 1e6
    return (u["input_tokens"]                * pin_t          # uncached input
          + u["cache_creation_input_tokens"] * pin_t * 1.25   # cache write
          + u["cache_read_input_tokens"]      * pin_t * 0.10   # cache read
          + u["output_tokens"]               * pout / 1e6)    # output

u = {"input_tokens": 500, "cache_creation_input_tokens": 0,
     "cache_read_input_tokens": 20000, "output_tokens": 800}
print(f"${full_cost(u, 'claude-opus-4-8'):.5f}")   # $0.03250

PRICES = {"claude-opus-4-8": (5.0, 25.0)}
_total = {"usd": 0.0}

def logged_create(**kw):
    resp = client.messages.create(**kw)
    u = resp.usage
    pin, pout = PRICES[kw["model"]]
    usd = u.input_tokens/1e6*pin + u.output_tokens/1e6*pout
    _total["usd"] += usd
    print(f"req={resp._request_id} in={u.input_tokens} out={u.output_tokens} "
          f"${usd:.5f} total=${_total['usd']:.5f}")
    return resp

import time, anthropic

def call_with_backoff(**kw):
    try:
        return client.messages.create(**kw)
    except anthropic.RateLimitError as e:
        wait = int(e.response.headers.get("retry-after", "60"))
        print("rate limited; sleeping", wait,
              "remaining:", e.response.headers.get("x-ratelimit-remaining-tokens"))
        time.sleep(wait)
        return client.messages.create(**kw)

PRICES = {"claude-haiku-4-5": (1.0,5.0), "claude-sonnet-4-6": (3.0,15.0),
          "claude-opus-4-8": (5.0,25.0)}

def route(quality, est_in, est_out, remaining_usd):
    tier = {"simple": "claude-haiku-4-5", "standard": "claude-sonnet-4-6",
            "hard": "claude-opus-4-8"}[quality]
    pin, pout = PRICES[tier]
    est = est_in/1e6*pin + est_out/1e6*pout
    if est > remaining_usd:
        return f"REJECT — est ${est:.4f} exceeds remaining ${remaining_usd:.4f}"
    return f"{tier} (est ${est:.4f})"

print(route("simple",   2000, 500, 100.0))   # haiku
print(route("hard",  4_000_000, 1_000_000, 5.0))  # REJECT — over budget

Knowledge check check yourself