Message Batches API

1 · What the Message Batches API is essential

Here's the moment this chapter is written for. Your product manager drops a CSV of 80,000 support tickets on your desk on Friday afternoon and asks for each one classified by Monday. You do the obvious thing: a for loop that calls Claude once per ticket. You run the numbers — a couple of seconds per call, 80,000 calls — and realize the loop won't even finish before Monday, let alone leave you time to check the results. And you're paying full price for every single call. There has to be a better way to do bulk work that nobody is sitting and waiting on. There is, and it's the subject of this chapter.

The mental shift is this: not all LLM work is a conversation. A conversation is interactive — a human types, waits, reads, types again, and every millisecond of delay is felt. But a huge amount of real-world LLM work is bulk processing: you have a pile of independent jobs, none of them urgent, and you just want them all done cheaply by some reasonable deadline. Think of the difference between a barista making your coffee while you wait at the counter versus a bakery baking ten thousand loaves overnight. Same oven, completely different economics — the bakery runs at night, in bulk, off-peak, and charges less per loaf because of it. The Message Batches API is Claude's overnight bakery.

The common mistake is reaching for a batch when a human actually is waiting — wiring your live chat endpoint to batches.create to "save money." That trades away the one thing your users care about (a fast reply) for a discount they'll never notice, because your chat now takes minutes to respond. Batching is for the pile-of-jobs case, never the someone's-watching case. Keep that line clear and the rest of this chapter is just plumbing.

Most of the time you call Claude one request at a time and wait for the answer — that's the synchronous Messages API you already know. But sometimes you have thousands of requests and no human waiting on any single one: classify a week of support tickets overnight, summarize 50,000 documents, generate embeddings-style labels for a whole catalog. For that, Anthropic gives you the Message Batches API.

A batch is exactly what it sounds like: you hand Claude a list of independent requests in one call, Anthropic processes them asynchronously (in the background, on its own schedule), and you come back later to collect the results. Each request in the batch is an ordinary Messages API request — same model, max_tokens, messages — just wrapped with a label you choose (custom_id) so you can tell the answers apart.

The trade you're making: you give up immediacy (results aren't instant — most batches finish within an hour, and the hard cap is 24 hours) and you get back ~50% off every input and output token. For non-urgent bulk work, that's a huge lever.

2 · The batch lifecycle essential

Every batch you ever run follows the same four beats — memorize the shape and you've learned the whole API. The single biggest source of confusion for newcomers is expecting a batch to behave like a normal function call: you call it, you get your answer back on the next line. It doesn't. A batch is fire-and-forget-then-come-back, and that asynchronous rhythm is the one genuinely new idea in this chapter.

The everyday analogy is dropping film off to be developed — and it's worth walking through because the API maps onto it almost exactly. You don't stand at the counter while your photos develop. You hand over the roll (that's submit), you get a little paper ticket with a number on it (that's the batch id), you go about your day and periodically check whether it's ready (that's poll), and eventually you come back with your ticket and collect the prints (that's retrieve). Before any of that you had to actually load the film and label the roll (that's build the requests). The photo lab doesn't phone you the instant your prints are done — you have to ask. That "you have to ask" property is the crux of async, and it's why polling exists at all.

The mistake to avoid is treating "the batch came back" as "every photo came out perfectly." When you collect your prints, some frames may be blurry or blank — but the roll is still "done." A batch reaching its finished state means Anthropic has stopped working on it, not that every item succeeded; you find out the per-item outcomes only when you look at the results. Hold that thought — it comes back in §5 and §6, and getting it wrong is the classic batch bug.

Every batch goes through the same four beats. Learn this shape once and the code below is just wiring:

Read it left to right. First you build a list of requests, each tagged with a custom_id. You submit the list and get back a batch object with an id and a processing_status. Because processing happens in the background, you poll — ask "are you done yet?" — until the status becomes "ended". Then you retrieve the results, which arrive in any order, and use your custom_ids to line each answer back up with the request that produced it. The whole run is billed at roughly half the normal per-token price.

3 · Recipe 1 — submit a batch essential

This is the step where a subtle design decision will either save you or sink you, and it takes exactly one field to get right. Submitting a batch is mechanically simple — you build a list and hand it over — but buried in each list item is the custom_id, and that little label is the difference between results you can use and results you can't.

Picture checking a coat at a busy theater. You hand over your coat and get a numbered tag; when you come back, you present the tag and get your coat back — not a random one. The coat check works because every coat carries a claim tag that you can tie back to yourself. A batch is identical: results come back in no guaranteed order, so if your 80,000 classifications arrive as an unlabeled shuffle, you have no way to know which answer belongs to ticket #8801 versus #8802. The custom_id is your claim tag. You stamp it onto each request going in, and Anthropic stamps it back onto each result coming out.

The classic beginner mistake is assuming results come back in the order you submitted them, and matching answers to inputs by position — answers[0] goes with tickets[0], and so on. This will appear to work in a tiny test and then silently corrupt your data at scale, because the ordering guarantee simply does not exist. Make your custom_id something meaningful and unique from your own data — a ticket ID, a database row key, a content hash — never a bare array index, and never a random value you can't reconstruct later. Get this habit right now and everything downstream (matching, error retries, idempotency in §9) falls into place.

Here is the real SDK, end to end for the submit step. Each request is wrapped in a Request with a custom_id you choose and a params block that is a normal non-streaming Messages request. The custom_id is your label — use something meaningful (a ticket ID, a row key) so you can match results later.

submit.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes real API calls
import anthropic
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

client = anthropic.Anthropic()   # reads ANTHROPIC_API_KEY from the environment

tickets = {
    "ticket-8801": "I was charged twice for my subscription this month.",
    "ticket-8802": "How do I export my data to CSV?",
    "ticket-8803": "The app crashes every time I open the reports tab.",
}

requests = [
    Request(
        custom_id=ticket_id,                         # YOUR label — match results by this
        params=MessageCreateParamsNonStreaming(
            model="claude-haiku-4-5",                # cheap+fast: classification is easy work
            max_tokens=64,
            system="Classify the ticket as one word: billing, howto, or bug.",
            messages=[{"role": "user", "content": text}],
        ),
    )
    for ticket_id, text in tickets.items()
]

batch = client.messages.batches.create(requests=requests)
print(batch.id)                  # e.g. msgbatch_01AbC...  — save this
print(batch.processing_status)   # "in_progress"

4 · Recipe 2 — poll until the batch ends intermediate

Now you hit the part of async that feels backwards the first time. You submitted the batch and got an id — but no results. Anthropic didn't hand you a callback, didn't email you, won't ping your server. The work happens in the background on Anthropic's schedule, and the only way to learn it's done is to ask, repeatedly, until the answer is yes. That's polling, and it's the natural consequence of the fire-and-forget model from §2.

Think of a slow-cooker with no timer bell. Dinner is cooking; nothing is going to alert you when it's ready. So every so often you lift the lid, glance in, and put it back. You don't stand there lifting the lid every two seconds — that's exhausting and accomplishes nothing — and you don't wander off for six hours either. You check on a sensible cadence. Polling is exactly this: call retrieve(id), read processing_status, and if it's not finished yet, wait a bit and check again.

The mistake almost everyone makes first is the tight loop — hammering retrieve with no pause between calls, spinning the CPU and burning through your API rate limit while the batch, which may take up to an hour, hasn't budged. Add a sleep between polls (30–60 seconds is plenty for work measured in minutes). A second, subtler trap: reading processing_status == "ended" as "everything worked." It doesn't mean that — as the film-lab analogy in §2 warned, "ended" only means Anthropic has stopped working the batch. Whether each item succeeded, errored, or expired is a separate question you'll answer in §5. Here, "ended" is purely your green light to go collect the results.

After submitting you have a batch id but no answers yet. You poll: call retrieve(id) on a loop and check processing_status. When it flips to "ended", processing is finished (whether every item succeeded, errored, or expired — "ended" just means the batch is done being worked on).

poll.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes real API calls
import time
import anthropic

client = anthropic.Anthropic()

while True:
    batch = client.messages.batches.retrieve("msgbatch_01AbC...")   # the id from submit
    if batch.processing_status == "ended":
        break
    # request_counts breaks the batch down while it runs
    counts = batch.request_counts
    print("still working:", counts.processing,
          "| done:", counts.succeeded, "| errored:", counts.errored)
    time.sleep(30)      # be patient — most batches finish within an hour, max 24h

print("batch ended")
print("succeeded:", batch.request_counts.succeeded)
print("errored:  ", batch.request_counts.errored)

5 · Recipe 3 — retrieve results & match by custom_id intermediate

This is payoff time — and also where the two warnings from earlier sections finally cash in. The batch has ended, so you go collect your answers. But the results don't arrive as one tidy list you asked for in order; they stream back in whatever order Anthropic finished them, each one wearing the custom_id claim tag you stamped on it back in §3. Your job here is to walk that shuffled stream and re-assemble it into something you can actually use.

The natural tool is a dictionary keyed by custom_id. Think of it like sorting incoming mail into a wall of labeled pigeonholes: each letter has an address on it, and you drop it into the matching slot regardless of the order the letters arrived. When the stream is exhausted, every slot holds the right answer next to the right question, and the arrival order — which you never controlled — has stopped mattering entirely. This is why the custom_id discipline was worth insisting on: the join happens here, and it only works because you labeled everything going in.

The mistake to guard against is assuming every result is a success and reaching straight for the message text. Recall the film-lab warning: an ended batch can still contain individual items that errored or expired. So before you touch the answer, you check result.result.type. On "succeeded", the full Message lives at result.result.message — the very same object a normal synchronous call returns, so you read its .content exactly as you always have. On any other outcome, you record why instead of crashing. That single branch is what turns a fragile script into one that survives a real 100,000-item run — and §6 drills into each failure type in turn.

Once the batch has ended, stream the results with .results(id). Each result carries the custom_id you set and a result whose .type tells you what happened. On success, the full Message is at result.result.message — the same object a normal call returns, so you read .content the same way. Build a dict keyed by custom_id so the arbitrary arrival order stops mattering.

results.py# needs: pip install anthropic + ANTHROPIC_API_KEY; makes real API calls
import anthropic

client = anthropic.Anthropic()

answers = {}
for result in client.messages.batches.results("msgbatch_01AbC..."):
    cid = result.custom_id                 # the label you chose at submit time
    kind = result.result.type              # succeeded | errored | canceled | expired
    if kind == "succeeded":
        msg = result.result.message        # a normal Message object
        text = next((b.text for b in msg.content if b.type == "text"), "")
        answers[cid] = text
    elif kind == "errored":
        # per-item failure — the rest of the batch is unaffected
        answers[cid] = f"ERROR: {result.result.error.type}"
    elif kind == "expired":
        answers[cid] = "EXPIRED — resubmit this one"
    elif kind == "canceled":
        answers[cid] = "CANCELED"

# arrival order was arbitrary; the dict lines everything back up by id
for cid in sorted(answers):
    print(cid, "->", answers[cid])

ticket-8801 -> billing
ticket-8802 -> howto
ticket-8803 -> bug

6 · Error handling per item advanced

Here is where batch error handling stops resembling everything you've learned about error handling — and it's a genuine relief once it clicks. In normal code, an error is an event: something throws, execution jumps, you catch it or you crash. But in a batch there is no single call to wrap in try/except. The failures already happened, hours ago, in the background. By the time you see them they aren't exceptions to catch — they're just data, one outcome stamped on each result, sitting quietly in the stream alongside the successes.

The analogy is a returned batch of lab tests. You don't get a phone call the moment one sample is contaminated; you get a report where each line reads "clear," "inconclusive — redraw," or "sample expired, please resubmit." Nothing is on fire. You simply read down the column and route each line to the right follow-up. That's the whole posture shift: instead of defending against failures as they strike, you reconcile a finished ledger of outcomes. One malformed request out of 100,000 doesn't sink the run — it's one line in the report, and the other 99,999 are unaffected.

The trap is treating all four outcomes the same — blindly retrying everything that isn't a success, or worse, discarding failures silently. They demand different reactions: a malformed request you wrote wrong (errored with invalid_request) will fail identically on retry, so fix it, don't loop on it; a transient server error is safe to re-run as-is; an expired item never ran and just needs resubmitting; a canceled item is only worth resubmitting if you still want it. Learn to tell them apart and your retry logic becomes precise instead of a blunt "try again" that wastes money on requests that can never succeed. The table below is the decision key.

In the synchronous API a bad request raises an exception and you deal with it right there. In a batch there is no single call to wrap in try/except — instead, each item carries its own outcome. The four result.result.type values map to four different reactions:

The practical rule: treat the results stream as a reconciliation step. Walk every item, route successes to your output, and collect the errored / expired custom_ids into a "retry" list you can feed straight into the next batch.

7 · Batch vs realtime vs streaming — when to use which advanced

You now know how to run a batch — but the more valuable engineering skill is knowing when not to. The same Claude model is available through three different delivery modes, and reaching for the wrong one is one of the most common architecture mistakes on LLM products. The good news: the choice collapses to a single question, and once you internalize it you'll never agonize over it again.

Here's the question: is a human blocked, right now, waiting on this exact response? That's it. Everything flows from the answer. If yes, you're in interactive territory and latency is king — you want synchronous for a quick reply, or streaming when the reply is long and you'd rather show words appearing than a spinner. If no — the work can happen overnight, or "sometime in the next hour" — you're in bulk territory and cost is king, so you batch and pocket the discount. It's the same fork we drew in §1, now formalized into a routing decision.

The analogy worth carrying: synchronous and streaming are the express checkout lane — fast, per-item, you pay full retail because you're not waiting. Batch is the warehouse pallet order — slower, delivered on a schedule, dramatically cheaper per unit because nobody needs any single item this instant. The mistake is confusing streaming with batching because both sound like "efficiency." They solve opposite problems: streaming minimizes the perceived latency of one interactive request (the human sees progress sooner), while batching maximizes throughput per dollar across many non-interactive requests. Streaming makes one person happier; batching makes ten thousand jobs cheaper. Never batch a latency-sensitive path just to save money — you'll save pennies and lose the user.

The API gives you three delivery modes for the same underlying model. Choosing correctly is a latency-vs-cost decision:

A simple test: is anyone blocked on this exact response? If yes → synchronous or streaming. If the work can happen overnight or "sometime in the next hour" → batch, and take the discount. Streaming and batching solve opposite problems — streaming minimizes perceived latency for one interactive request; batching maximizes throughput per dollar for many non-interactive ones. Don't reach for a batch just to save money on a latency-sensitive path.

8 · Cost math (this one runs offline) professional

"~50% cheaper" is an easy phrase to nod along to and a hard one to actually reason about — so let's turn it into dollars you can put in a decision. The whole reason batching exists is the discount, yet most engineers never do the arithmetic before committing a big job. That's a mistake in both directions: you either skip a batch that would have saved real money, or you accept up-to-an-hour latency for a workload where the savings turn out to be trivial.

The intuition to hold is that the saving is a flat percentage, so it scales linearly with volume — halving the per-token price means the total is always exactly half, whether the job is 50 requests or 50 million. That linearity is what makes the call easy: the discount is never the variable. A useful mental picture is a "buy in bulk, get 50% off" wholesale rate — the unit discount is fixed, so the only thing that changes with size is how many dollars that fixed percentage adds up to. At small volume the absolute saving is pocket change and not worth the async complexity; at large volume it's a line item your finance team will notice.

The mistake is treating "is batch cheaper?" as the decision — it isn't, because the answer is always yes. Per token, batch always wins; that's not in question. The real decision is entirely about latency: can this workload tolerate results landing minutes-to-an-hour later instead of instantly? If yes, the discount is essentially free money and you take it. If no, no amount of savings justifies making a waiting user wait longer. The calculator below exists to make the dollar figure concrete so you can weigh it honestly against that latency question — it's pure stdlib, runs with a plain python file.py, no API key and no network, and simply compares the realtime cost of N requests against the batch cost.

Before you commit a big job to a batch, it's worth doing the arithmetic: how much does the ~50% discount actually save, and is the added latency worth it? The helper below is pure stdlib and runs with a plain python file.py — no API key, no network. It compares the realtime cost of N requests against the batch cost.

cost.pydef batch_savings(num_requests, in_tokens, out_tokens,
                  in_price_per_mtok, out_price_per_mtok, batch_discount=0.50):
    """Compare realtime vs batch cost for a bulk job. Prices are $ per 1M tokens.
    batch_discount=0.50 means batch pays 50% of the realtime per-token price."""
    per_request = (in_tokens * in_price_per_mtok
                   + out_tokens * out_price_per_mtok) / 1_000_000
    realtime = per_request * num_requests
    batch = realtime * (1 - batch_discount)
    return {
        "realtime_usd": round(realtime, 2),
        "batch_usd": round(batch, 2),
        "saved_usd": round(realtime - batch, 2),
        "saved_pct": round(100 * (realtime - batch) / realtime),
    }

# 50,000 classification calls on Haiku ($1 in / $5 out per 1M tokens)
result = batch_savings(
    num_requests=50_000, in_tokens=400, out_tokens=8,
    in_price_per_mtok=1.00, out_price_per_mtok=5.00,
)
for k, v in result.items():
    print(f"{k}: {v}")

realtime_usd: 22.0
batch_usd: 11.0
saved_usd: 11.0
saved_pct: 50

9 · Tech-lead — production batch pipelines tech-lead

The gap between "I ran a batch in a notebook" and "we run batches in production" is the gap this section closes — and it's mostly about what happens when things go wrong at 3 a.m. A one-off script assumes the happy path: submit, poll, retrieve, done. A production pipeline assumes the poller will crash mid-run, the job will be 500,000 items when the limit is 100,000, and someone will re-run the whole thing next week by accident. Designing for those realities is what a tech lead actually owns here.

The shift in mindset is from calling an API to operating a job queue you didn't have to build. Anthropic hands you an at-least-once background worker pool for free — but "at-least-once" and "background" are precisely the properties that demand discipline. Think of a warehouse fulfillment system: every package needs a stable tracking number (so a re-scan doesn't create a duplicate shipment), oversized orders get split across trucks (so one truck's breakdown doesn't strand everything), and you keep a manifest of what shipped versus what bounced (so you can re-send only the failures). The three scale limits — 100,000 requests or 256 MB per batch, processing within 24 hours, results retained for 29 days — are the truck capacities and warehouse hours you design around.

The two mistakes that separate a robust pipeline from a fragile one are both about identity and bookkeeping. First: using a random UUID for custom_id instead of a stable key from your own data — do that and a crash-and-resubmit duplicates work and double-charges you, because the re-run can't recognize what already succeeded. A content hash or row ID makes a resubmit an idempotent no-op. Second: not persisting the batch id the instant create() returns — the id is your only handle on the job, so if your poller dies before you wrote it down, you've lost a running batch you're still paying for. Get idempotency and durable bookkeeping right and the pipeline survives crashes, partial failures, and accidental reruns without duplicating spend or losing results — the checklist below is that discipline written out.

A lead owns the batch as a pipeline, not a one-off call. The scale limits set the shape: a single batch holds up to 100,000 requests or 256 MB, processes within 24 hours, and keeps results available for 29 days. Design around those numbers:

Production batch checklist

1. Idempotency via custom_id. Make custom_id a stable key from your own data (a row ID, a content hash) — not a random UUID. Then a resubmit after a crash overwrites cleanly instead of duplicating work, and reconciliation is a simple keyed merge.
2. Chunk large jobs. A 500k-item run won't fit in one batch — split into chunks under the 100k / 256MB limits and track each chunk's batch id. Chunking also bounds blast radius: one bad chunk fails alone.
3. Persist the batch id immediately. The id is your only handle. Write it to a durable store the moment create() returns, before you start polling — if your poller dies, you can resume from the id.
4. Reconcile every item. Walk the full results stream, route succeeded to output, and collect errored / expired custom_ids into a retry batch. Never assume 100% success.
5. Monitor request_counts. Alert on rising errored counts — a spike usually means a bug in how you built the requests, caught early before you burn the whole job.
6. Lean on prompt caching + a cheap model. A shared system prompt across all items caches well; classification-shaped work runs fine on claude-haiku-4-5. Stack those on top of the 50% batch discount for the lowest cost per item.

The mental model: a batch pipeline is an at-least-once job queue you don't have to run. Anthropic owns the workers and the scheduling; your job is to build clean requests with stable ids, persist the handle, and reconcile the outcomes. Get idempotency and reconciliation right and the pipeline survives crashes, partial failures, and reruns without duplicating spend or losing results.

IN, OUT = 5.0, 25.0   # $ per million tokens, Opus 4.8

def realtime_cost(in_tok, out_tok):
    return in_tok/1e6*IN + out_tok/1e6*OUT

def batch_cost(in_tok, out_tok):
    return realtime_cost(in_tok, out_tok) * 0.5   # Batches = 50% off

i, o = 2_000_000, 500_000
print(f"realtime: ${realtime_cost(i,o):.2f}")   # $22.50
print(f"batch   : ${batch_cost(i,o):.2f}")      # $11.25
print(f"saved   : ${realtime_cost(i,o)-batch_cost(i,o):.2f}")  # $11.25

import anthropic
from anthropic.types.message_create_params import MessageCreateParamsNonStreaming
from anthropic.types.messages.batch_create_params import Request

client = anthropic.Anthropic()
texts = ["Great product!", "Awful service.", "It was fine."]

batch = client.messages.batches.create(requests=[
    Request(
        custom_id=f"item-{i}",
        params=MessageCreateParamsNonStreaming(
            model="claude-haiku-4-5", max_tokens=16,
            messages=[{"role": "user",
                       "content": f"Sentiment (positive/negative/neutral), one word: {t}"}],
        ),
    )
    for i, t in enumerate(texts)
])
print(batch.id, batch.processing_status)   # msgbatch_..., "in_progress"

import time

while True:
    b = client.messages.batches.retrieve(batch.id)
    if b.processing_status == "ended":
        break
    time.sleep(30)                       # most batches finish within the hour

out = {}
for r in client.messages.batches.results(batch.id):
    match r.result.type:
        case "succeeded":
            msg = r.result.message
            out[r.custom_id] = next((b.text for b in msg.content if b.type == "text"), "")
        case "errored":
            # invalid_request -> fix & resubmit; server errors -> safe to retry
            out[r.custom_id] = f"ERR:{r.result.error.type}"
        case "expired" | "canceled":
            out[r.custom_id] = r.result.type
print(out)

shared_system = [
    {"type": "text", "text": "You are a contract analyst."},
    {"type": "text", "text": LARGE_CONTRACT_TEXT,
     "cache_control": {"type": "ephemeral"}},   # cache the big shared block
]
questions = ["Termination clause?", "Payment schedule?", "Governing law?"]

batch = client.messages.batches.create(requests=[
    Request(custom_id=f"q-{i}",
        params=MessageCreateParamsNonStreaming(
            model="claude-opus-4-8", max_tokens=512,
            system=shared_system,
            messages=[{"role": "user", "content": q}]))
    for i, q in enumerate(questions)
])

def collect(batch_id):
    ok, retry = {}, []
    for r in client.messages.batches.results(batch_id):
        if r.result.type == "succeeded":
            ok[r.custom_id] = r.result.message
        elif r.result.type == "errored" and r.result.error.type == "invalid_request":
            pass                          # permanent — do NOT blindly retry; log it
        else:                             # errored(server) / expired -> retryable
            retry.append(r.custom_id)
    return ok, retry

# Rebuild requests for the retry ids from your source-of-truth store, keeping ids:
def resubmit(retry_ids, source):
    if not retry_ids:
        return None
    return client.messages.batches.create(requests=[
        Request(custom_id=cid, params=source[cid]) for cid in retry_ids])

def choose_mode(latency_sensitive, volume, long_output):
    if latency_sensitive and long_output:
        return "streaming — token-by-token so the user sees progress, no HTTP timeout"
    if latency_sensitive:
        return "realtime (messages.create) — sub-second single response"
    if volume >= 1000:
        return "batch — 50% cheaper, async, up to 24h; ideal for bulk offline jobs"
    return "realtime — small volume, not worth batch orchestration"

print(choose_mode(True,  1,     True))    # coding assistant  -> streaming
print(choose_mode(True,  1,     False))   # chat turn         -> realtime
print(choose_mode(False, 50000, True))    # nightly enrichment-> batch

Knowledge check check yourself