Bedrock & Claude

Converse: one API for every model essential

Here's the trap this API exists to save you from. You wire your app to Claude on Bedrock, ship it, and it works beautifully. Six months later a cheaper or smarter model lands — maybe a new Claude, maybe a different provider entirely — and your product owner asks to A/B test it. In the old world that meant a rewrite: every provider on Bedrock had its own request body, its own field names, its own way of nesting the reply. Swapping models meant re-plumbing your code. Converse was built precisely so that switch becomes a one-line change instead of a project.

Think of Bedrock as a hotel with many rooms, each a different foundation model, and Converse as a universal keycard. Before Converse, every door took a different physical key — you had to carry a keyring and know which key opened which room. Converse gives you one card that opens them all: the same converse() call, the same messages/system/inferenceConfig shape, whether the room holds Claude, an Amazon model, or a third party. The only thing you change to walk into a different room is the modelId string.

Now the precise definition. Converse is a single, model-agnostic inference API on the bedrock-runtime client that normalizes requests and responses across every provider Bedrock hosts. It standardizes three things you'd otherwise have to look up per model: how you send the system prompt, how you structure messages, and how you set inference parameters like max tokens and temperature. Under the hood Bedrock translates that uniform shape into whatever the underlying model actually expects, then translates the reply back into a uniform response. You get to think in one vocabulary.

The one shape you must internalize — and the mistake nearly everyone makes first — is that message content is a list of typed blocks, never a bare string. You write "content": [{"text": "..."}], not "content": "...". That feels like needless ceremony for a plain-text question, but it's the design decision that pays off later: because content is already a list of blocks, adding an image block or a tool-result block in the same message is just appending another entry — no new API, no different call. The list-of-blocks shape is what lets one API stretch from "ask a question" all the way to multimodal, tool-using agents. If your first Converse call throws a validation error, this is almost always why: you passed a string where Bedrock wanted a list of blocks.

Bedrock's Converse API is the recommended entry point. It gives one consistent request shape across all providers, so switching from Claude to another model is a one-line change. The messages format will feel familiar from the Anthropic SDK.

converse.pyimport boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL = "anthropic.claude-3-5-sonnet-20241022-v2:0"

resp = brt.converse(
    modelId=MODEL,
    system=[{"text": "You are a careful SRE assistant. Be concise."}],
    messages=[{"role": "user", "content": [{"text": "In one sentence, what is Amazon Bedrock?"}]}],
    inferenceConfig={"maxTokens": 200, "temperature": 0.2},
)

print(resp["output"]["message"]["content"][0]["text"])
u = resp["usage"]
print("tokens in/out:", u["inputTokens"], u["outputTokens"])

Amazon Bedrock is a fully managed AWS service that provides API access to foundation models — including Anthropic's Claude — for building generative-AI apps.
tokens in/out: 28 41

Streaming essential

You've felt this failure as a user. You type a question into a chatbot, hit enter, and then… nothing. A spinner. Three seconds, five, eight — long enough that you start wondering whether it crashed. Then, all at once, a wall of text appears. The model wasn't slow to think; it finished the first words almost immediately. It was slow to show, because your code waited for the entire completion before rendering a single character. Streaming fixes exactly this: it lets the words appear as they're generated, so the wait feels like a tenth of what it is.

The analogy is a phone call versus a letter. A non-streamed call (converse) is a letter: the other side writes the whole thing, seals it, and you read it only when it arrives complete. A streamed call (converse_stream) is a live phone conversation: you hear each word as it's spoken. The total talking time is identical — the model generates tokens at the same rate either way — but on the phone you're engaged from the first syllable instead of staring at a sealed envelope. That difference in perceived latency is why every serious chat UI streams.

Mechanically, here's what changes. converse_stream makes the same request as converse, but instead of returning one finished reply it returns a stream of events you iterate over as they arrive. Those events describe the life of the response: the message starting, a content block opening, small deltas of text arriving one after another, the block closing, the message ending, and finally a metadata event carrying token usage. The word delta is the key one — it means "a little more" — and the incremental text you actually want to print lives specifically inside the contentBlockDelta events, under delta.text.

The common mistake is to treat every event as if it carries text and blow up on the ones that don't (the start, stop, and metadata events have no delta). The correct pattern is to filter for the event kind you care about — keep only contentBlockDelta, read its text chunk, and ignore the rest of the lifecycle. And because streaming hands you the pieces rather than a finished object, if you also need the full text or the token usage at the end, it's on you to accumulate the deltas into a string and read the final metadata event yourself. Nothing is harder here than a plain converse call — you're just consuming the answer as a sequence instead of a single blob.

For anything a human waits on, stream. converse_stream yields events as the model generates. You accumulate contentBlockDelta events for the text.

stream.pyimport boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
MODEL = "anthropic.claude-3-5-sonnet-20241022-v2:0"

stream = brt.converse_stream(
    modelId=MODEL,
    messages=[{"role": "user", "content": [{"text": "Count to five slowly."}]}],
    inferenceConfig={"maxTokens": 100},
)

for event in stream["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"]["text"], end="", flush=True)
print()

InvokeModel: the raw escape hatch intermediate

Sooner or later the tidy abstraction won't have the knob you need. You're reading Claude's documentation, you find a parameter that does exactly what your feature requires — and it's nowhere in the Converse request shape. Or you inherit a codebase written before Converse existed and every model call is in some older, chattier form you don't recognize. This is the moment you drop one level down, from the polished universal keycard to the master key: invoke_model.

The analogy: Converse is an automatic car — it shifts gears for you and you rarely think about it. invoke_model is manual transmission. You now control the clutch, which means you can do things the automatic won't let you, but you also have to operate everything yourself and it's easier to stall. Concretely, with invoke_model you hand Bedrock the provider's native request body verbatim — you build the exact JSON that model expects, serialize it yourself, send it, then read and parse the raw response yourself. Converse's conveniences — uniform field names, automatic JSON encoding, automatic decoding — all become your job.

Here's the precise mental model. invoke_model is the lower-level, provider-specific call: there is no shared shape, so what you send depends entirely on which model you're invoking. For Claude, that body is the Anthropic Messages format, exactly as you'd send it to the Anthropic API directly — which is a genuine payoff, because everything you already know about calling Claude natively transfers here unchanged. Note the details that shift from Converse: the field names are snake_case (max_tokens, not maxTokens), and there is one non-negotiable field that trips up everyone porting from the SDK — anthropic_version, set to "bedrock-2023-05-31". Omit it and the call fails; it's Bedrock's way of pinning which Claude request schema you're speaking.

The mistake to avoid is reaching for invoke_model by default because it feels closer to the metal. It isn't a better API — it's a more manual one, and every call site you write in native JSON is a call site you'll have to rewrite by hand the day you switch models. The rule of thumb: prefer Converse for all new code, and drop to invoke_model only for the two specific reasons above — a provider-specific field Converse doesn't surface, or legacy code you must match. It's an escape hatch, not a front door.

Before Converse existed, you called invoke_model with each provider's native JSON body. You will still see it in older code and when you need a Claude-specific field Converse does not surface. For Claude the body is the Anthropic Messages format verbatim.

invoke_model.pyimport boto3, json

brt = boto3.client("bedrock-runtime", region_name="us-east-1")

body = {
    "anthropic_version": "bedrock-2023-05-31",   # required for Claude on Bedrock
    "max_tokens": 200,
    "messages": [{"role": "user", "content": "Say hello."}],
}
resp = brt.invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    body=json.dumps(body),
)
payload = json.loads(resp["body"].read())
print(payload["content"][0]["text"])

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = brt.converse(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    messages=[{"role": "user", "content": [{"text": "Define latency in one line."}]}],
    inferenceConfig={"maxTokens": 128, "temperature": 0.2},
)
print(resp["output"]["message"]["content"][0]["text"])

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = brt.converse(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    system=[{"text": "You answer in exactly one sentence."}],
    messages=[{"role": "user", "content": [{"text": "What is a VPC?"}]}],
    inferenceConfig={"maxTokens": 128},
)
u = resp["usage"]
print(u["inputTokens"], u["outputTokens"])

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
resp = brt.converse_stream(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    messages=[{"role":"user","content":[{"text":"Count to five."}]}],
)
for event in resp["stream"]:
    if "contentBlockDelta" in event:
        print(event["contentBlockDelta"]["delta"]["text"], end="")

import boto3, json

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
body = {
    "anthropic_version": "bedrock-2023-05-31",   # required
    "max_tokens": 128,
    "messages": [{"role":"user","content":"Say hi."}],
}
resp = brt.invoke_model(
    modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
    body=json.dumps(body),
)
out = json.loads(resp["body"].read())
print(out["content"][0]["text"])

import boto3, logging

brt = boto3.client("bedrock-runtime", region_name="us-east-1")
log = logging.getLogger("llm")

def ask(prompt, model="anthropic.claude-3-5-sonnet-20241022-v2:0"):
    r = brt.converse(
        modelId=model,
        messages=[{"role":"user","content":[{"text": prompt}]}],
        inferenceConfig={"maxTokens": 512},
    )
    u = r["usage"]
    log.info("tokens in=%s out=%s", u["inputTokens"], u["outputTokens"])
    return r["output"]["message"]["content"][0]["text"]

# print(ask("Summarize the CAP theorem."))

import boto3

brt = boto3.client("bedrock-runtime", region_name="us-east-1")

class Chat:
    def __init__(self, max_turns=10):
        self.messages, self.max = [], max_turns * 2
    def send(self, text):
        self.messages.append({"role":"user","content":[{"text": text}]})
        r = brt.converse(
            modelId="anthropic.claude-3-5-sonnet-20241022-v2:0",
            messages=self.messages,
            inferenceConfig={"maxTokens": 512},
        )
        msg = r["output"]["message"]
        self.messages.append(msg)                 # keep assistant turn
        self.messages = self.messages[-self.max:] # bound history
        return msg["content"][0]["text"]

# offline: history bookkeeping without a network call
c = Chat(max_turns=1)
c.messages = [{"role":"user","content":[{"text":"a"}]}]*5
c.messages = c.messages[-c.max:]
print(len(c.messages))   # 2

Knowledge check check yourself