Environment & Your First API Call

Why we start with one call essential

Imagine trying to debug a factory when you've never watched a single machine run. That's what happens to people who skip this chapter: they wire together a RAG system or an agent, something misbehaves, and they have no idea whether the fault is in the retrieval, the loop, the prompt, or the one API call buried three layers down — because they never learned what a healthy single call even looks like. Every mysterious agent bug you'll ever chase traces back to a request that stopped early, cost too much, or returned something you didn't expect. Learn to read one call cold, and those bugs become obvious instead of terrifying.

Here's the analogy that makes it click: the single API call is the atom of this entire field. A RAG pipeline is atoms bonded with a retrieval step; an agent is the same atoms arranged in a loop; a multi-agent system is loops of loops. Chemists don't start with proteins — they start with the atom, because everything larger is just atoms combined. Skip the atom and every molecule above it is a black box.

The common mistake — and it's an understandable one — is to treat the single call as "the boring part" and rush to the exciting agent code. But the call is where all the money is spent (you pay per token, per call), where all the failures originate (a truncated reply, a refusal, a timeout), and where all the state lives (or rather, doesn't — see below). Master it here, in isolation, with nothing else to blame, and you'll move faster through every chapter that follows.

It's tempting to jump straight to agents. Resist that. A RAG pipeline is many single calls plus retrieval; an agent is many single calls in a loop. If you don't deeply understand one request — what you pay for, why it stops, how it streams, how it fails — you'll be debugging blind later. This chapter is the foundation the whole course stands on.

Lab 1.1 · Set up the environment essential

Every developer has a horror story that begins "it worked on my machine." The usual culprit is a tangle of packages installed globally: this project needs one version of a library, that project needs another, and installing for one silently breaks the other. The fix has been standard practice for years, and it's the very first thing you'll do here — give this course its own sealed workspace so nothing it installs can reach out and disturb anything else on your system.

Think of a virtual environment as a clean, private toolbox for one project. Instead of throwing every tool you own into one giant shared drawer (your system Python), you get a fresh box that starts empty and holds only the tools this course needs — at exactly the versions it expects. Delete the box and your system is untouched; hand the box's contents list (its packages) to a teammate and they get an identical setup. That reproducibility is why "which packages, at which versions" stops being a mystery.

The common beginner mistake is to skip this and pip install straight into the system Python — it feels faster, and it works right up until a second project wants a conflicting version and you're stuck in "dependency hell." The three steps below cost thirty seconds and save that entire class of pain: create an isolated environment, activate it, install the SDK inside it, and confirm the import. Do this once and the rest of the course just works.

Lab 1.2 · API keys, the safe way essential

There is a genre of horror story that plays out on GitHub every single day: a developer pastes their API key straight into a script, commits it, pushes to a public repo, and within minutes automated bots that scrape GitHub for exactly this have found it and started spending. People have woken up to four-figure bills from a key that lived in a public commit for under an hour. This section exists so that story is never yours — and the habit it teaches is the same one production teams enforce with tooling and audits.

The mental model that keeps you safe: treat your API key exactly like the key to your house. You don't tape your house key to the front door, and you don't staple a photocopy of it into every letter you mail. Yet hardcoding a key into a .py file does precisely that — the key rides along in every copy of the code, into git history (where deleting it later doesn't actually remove it), into logs, into screenshots, into that snippet you paste into a chat to ask for help. A secret's danger is proportional to how many places it lives, so the whole game is keeping it in one place your code can read but your repo cannot.

The standard solution separates the secret from the code: the key lives in a small .env file on your machine, that file is named in .gitignore so git refuses to track it, and your program reads the key from the environment at runtime rather than having it baked in. The single most common mistake here isn't forgetting the .env file — it's forgetting to git-ignore it, so the secret gets committed anyway. Do both halves and the key exists exactly where it should: on your machine, in your environment, and nowhere a copy of your code can carry it.

Your API key is a credential that spends real money. Never hardcode it in a .py file, never commit it to git. The standard pattern is an environment variable loaded from a .env file that's git-ignored.

Lab 1.3 · Your first call intermediate

This is the moment the abstract becomes real. Everything so far has been setup; the fifteen lines below are the first time your code actually talks to a model and gets words back. And here's the reassuring truth that surprises people: this exact shape — build a client, describe what you want, read the reply — is the entire API. Not the beginner version of it. The production RAG systems and autonomous agents later in this course make this same call; they just make it many times, with more fields filled in. Learn these lines and you've learned the thing.

The mental picture is a well-mannered letter and reply. Your request is a letter: it names who should read it (model), how long the answer may be (max_tokens), the standing instructions the reader should follow (system), and the actual conversation so far (messages). You drop it in the post, and a reply comes back containing the text, a note on why the writer stopped, and an itemised bill for the words used. There's no phone line kept open, no ongoing session — one letter out, one letter back, and the correspondent forgets you the instant they've replied.

Two details trip up almost everyone on their first call, so meet them now rather than in a stack trace. First, resp.content is a list of blocks, not a plain string — you loop over it and pull the text ones, because later replies can also carry thinking and tool_use blocks in that same list. Second, the reply always tells you why it ended via stop_reason, and it always tells you what it cost via resp.usage — two fields beginners ignore and professionals read on every single call. The lab uses the exact model ID string claude-opus-4-8; type the code out rather than paste it, because typing is how the shape lodges in your memory.

first_call.pyfrom dotenv import load_dotenv
from anthropic import Anthropic

load_dotenv()
client = Anthropic()                       # reads the key from the env

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=300,
    system="You are a concise technical tutor.",
    messages=[
        {"role": "user", "content": "Explain what an API token is in 2 sentences."},
    ],
)

for block in resp.content:            # content is a LIST of blocks
    if block.type == "text":
        print(block.text)

print("---")
print("stop reason:", resp.stop_reason)
print("input tokens:", resp.usage.input_tokens)
print("output tokens:", resp.usage.output_tokens)

terminalpython first_call.py

An API token is a unit of text... (a couple of sentences)
---
stop reason: end_turn
input tokens: 24
output tokens: 41

Anatomy of a request intermediate

The strangest thing about talking to an LLM is the thing nobody warns you about: the model has no memory. None. It does not remember your last question, your name, or that you introduced yourself thirty seconds ago. Every request arrives at a model that has never met you and will forget you the moment it replies. Newcomers build a chatbot, watch it "remember" earlier turns, and assume the server is keeping notes — then are baffled when a fresh script "forgets" everything. It was never remembering; they were resending the history without realising it.

The word for this is stateless, and the cleanest analogy is a vending machine, not a bartender. A bartender remembers your usual; a vending machine does not. Each time you walk up you must insert the full payment and press the full selection — the machine holds no tab, keeps no memory of your last visit, and treats every transaction as the first. An LLM call is the vending machine: the request must contain everything the model needs to answer, every time, because there is no tab.

That single fact reshapes how you think about a request, and it's why the field breakdown below matters. Look at what you're sending: model and max_tokens and system are the machine's settings, but messages is the whole conversation-so-far, re-fed on every call. To continue a chat you don't "keep talking" — you take the model's reply, append it to your messages list, and resend the entire transcript. The practical consequence, and the mistake that surprises people later, is that you own the memory, not the server: every "stateful" feature you build in later chapters is really just you managing that growing list. The diagram traces this loop, and the table decodes each field you send.

Every field you sent, explained:

The messages array is a transcript

To have a multi-turn conversation, you append to messages and resend the whole thing:

multi_turn.pymessages = [
    {"role": "user",      "content": "My name is Sam."},
    {"role": "assistant", "content": "Nice to meet you, Sam!"},
    {"role": "user",      "content": "What's my name?"},   # model sees the history
]
# The API is stateless — this whole list is sent every call.

Reading the response — stop reasons intermediate

Here is a bug that ships to production constantly, always looks fine in testing, and is maddening to trace: the model runs out of room, stops mid-sentence, and your code cheerfully treats the half-finished answer as complete — storing it, parsing it, showing it to a user — because it never asked why the reply ended. It works every time in your quick tests (short answers never hit the cap) and then fails the day a real user asks something that needs a long answer. The fix is one field you must learn to read before anything else: stop_reason.

The analogy: a reply that ends is like a phone call that ends, and "the call ended" tells you nothing until you know how. Did the other person finish their thought and say goodbye (end_turn)? Did the line get cut off because your prepaid minutes ran out (max_tokens)? Did they pause to go look something up and expect you to wait (tool_use, pause_turn)? Or did they politely decline to discuss the topic and hang up (refusal)? Same silence on your end, four completely different situations — and you handle each one differently. Reading stop_reason is asking "how did this end?" before you act on what was said.

The specific trap to internalise: code that reaches for resp.content[0].text the instant a reply lands is making two dangerous assumptions at once — that there is a text block (a refusal may leave content empty, so this crashes with an index error) and that the text is complete (a max_tokens stop means it isn't, so downstream logic silently works on a fragment). Both bugs vanish the moment you branch on stop_reason first. This one habit — check why it stopped, then use the content — is the difference between a demo and a service, and it's why the table below is worth memorising.

Before you use content, check why the model stopped. This one habit prevents a whole class of production bugs.

Lab 1.4 · Streaming advanced

You already know how streaming feels, even if you've never written a line of it. It's the difference between a chat interface that types its answer out live, word by word, and one that shows a spinner for eight seconds and then dumps a paragraph all at once. The first feels alive and responsive; the second feels broken even when it's working perfectly. That gap is entirely about when the words arrive, not how good they are — and closing it is what this lab is for.

The mechanism is best pictured as water from a tap versus a bucket. A normal (non-streaming) call fills a bucket: the model generates the entire reply on the server, then hands you the whole thing at once — you wait, staring at nothing, until it's done. Streaming opens the tap: each chunk of text flows to you the instant the model produces it, so you can print it, render it, or pipe it onward while the rest is still being written. Same total water, dramatically different experience.

Streaming isn't only about feel, though — there's a hard engineering reason too, and it's the one beginners hit by surprise. A long reply that takes the model a full minute to finish can blow past the HTTP request's timeout while you sit waiting for the bucket to fill, and the whole call fails with a connection error. Streaming sidesteps this entirely: because data is flowing continuously, the connection stays alive and there's no single long silence to time out on. The rule that falls out of this: for anything short, either style is fine; for anything long, or anything a human is watching, stream. The lab below also shows the detail people forget — even while streaming you can still call get_final_message() at the end to recover the complete reply, its stop_reason, and its token usage.

For anything that produces more than a sentence or two, stream. It shows tokens as they arrive (better UX) and avoids HTTP timeouts on long outputs.

streaming.pyfrom dotenv import load_dotenv
from anthropic import Anthropic
load_dotenv()
client = Anthropic()

with client.messages.stream(
    model="claude-opus-4-8",
    max_tokens=500,
    messages=[{"role":"user", "content":"Write a haiku about databases, then explain it."}],
) as stream:
    for text in stream.text_stream:      # yields text chunks as they arrive
        print(text, end="", flush=True)

    final = stream.get_final_message()     # full Message once done
    print("\n---\ntokens:", final.usage.output_tokens)

Lab 1.5 · Handling errors like a service advanced

The single biggest difference between a script and a service is what happens on a bad day. A script that assumes the network is always up, the API is never busy, and the model always answers works flawlessly in the demo and then crashes with a raw stack trace the first time a real user leans on it. Production code assumes the opposite — that things will fail — and decides in advance what to do about each kind of failure. That planning-for-the-bad-day mindset is what this final lab installs.

The key insight is that not all failures are the same, so a blanket "catch everything" is the wrong tool. Think of it like a hospital triage nurse: someone with a paper cut, someone having a heart attack, and someone who wandered into the wrong building all walk through the same door, but you'd be negligent to treat them identically. API failures triage the same way. A rate limit (429) or a server hiccup (5xx) is transient — wait a moment and retry, and it'll likely succeed. A bad request or an authentication error (4xx) is fatal — retrying changes nothing, so surface it loudly rather than hiding it. A dropped network connection is transient too. Different diagnoses, different treatments.

This is exactly why the common beginner reflex — a single broad except Exception: wrapped around everything — is a trap: it flattens that triage, silently swallowing the fatal 4xx you needed to see right alongside the retryable 429 you could have recovered from, leaving you blind to which actually happened. The professional pattern, which the lab demonstrates, is to catch named exceptions ordered most-specific to least-specific, so each failure gets its own response. (The SDK already auto-retries 429/5xx for you with backoff, but you still classify and handle what escapes.) Combine that with the stop_reason guards from the previous section and you have the skeleton every production call in this course is built on.

Networks fail, rate limits trip, servers overload. The SDK auto-retries 429/5xx, but you should still catch and classify errors.

safe_call.pyimport anthropic
from anthropic import Anthropic
client = Anthropic()

def ask(prompt: str) -> str:
    try:
        resp = client.messages.create(
            model="claude-opus-4-8", max_tokens=500,
            messages=[{"role":"user","content":prompt}],
        )
    except anthropic.RateLimitError as e:      # 429
        return "Busy right now — try again shortly."
    except anthropic.APIStatusError as e:      # other non-2xx
        if e.status_code >= 500:
            return "Service issue — please retry."
        raise
    except anthropic.APIConnectionError:          # network
        return "Network problem — check your connection."

    if resp.stop_reason == "refusal":
        return "I can't help with that request."
    return next((b.text for b in resp.content if b.type=="text"), "")

print(ask("Give me one tip for learning to code."))

Exercises advanced

cost = (resp.usage.input_tokens * 5 +
        resp.usage.output_tokens * 25) / 1_000_000
print(f"cost: ${cost:.6f}")

messages = []
while True:
    user = input("you> ")
    if user == "quit": break
    messages.append({"role":"user","content":user})
    resp = client.messages.create(model="claude-opus-4-8",
             max_tokens=500, messages=messages)
    reply = next(b.text for b in resp.content if b.type=="text")
    print("bot>", reply)
    messages.append({"role":"assistant","content":reply})

class Usage:
    input_tokens = 24
    output_tokens = 41
class Resp:
    usage = Usage()

resp = Resp()
print("input:", resp.usage.input_tokens)
print("output:", resp.usage.output_tokens)
print("total:", resp.usage.input_tokens + resp.usage.output_tokens)

class Block:
    def __init__(self, type, text=""):
        self.type = type; self.text = text
class Usage:
    input_tokens = 24; output_tokens = 41
class Resp:
    content = [Block("text", "An API token is a chunk of text. "),
               Block("text", "Models are billed per token.")]
    usage = Usage()

def text_of(resp):
    return "".join(b.text for b in resp.content if b.type == "text")

resp = Resp()
print(text_of(resp))
cost = (resp.usage.input_tokens * 5 + resp.usage.output_tokens * 25) / 1_000_000
print(f"cost: ${cost:.6f}")

def first_text(resp):
    return next((b.text for b in resp.content if b.type == "text"), "")

def handle(resp):
    sr = resp.stop_reason
    if sr == "end_turn":
        return "use: " + first_text(resp)
    if sr == "max_tokens":
        return "TRUNCATED (raise max_tokens or stream): " + first_text(resp)
    if sr == "tool_use":
        return "run the requested tool, send result back"
    if sr == "refusal":
        return "handle refusal gracefully (content may be empty)"
    if sr == "pause_turn":
        return "re-send to resume"
    return "unknown stop reason"

class B:
    def __init__(self, t, x=""): self.type=t; self.text=x
class R:
    def __init__(self, sr, content): self.stop_reason=sr; self.content=content

print(handle(R("end_turn", [B("text","done.")])))
print(handle(R("max_tokens", [B("text","half a sen")])))
print(handle(R("refusal", [])))  # note: empty content, must not crash

def append_turn(messages, role, text):
    assert role in ("user", "assistant")
    if not messages:
        assert role == "user", "first message must be user"
    else:
        assert role != messages[-1]["role"], "roles must alternate"
    messages.append({"role": role, "content": text})
    return messages

m = []
append_turn(m, "user", "My name is Sam.")
append_turn(m, "assistant", "Nice to meet you, Sam!")
append_turn(m, "user", "What's my name?")
print(len(m), "turns; first role =", m[0]["role"])

for bad in [("assistant","x")]:  # first must be user
    try:
        append_turn([], *bad)
    except AssertionError as e:
        print("rejected:", e)

import time, logging
import anthropic
from anthropic import Anthropic

client = Anthropic()
log = logging.getLogger("ask")

def ask(prompt: str, retries: int = 3) -> str:
    for attempt in range(retries):
        try:
            resp = client.messages.create(
                model="claude-opus-4-8", max_tokens=500,
                messages=[{"role": "user", "content": prompt}],
            )
            if resp.stop_reason == "refusal":
                return "I can't help with that request."
            if resp.stop_reason == "max_tokens":
                log.warning("truncated output")
            return next((b.text for b in resp.content if b.type == "text"), "")
        except anthropic.RateLimitError:            # 429 — retryable
            wait = 2 ** attempt
            log.warning("rate limited; backoff %ss", wait); time.sleep(wait)
        except anthropic.APIStatusError as e:       # other non-2xx
            if e.status_code >= 500 and attempt < retries - 1:
                time.sleep(2 ** attempt); continue    # 5xx — retryable
            raise                                     # 4xx — fatal, don't hide
        except anthropic.APIConnectionError:        # network — retryable
            time.sleep(2 ** attempt)
    return "Service unavailable after retries."

print(ask("Give me one tip for learning to code."))

from collections import defaultdict

# price per 1M tokens, per model: (input, output)
PRICES = {
    "claude-opus-4-8":  (5.0, 25.0),
    "claude-haiku-4-5": (1.0,  5.0),
}

class CostMeter:
    def __init__(self):
        self.ledger = defaultdict(lambda: {"in": 0, "out": 0, "usd": 0.0})

    def record(self, feature, model, in_tok, out_tok):
        pin, pout = PRICES[model]
        usd = (in_tok * pin + out_tok * pout) / 1_000_000
        row = self.ledger[feature]
        row["in"] += in_tok; row["out"] += out_tok; row["usd"] += usd
        return usd

    def report(self):
        for feat, r in sorted(self.ledger.items(), key=lambda kv: -kv[1]["usd"]):
            print(f"{feat:16} {r['in']:>7} in {r['out']:>7} out  ${r['usd']:.4f}")

    # def metered_call(self, feature, **kw):
    #     resp = client.messages.create(**kw)     # real SDK — needs API key
    #     self.record(feature, kw["model"],
    #                 resp.usage.input_tokens, resp.usage.output_tokens)
    #     return resp

m = CostMeter()
m.record("rag-search",  "claude-haiku-4-5", 120_000, 30_000)
m.record("rag-search",  "claude-haiku-4-5",  80_000, 20_000)
m.record("chat",        "claude-opus-4-8",   40_000, 60_000)
m.report()

Knowledge check check yourself