The Cheat Sheet
Keep this open in a tab while you work through the labs. Every core idea in the course, compressed to what you need at a glance.
Core concepts
🔤 Tokens
The unit the model reads and writes. Roughly ¾ of a word in English (≈4 characters). "hamburger" ≈ 3 tokens.
You pay per token — input (what you send) + output (what it generates), at different rates.
Context window = max tokens (input + output) the model can hold at once. Overflow it and you must trim, summarize, or chunk.
💬 The request
Everything is messages.create(). Key fields:
model | which model |
max_tokens | output cap |
system | the rules/role |
messages | the conversation |
effort | how hard it works |
Stateless: the API remembers nothing. You resend the whole history every call — that is "memory".
🎚️ Effort & thinking
thinking:{type:"adaptive"} — model decides how much to reason.
effort: low → max. Higher = smarter + slower + pricier.
| low | classify, route |
| medium | most work |
| high/max | agents, hard reasoning |
📦 Structured output
Force schema-valid JSON instead of parsing free text.
Define shape with Pydantic; use Literal[...] for fixed value sets → the model literally can't return anything off-list.
Rule: if code consumes the output, constrain it. Never regex model text.
🧲 Embeddings
Text → a vector of numbers where similar meaning = close together.
Retrieval = embed the question, find the nearest chunk vectors (cosine similarity).
Golden rule: embed queries and documents with the same model, or similarity is garbage.
🏷️ Model tiers
| Opus | reasoning, agents, RAG gen |
| Sonnet | balanced, high-volume |
| Haiku | classify, rerank — cheap+fast |
Match tier to task difficulty. Using Opus for sentiment is like renting a truck to carry a letter.
Stop reasons — always check before reading content
stop_reason | Means | Do |
|---|---|---|
end_turn | Finished normally | Use the content ✅ |
max_tokens | Truncated at output cap | Raise max_tokens / stream |
tool_use | Wants to call a tool | Run it, send result back |
refusal | Declined for safety | Don't read content[0] blindly — handle it |
pause_turn | Long tool run paused | Re-send to resume |
if resp.stop_reason == "refusal" and == "max_tokens" branches from day one. Code that assumes text is always there will break on a refusal.RAG in a nutshell
| Phase | Steps |
|---|---|
| Offline (once) | documents → chunk → embed → store vectors |
| Online (per question) | embed question → retrieve (vector + keyword) → re-rank → build context → generate + cite |
Two rules that stop hallucination:
- "Answer only from the numbered context."
- "If it's not there, say you don't know." ← test this works!
The agent loop
An agent is a while loop around the API call. The model can't run tools — it asks you to via a tool_use block; you run it and hand back the result.
Three things people get wrong:
Append the assistant turn (resp.content) verbatim before results |
Every tool_result needs the matching tool_use_id |
| All parallel results go back in one user message |
Cost & caching
💾 Prompt caching
Reuse a big stable prefix at ~10% price. Prefix match: any byte change invalidates everything after it.
Order: stable first (system, tools) → volatile last (the question).
🪫 Cache killers
Silent zero-hit causes:
datetime.now() in prefix |
| UUID / request-id up front |
unsorted json.dumps() |
| per-user tool set |
✂️ Spend less
| route easy → small model |
| semantic cache repeats |
right-size max_tokens |
| batch non-urgent jobs |
Top 10 gotchas (bookmark this)
| # | Gotcha | Fix |
|---|---|---|
| 1 | content is a list, not a string | Loop, check block.type=="text" |
| 2 | Assuming text exists on a refusal | Check stop_reason first |
| 3 | Truncated output looks complete | Handle max_tokens; raise it / stream |
| 4 | Regex-parsing model output | Use output_format schema |
| 5 | Query & docs embedded differently | One embed() everywhere |
| 6 | "It hallucinates" (RAG) | It's retrieval — test top-k recall |
| 7 | Agent loop never ends | Hard MAX_STEPS cap |
| 8 | Missing / wrong tool_use_id | Copy block.id into result |
| 9 | Cache never hits | No volatile bytes in the prefix |
| 10 | Frontier model for trivial work | Route to Haiku |
resp.stop_reason, resp.usage, and (for RAG) the retrieved chunk IDs. 90% of bugs reveal themselves in those three values.