HTTP, DNS, REST & JSON
The plumbing under every API and LLM call — DNS, HTTP, REST, JSON — climbing to a resilient, reusable client with retries, rate-limit handling, and pagination. This is the SDK, unwrapped.
Building an API, calling an LLM, or deploying a service all use the web's plumbing: DNS, HTTP, REST, JSON. This chapter demystifies each, then climbs to a resilient, reusable API client. One block makes a real call (marked); the rest run offline using stdlib and fakes, so every runnable example works with zero errors.
The words you'll hear (in plain terms):
| Term | What it actually means |
|---|---|
| HTTP | the request/response protocol (methods, status codes, headers). |
| DNS | turns a name (example.com) into an IP address. |
| REST | a convention for APIs over HTTP verbs on resource URLs. |
| JSON | the text format structured data travels in. |
| idempotent | safe to retry (GET); POST is not. |
What you need before starting:
- Python basics;
pip install requestsfor the one live call. - Having called an API helps but isn't required.
- Runnable blocks are offline; the one network call is labeled.
New to the topic? Read this box, then take the chapters in order — each section is tagged essential → expert so you always know the depth you're at.
Learning objectives
- Trace DNS → TCP → TLS → HTTP for a request.
- Use methods, status codes, headers, and JSON from Python.
- Build a resilient client with retries, backoff, and timeouts.
- Standardize a reusable, testable client for a team.
code/df4-how-web-works/. Python runs offline; configs are ready to use.1 · What happens when you hit a URL essential
Here's a moment you've probably had without noticing. You type an address, hit Enter, and a page appears — feels instant, feels like magic. But the day it doesn't work, that magic turns into a wall: "DNS_PROBE_FINISHED", "connection timed out", "certificate invalid", "502 Bad Gateway". Each of those errors is naming a different step that failed — and if you don't know the steps, every one of them looks like the same unhelpful "the internet is broken". This section hands you the map so those errors start telling you exactly where to look.
Think of visiting a friend in a city you've never been to. First you need their address — the name "Priya's place" is useless to a taxi, so you look it up in your contacts (that's DNS turning a name into a number). Then you actually travel there and make sure the door's the right one and the line is private (that's TCP opening the road and TLS locking it). Only then do you knock and have a conversation — "got any coffee?" / "sure, here" (that's the HTTP request and response). You can't have the conversation before you've arrived, and you can't arrive before you know the address. The order isn't optional; it's the physics of the trip.
Why does this matter for an AI engineer specifically? Because every LLM call, every API call, every pip install is exactly this same four-step trip, just hidden behind one tidy function. When client.messages.create(...) hangs or throws, it failed at one of these four boxes — and knowing which one ("is it DNS? is the TLS cert expired? did the server 500?") is the difference between a five-minute fix and an afternoon of flailing. The common beginner mistake is treating the whole chain as a black box and blaming "the API" — when 90% of the time the trace points at something specific and boring, like a wrong hostname (DNS), a missing timeout (the pipe never opened), or an expired token (an HTTP 401 the server sent back on purpose). Learn to read the chain and you stop guessing.
Type example.com → the browser resolves DNS to an IP, opens a TCP connection, negotiates TLS (for https), sends an HTTP request, and gets a response (status + headers + body). That round trip is the whole web.
This strip shows what really happens in the split second between typing a web address and seeing a page. Read it left to right — each box is one step, and the arrows are the order they happen in. Every API call and every LLM call goes through this same chain.
- Box 1 · Type URL (
example.com) — the human-friendly name you type. A name is not an address the network can use yet, so it must be looked up. - Box 2 · DNS → IP ("the phone book") — DNS is the internet's phone book: it translates the name
example.cominto a numeric IP address (like93.184.216.34) that identifies the actual server machine. - Box 3 · TCP+TLS connect ("the pipe") — your computer opens a connection to that IP. TCP is the reliable pipe the data flows through; TLS is the lock that encrypts it (that's the 's' and the padlock in
https). - Box 4 · HTTP req/resp (
GET / → 200) — now the real conversation: your browser sends an HTTP request ("GET me the home page") and the server sends back a response.200is the status code meaning "OK, here it is". - The arrows show this is a sequence — you can't connect before you know the IP, and you can't send the request before the pipe is open. Name → address → pipe → conversation.
In short: A URL is just a name; the network needs an address (DNS), a pipe (TCP+TLS), then a request/response (HTTP). Every SDK you'll use — including the LLM one — hides these four steps behind a single function call.
2 · HTTP methods & status codes essential
A war story you'll eventually live. Someone writes a script that submits an order with a POST. The network hiccups, the script doesn't hear back, so — being helpful — it retries. The server actually received both requests. The customer gets charged twice. That entire bug is one missing piece of knowledge: POST is not idempotent. If HTTP methods feel like trivia to memorize, this is why they aren't — the "idempotent?" column below is the exact thing that decides whether your retry logic is safe or a foot-gun.
Start with the plain-English picture. An HTTP method (also called a verb) is you telling the server what kind of thing you want to do, and a status code is the server's three-digit reply telling you how it went. That's the whole conversation: you send a verb + a URL, it sends back a number + some data. Think of a librarian: "GET me that book" (read), "PUT this book back exactly here" (replace), "DELETE this book" (remove). And the librarian always answers with a coded reply — a thumbs-up, "we don't have that", "you're not allowed", or "sorry, the shelf collapsed".
The single most useful idea in the whole section is idempotent, and it sounds scarier than it is. Idempotent just means doing it twice is the same as doing it once — like a light switch labelled "off": flip it off, flip it off again, the room's still just off. GET, PUT, and DELETE are idempotent (reading a page twice, or deleting an already-deleted record, changes nothing extra), so they're safe to retry. POST ("create a new thing") is not — retry it and you may create two things. That's why the retry code later in this chapter only auto-retries the safe verbs.
Now the status codes, and here's the trick that saves you from memorizing all of them: the first digit tells the whole story. 2xx = "great, it worked." 3xx = "it moved, look over there." 4xx = "you messed up" (bad request, not logged in, not allowed, doesn't exist). 5xx = "the server messed up." The mistake to avoid is lumping 4xx and 5xx together as "an error" and retrying both — a 4xx is your bug (retrying sends the same broken request and fails identically), while a 5xx is often transient (the server was briefly overloaded; retrying may well succeed). One digit tells you whether to fix your code or just wait and try again. Keep the two reference tables below close — you'll consult them constantly for the rest of your career.
Start at the response box on the left and follow the arrow to the four coloured bands — each band is one class of status code, decided entirely by its first digit.
- The green 2xx band is success — the body is real data, so use it.
- The amber 3xx band is a redirect — the resource moved, look elsewhere.
- The red 4xx band is your mistake (bad request, not logged in, not found). The dashed arrow leads to fix & don't retry — retrying the same broken request just fails identically.
- The muted 5xx band is the server's mistake, often transient. Its dashed arrow leads to wait & retry — backing off may well succeed.
- The
≠versusretry okon the right is the payoff: 4xx and 5xx look like "an error" but demand opposite reactions — that's the trap to avoid.
In short: Read the first digit, pick the reaction. 4xx = fix your code; 5xx = wait and try again. Never retry both blindly.
| Method | Means | Idempotent? |
|---|---|---|
| GET | read | yes (safe to retry) |
| POST | create | no |
| PUT | replace | yes |
| PATCH | update part | no |
| DELETE | remove | yes |
| Status | Means |
|---|---|
| 2xx | success (200 ok, 201 created, 204 no content) |
| 3xx | redirect (301 moved, 304 not modified) |
| 4xx | you erred (400 bad, 401 unauth, 403 forbidden, 404 missing, 429 rate-limited) |
| 5xx | server erred (500, 502 bad gateway, 503 unavailable) |
3 · JSON — how data travels essential
Picture two people who don't speak the same language trying to hand each other a filled-in form. Your Python program thinks in dicts and lists; the server across the world might be written in Go or Rust or JavaScript and thinks in its own structures. They need a neutral, agreed-upon paper form that both sides can read and write — plain text, unambiguous, no surprises. That agreed form is JSON, and it's why an API written in any language can talk to your Python in any other. It is, quite literally, the shared handwriting of the entire web.
Concretely, JSON (JavaScript Object Notation) is just text that describes structured data using a tiny, strict vocabulary: objects in {curly braces} (which become Python dicts), arrays in [square brackets] (Python lists), plus strings, numbers, true/false, and null. That's the entire language — you can learn it in five minutes, which is exactly the point. Because it's just text, it can travel down a network pipe; because it's structured, your program can turn it back into real values it can work with. The two verbs you'll use forever are parse (text → Python values, json.loads) and serialize (Python values → text, json.dumps) — "loads" for load-from-string, "dumps" for dump-to-string.
The mistake that bites everyone at least once: confusing the JSON text with the Python object. A response body arrives as a string that looks like a dict — '{"user": "ava"}' — and beginners try to do body["user"] on it and get a baffling error, because you can't index a string by a key. You must json.loads() it first to get the real dict. The reverse trap is just as common: JSON only knows that handful of types, so a Python datetime, set, or tuple can't be serialized as-is (dumps raises a TypeError on the datetime, and silently turns your tuple into a list). The fix is to convert those to JSON-native types — a date to its isoformat() string — before sending. Get the parse/serialize direction straight in your head and a whole category of API bugs simply never happens.
JSON is the API lingua franca. Python's json maps it to dicts/lists and back.
json.loads turns received text into real Python dicts/lists you can index; json.dumps turns your Python data back into text to send. Confusing the text with the object is the bug that bites everyone.
Read it as a two-way street between the two boxes: text on the left (what travels over the network) and real Python values on the right (what your code works with).
- The left amber box is JSON text — a plain string like
{"user":"ava"}. It looks like a dict but it isn't one; you can't dobody["user"]on it. - The right blue box is the same data as a real Python
dict— now you can index it by key. - The top arrow is
json.loads("load from string") — parsing text into values, the direction you use on a response body. - The bottom arrow is
json.dumps("dump to string") — serializing values back into text, the direction you use to send a request body.
In short: loads = text→object (parse an incoming body first); dumps = object→text (before you send). Keep the direction straight and a whole class of API bugs disappears.
json_demo.pyimport json
data = json.loads('{"user":"ava","roles":["dev","admin"],"active":true,"age":30}')
print(data["roles"][0], "| active:", data["active"], "| type:", type(data["active"]).__name__)
print(json.dumps({"ok": True, "items": [1, 2]}, indent=2))
dev | active: True | type: bool
{
"ok": true,
"items": [
1,
2
]
}
JSON is the text format that data travels in between programs — almost every API sends and receives JSON. This tiny program does both directions: it reads a JSON string into Python, then writes Python data back out as JSON.
json.loads(...)means "load from string": it takes the JSON text (the part in quotes) and turns it into normal Python values — a dict here. Now you can look things up like any dictionary.data["roles"][0]reads the first role out of the list, andtype(data["active"]).__name__shows that JSONtruebecame a real Pythonbool— the types are converted for you, not left as text.json.dumps(...)is the reverse — "dump to string": it turns a Python dict back into JSON text.indent=2pretty-prints it with 2-space indentation so it's easy to read.
What the output means: The first line proves the values came through as real Python types (active: True is a bool, not the word "true"). The rest is the same data printed back as neat, indented JSON — note JSON writes it lowercase true.
Try this: Add "score": 9.5 inside the input string and print data["score"] — you'll get a Python float. JSON numbers, strings, lists, and true/false all map to the matching Python types automatically.
4 · Make a real request intermediate
Up to now the web has been theory — a diagram, some tables, a format. This is the moment it becomes real: you're going to make an actual program reach across the internet, touch a server, and bring something back. And here's the reassuring part — after all that build-up about DNS, TCP, TLS, and HTTP, the code to do it is two lines. The requests library folds that entire four-step trip from section 1 behind a single friendly function call. That collapse — from "understand the whole stack" to "call one function" — is the whole reason libraries exist.
The mental model to carry: requests is a very thin, very polite wrapper around the exact request/response conversation you already learned. requests.get(url) means "do a GET" (fetch/read), requests.post(url, json=...) means "do a POST" (send/create). What comes back is a response object with everything the server said, sliced into convenient pieces: r.status_code is that three-digit reply from section 2, r.json() parses the JSON body from section 3 into a Python dict, and there are headers, cookies, and the raw text if you want them. It's not doing anything mysterious — it's the same trip, with the tedious parts handled for you. This is literally what the Anthropic and AWS SDKs do internally; peel back client.messages.create(...) and you find a requests-style POST with an auth header and a JSON body.
The one habit to build from your very first real call: always pass a timeout. It's the single most common production foot-gun in this whole chapter, and it's invisible in testing. Without a timeout, if the server accepts your connection but then never answers — not an error, just silence — your program waits forever. Not thirty seconds; forever. One hung upstream service and your whole app freezes with no error to debug. timeout=10 says "if there's no answer in ten seconds, give up and raise" — turning a silent hang into a catchable exception you can retry or report. New engineers skip it because everything works on their fast local network; the omission only surfaces at 3am under real load. Add the timeout every single time, even in throwaway scripts, until it's muscle memory. The lab below is the one block in this chapter that actually hits the network — run it where you have internet.
The requests library makes HTTP trivial — this is what the LLM/AWS SDKs do under the hood. This one block hits the network.
requests.get(url) collapses the DNS→TCP→TLS→HTTP chain from section 1 into a single line and hands back a response object with the status and parsed body ready to use. This is exactly what an LLM SDK does inside.
Read left to right: your one line of code feeds into the machinery requests hides, and out comes a tidy response object.
- The left blue box is all you actually write —
requests.get(url). - The middle stack is the four-step trip from section 1 (DNS, TCP, TLS, HTTP) that the library performs for you so you don't hand-code any of it.
- The right green box is what you get back: a response object whose pieces map straight onto earlier sections —
r.status_codeis the status from section 2,r.json()parses the JSON body from section 3. - The arrows show it's still a round trip — the same chain you learned, just wrapped. The library isn't magic; it's this diagram with the tedious parts done for you.
In short: requests is a thin wrapper — one line in, a response object out — and the Anthropic/AWS SDKs are the same shape over the same trip.
live_request.pyimport requests
# NOTE: makes real network calls — run where you have internet.
r = requests.get("https://httpbin.org/get", params={"q": "hello"}, timeout=10)
print("status:", r.status_code, "| args:", r.json()["args"])
r = requests.post("https://httpbin.org/post", json={"name": "Ava"}, timeout=10)
print("server saw:", r.json()["json"])
status: 200 | args: {'q': 'hello'}
server saw: {'name': 'Ava'}
This is the one block that actually touches the internet. It uses the popular requests library to make two real HTTP calls to httpbin.org, a free site that simply echoes back whatever you send — perfect for seeing how requests work.
requests.get(url, params={...}, timeout=10)makes a GET request — GET means "read / fetch something". Theparamsbecome the?q=hellopart of the URL (the query string).timeout=10means "give up after 10 seconds" so your program never hangs forever.r.status_codeis the status code —200means success.r.json()parses the response body (which is JSON) into a Python dict, and["args"]pulls out the query values the server saw.requests.post(url, json={...})makes a POST request — POST means "send / create something". Thejson=argument sends your dict as a JSON body; the echo server hands it right back so you can confirm it arrived.
What the output means: status: 200 confirms the call succeeded, and the echoed args and json show the server received exactly what you sent — proof the round trip worked.
Try this: Change "q": "hello" to your own word and re-run — the server will echo it straight back. This GET-plus-POST pattern is the core of every API you'll ever call.
5 · Status-handling logic intermediate
Here's the bug that hides in almost every first-draft API script. Someone calls an endpoint, immediately does r.json()["result"], and it works great — until the day the server returns a 404 or a 500. Now the body isn't the data you expected, it's an error page or empty, and your code explodes with a cryptic KeyError three functions away from the actual problem. The root cause is skipping a step: they used the response before checking whether it succeeded. Status-handling is that missing step, and it's what separates a script that works in the demo from a client that survives contact with reality.
The plain idea: a response is a reply, and not every reply is a yes. Before you trust the body, you look at the status code and decide, per code, what it means for you. Think of checking your mail: a normal letter you open and read (200 — use the data); "no such address, returned to sender" isn't a catastrophe, it just means the thing you asked for isn't there (404 — often a perfectly normal "not found", not a crash); "mailbox full, try later" is a temporary hint to slow down and come back (429 — back off and retry); but a letter that says "the post office is on fire" is a genuine emergency you want to hear about loudly (unexpected 5xx — raise an error). Same envelope, four very different reactions.
The subtle skill — and the thing interviewers probe — is that "error" is not one bucket. The trap is writing if r.status_code != 200: raise, which treats a routine "not found" as a disaster and crashes on codes that are actually fine (like a 201 Created). A good handler branches deliberately: some non-200 codes are expected outcomes you translate into normal values (404 → None), some are signals you act on (429 → retry), and only the genuinely-unexpected ones should raise and interrupt the program. Notice too that this logic is pure — it takes a response and returns a decision, touching no network — which is exactly why the lab can prove it with a fake Resp object offline. That separation (network on one side, decision logic on the other) is a pattern you'll reuse in every testable client you build later in this chapter.
Branch on the status: 200 use it, 404 is a normal "not found", unexpected 4xx/5xx raise. Here's that logic as a pure function — offline-runnable and exactly what you wrap around a response.
Start at handle(r) on the left and follow each arrow to the branch that fires for that status code — one response goes in, one deliberate reaction comes out.
- The green 200 branch is the happy path: the body is real data, so return it.
- The muted 404 branch treats "not found" as a normal outcome, returning
Noneinstead of crashing. - The amber 429 branch is a signal — the server says slow down — so you back off and retry (the logic from section 6).
- The red "else → raise" branch is the catch-all for genuinely unexpected codes (like a surprise 500) — the only case that should interrupt your program.
- The right-hand labels group them: expected outcomes, a signal to act on, and the truly-unexpected — the whole point is that they are not the same.
In short: Branch per code — data, None, retry, raise — instead of lumping everything non-200 into one "error". That distinction is what makes a client survive real traffic.
status.pyclass Resp: # stand-in for requests.Response
def __init__(self, status, data=None): self.status_code, self._d = status, data
def json(self): return self._d
def raise_for_status(self):
if self.status_code >= 400: raise RuntimeError(f"HTTP {self.status_code}")
def handle(r):
if r.status_code == 200: return r.json()
if r.status_code == 404: return None # normal outcome
if r.status_code == 429: return "RATE_LIMITED" # back off + retry
r.raise_for_status() # unexpected -> raise
print("200 ->", handle(Resp(200, {"ok": True})))
print("404 ->", handle(Resp(404)))
print("429 ->", handle(Resp(429)))
try: handle(Resp(500))
except RuntimeError as e: print("500 ->", e)
200 -> {'ok': True}
404 -> None
429 -> RATE_LIMITED
500 -> HTTP 500
A response isn't always a success — you must decide what to do based on the status code. This shows that decision logic as one small function. It uses a fake Resp object instead of a real call so it runs anywhere with no internet.
class Respis a pretend response with just astatus_codeand some data — enough to test the logic.raise_for_status()imitates the real library: it throws an error when the status is400or higher.- Inside
handle(r)the code branches on the status:200→ return the data (success);404→ returnNonebecause "not found" is a normal outcome, not a crash;429→ return a signal meaning "you're rate-limited, back off and retry". - Any other bad status falls through to
r.raise_for_status(), which raises — because an unexpected500is a real problem you want to hear about, not swallow silently.
What the output means: Each line shows the branch that fired: 200 gives the data, 404 gives None, 429 gives RATE_LIMITED, and 500 is caught as a raised HTTP 500 error.
Try this: Add a line print(handle(Resp(201))). 201 ("created") isn't handled, so it falls through to raise_for_status — showing you'd want to treat other 2xx codes as success too.
6 · Advanced — retries with exponential backoff advanced
Try this the next time a webpage fails to load: hit refresh. It usually just works the second time. You've been doing manual retries your whole life without naming it — and that instinct is correct, because a huge fraction of network failures are transient: a packet dropped, a server was busy for 50 milliseconds, a connection blipped. The thing failed for no lasting reason, and simply asking again succeeds. Production clients automate exactly that refresh-and-try-again instinct. The catch is that doing it naively — retrying instantly, forever, on everything — turns a small blip into a self-inflicted outage. This section is about retrying well.
The core technique has a name that sounds fancier than it is: exponential backoff. Instead of retrying at a fixed pace, you wait a little, then double the wait each time — 0.01s, then 0.02s, then 0.04s, and so on. The reason is human, almost polite: if a server is struggling, the worst thing a thousand clients can do is all hammer it again the same instant it's trying to recover — you'd knock it right back down. Backing off exponentially gives the server breathing room that grows as the trouble persists. It's the difference between a crowd calmly filing back into a building versus everyone stampeding the door at once the moment it reopens.
Two refinements turn a decent retry loop into a production one. First, jitter — a small random amount added to each delay. Without it, a fleet of clients that all failed at the same moment will also all retry at the same moment (and again, and again) — a synchronized "thundering herd" that DDoSes your own server in perfect lockstep. Jitter smears those retries across time so they don't resonate. Second, and this is the one people get wrong: only retry what's actually retryable. Timeouts, connection errors, 5xx, and 429 are worth retrying — they're transient or the server explicitly asked you to wait. A 4xx like 400 Bad Request is not: your request itself is malformed, so retrying it identically just fails identically, wasting time and load. And always cap the attempts — a retry loop with no ceiling doesn't add resilience, it adds a hang. Retryable errors only, growing delays, jitter, a hard limit: that's the whole recipe, and it's the resiliency habit first met in py4.
Networks fail transiently. Production clients retry with exponential backoff (and jitter), cap attempts, and only retry retryable errors (timeouts, 5xx, 429 — not a 400). The resiliency habit from py4.
Read left to right along the time axis: each red "fail" is a failed attempt, each amber box is the pause before the next try, and the green box is success.
- The red boxes are transient failures — a dropped packet, a briefly-busy server — the kind worth retrying.
- The amber pause boxes grow wider (0.01s → 0.02s → 0.04s): the wait doubles each time, which is what "exponential backoff" means — it gives a struggling server room to recover.
- The green "ok" box is the retry that finally succeeds, ending the loop.
- The top note names the two refinements: add jitter so a fleet of clients doesn't retry in lockstep, and only retry 429/5xx/timeouts (never a 400) — plus always cap the attempts.
In short: Double the wait each retry, add jitter, retry only retryable errors, and stop after N tries — a small blip is absorbed without stampeding the server.
retry.pydef with_retry(fn, tries=4, base=0.01):
for attempt in range(tries):
try:
return fn()
except (TimeoutError, ConnectionError) as e:
if attempt == tries - 1: raise
delay = base * (2 ** attempt) # 0.01, 0.02, 0.04 (+ jitter in prod)
print(f" attempt {attempt+1} failed ({e}); backing off {delay:.3f}s")
calls = {"n": 0}
def flaky():
calls["n"] += 1
if calls["n"] < 3: raise ConnectionError("transient")
return "ok"
print("result:", with_retry(flaky), "after", calls["n"], "attempts")
attempt 1 failed (transient); backing off 0.010s
attempt 2 failed (transient); backing off 0.020s
result: ok after 3 attempts
Networks fail randomly — a call that failed a second ago often works if you just try again. This retry with backoff wrapper re-runs a function a few times, waiting a little longer between each attempt, instead of giving up on the first hiccup.
with_retry(fn, tries=4, base=0.01)takes the function to run (fn), how many attempts to allow, and a base wait time. Thefor attempt in range(tries)loop is the "try again" mechanism.try: return fn()attempts the call; if it works, we return immediately. Theexcept (TimeoutError, ConnectionError)only catches transient network errors — the kind worth retrying. On the last attempt itraises so failures aren't hidden.delay = base * (2 ** attempt)is exponential backoff: the wait doubles each time — 0.01, then 0.02, then 0.04 seconds — so you don't hammer a struggling server. (Real code adds a little random "jitter" too.)- The
flaky()test function fails the first two times, then succeeds — a controlled way to prove the retry loop actually recovers.
What the output means: You see two "attempt failed … backing off" lines (the doubling delays), then result: ok after 3 attempts — the third try succeeded, exactly as designed.
Try this: Change if calls["n"] < 3 to < 5. Now it fails more times than tries allows, so the retries run out and the error is raised — showing why capping attempts matters.
7 · Advanced — rate limiting & pagination advanced
Two of the most common ways a working script quietly goes wrong in production aren't crashes — they're worse, because they're silent. The first: you loop over an API a bit too eagerly, and suddenly every call comes back 429 and your integration is throttled or banned for an hour. The second, sneakier one: you fetch "all the users", get back a tidy list of 100, ship it — and never realize the account actually has 4,000 users and you only ever saw the first page. No error, no crash, just wrong, incomplete data flowing downstream. Both come from the same blind spot: assuming an API will hand you everything, as fast as you ask. Real APIs don't work that way, and this section is about the two guardrails they use.
Rate limiting is the API saying "you're going too fast, slow down." Think of a theme-park ride that lets a fixed number of people on per minute — get greedy and you're waved off to wait. The server signals this with a 429 Too Many Requests status, and often includes a Retry-After header that literally tells you how many seconds to wait before trying again. The polite, correct move is to listen: when you see a 429, back off (this is exactly where the backoff logic from section 6 plugs in) and, if Retry-After is present, honour it precisely rather than guessing. The mistake is treating 429 as a hard failure and either crashing or — far worse — immediately retrying in a tight loop, which is how you turn a brief throttle into a ban.
Pagination is the flip side: the API refusing to hand you a firehose. Asking for "all 50,000 orders" in one response would be enormous and slow, so servers hand results back in pages — a chunk of items plus a pointer (a next cursor or page number) to where the following chunk lives. It's a book: you read a page, and the page number tells you where to go next; when there's no next page, you're done. The way you consume it is a small loop — fetch a page, collect its items, follow its next pointer, repeat until next is empty. The classic bug is writing that loop with a hard-coded number of pages ("just grab the first 5") or forgetting the loop entirely and reading only page one — both give you a plausible-looking but partial answer that passes every quick test and fails on real accounts. The fix is to always drive the loop off the cursor the server gives you, never off an assumption about how much data exists. Handle both of these and your client is honest: it takes only what it's allowed, and it takes all of what's there.
Two realities of real APIs: they rate-limit you (respect 429 + Retry-After), and they paginate large results (follow next cursors/pages). Handle both or you'll get partial data and bans.
next pointer; the loop fetches, collects, and follows next until it's None. Hard-coding "just the first page" is how you silently ship partial data.
Read left to right along the top row — each box is one page the API returns — then down to the collected total.
- Each blue page box holds a chunk of
itemsplus anextcursor telling you where the following chunk lives. - The horizontal arrows are the loop following each page's
nextpointer to the next page — you never assume how many pages exist. - The red "stop" box fires when a page reports
next: None— that's the signal there are no more pages. - The green bar at the bottom is the payoff: the items from every page stitched into one complete list. Reading only page 1 would have given you
[1, 2]and quietly missed the rest.
In short: Loop fetch→collect→follow-next until next is empty. Drive it off the server's cursor, never a hard-coded page count, and you get all the data.
pagination.py# A fake paginated endpoint: 3 pages of results.
PAGES = {
None: {"items": [1, 2], "next": "p2"},
"p2": {"items": [3, 4], "next": "p3"},
"p3": {"items": [5], "next": None},
}
def fetch_page(cursor): return PAGES[cursor]
def fetch_all():
items, cursor = [], None
while True:
page = fetch_page(cursor)
items += page["items"]
cursor = page["next"]
if cursor is None: break
return items
print("all items across pages:", fetch_all())
all items across pages: [1, 2, 3, 4, 5]
Big result sets don't arrive all at once — APIs send them in pages, and each page tells you where the next one is. This shows how to follow those pages until there are none left, using a fake endpoint so it runs offline.
PAGESis a fake API: each page holds someitemsplus anextpointer (a cursor) to the following page. The last page'snextisNone, meaning "you're done".fetch_all()starts with an empty list andcursor = None(the first page). Thewhile Trueloop fetches a page, adds its items withitems += page["items"], then moves the cursor forward topage["next"].if cursor is None: breakis the stop condition — when a page reports no next cursor, the loop ends and we return everything collected. This is exactly how you'd walk a real API's pages without missing or repeating data.
What the output means: all items across pages: [1, 2, 3, 4, 5] — the loop stitched three separate pages into one complete list.
Try this: Add a fourth page "p3": {"items": [5], "next": "p4"} and a "p4" page ending in None. The same loop follows the extra page automatically — you never hard-code how many pages there are.
8 · Professional — a reusable client class professional
Imagine a codebase where fifteen different files each call the same API directly. One passes a 5-second timeout, another passes none. One remembers the auth header, another forgot it and only fails for admin users. One has retry logic, thirteen don't. Then the API's base URL changes — and now you're grep-ing through the whole repo, praying you catch every spot. This isn't a hypothetical; it's the natural end state of "just call requests.get wherever you need data." The reusable client class exists to kill this entire failure mode by giving every call one door to go through.
The idea is simple and you already use it daily: a client class is like the contacts app on your phone. You don't re-dial the full international number, remember the country code, and re-enter it every time you call your bank — you save it once, then just tap "Bank". A client class saves the settings for talking to one API in exactly that way: the base URL, the auth token, the default timeout, the error handling — configured once in __init__, then every method (get, post, …) just says "the thing at /users/1" and inherits all of it automatically. Change the timeout policy? One line, one place, everyone benefits. This is the Single-Responsibility idea from SD3 made concrete: one class owns "how we talk to this service," and nothing else needs to know the details. It's also exactly how the real Anthropic and AWS SDKs are shaped — an object you configure once and call many times.
The one technique that makes this class professional rather than merely tidy is dependency injection — a heavy name for a light idea: pass the network session in from outside instead of creating it inside the class. Why does that matter so much? Because the network is the one part you can't control or predict in a test — it's slow, it's flaky, it needs real credentials. If your client builds its own session internally, you can only test it by actually hitting the internet, which makes tests slow, brittle, and unrunnable in CI. But if the session is injected, your test can hand the client a fake session that returns canned responses instantly and offline — and now you can prove the client's real logic (URL building, auth headers, error handling, retries, pagination) with zero network. The seam where you swap real-for-fake is called the "network boundary," and putting a swappable joint there is what makes the whole thing testable. The beginner instinct — "just import requests and call it inside the method" — feels simpler and quietly makes the class impossible to test properly. Inject the session, and the fake in the lab below drops in exactly where requests.Session will live in production.
Teams don't scatter requests.get everywhere — they wrap an API in a client class with base URL, auth, timeouts, and error handling in one place (SD3 SOLID), with an injectable session so it's testable without a network.
requests its own way (some miss the timeout, the auth, the retry) — change the URL and you grep the whole repo. Right: one ApiClient owns base URL, auth, timeout and retries, and its injected session makes it testable offline.
Compare the two halves split by the dashed line: the left is the mess, the right is the fix.
- On the left, four red file boxes each call the API directly, and each carries a different setting — one has a timeout, one doesn't, one forgot auth, one has no retry. That inconsistency is the bug factory.
- On the right, the same four files all route through one green
ApiClient— the single "door" that owns base URL, auth, timeout and retries, so every call inherits them automatically. - The client makes just one outbound call to the API, and its settings are defined once — change the timeout policy in one line and every file benefits.
- The "inject session → test" box is the professional touch: because the network session is passed in from outside, a test can swap in a fake and exercise the client's logic with zero network.
In short: Route every call through one configured-once client instead of scattering requests.get; inject the session so the whole thing is testable offline. This is the shape of the real Anthropic/AWS SDKs.
api_client.pyclass ApiClient:
def __init__(self, base_url, token=None, timeout=10, session=None):
self.base_url = base_url.rstrip("/"); self.timeout = timeout
self.session = session # inject -> testable
if token and session: session.headers["Authorization"] = f"Bearer {token}"
def get(self, path, **params):
r = self.session.get(f"{self.base_url}/{path.lstrip('/')}",
params=params, timeout=self.timeout)
r.raise_for_status(); return r.json()
class FakeResp:
status_code = 200
def json(self): return {"user": "ava"}
def raise_for_status(self): pass
class FakeSession:
headers = {}
def get(self, url, params=None, timeout=None):
print("would GET:", url); return FakeResp()
client = ApiClient("https://api.example.com", token="secret", session=FakeSession())
print("result:", client.get("users/1"))
would GET: https://api.example.com/users/1
result: {'user': 'ava'}
Instead of scattering requests.get calls all over a codebase, teams wrap an API in one client class that keeps the base URL, auth token, and timeout in a single place. This is the same shape as the official LLM and AWS SDKs.
__init__stores the settings once:base_url(with any trailing slash stripped), atimeout, and asession. If atokenis given, it's added as anAuthorization: Bearer …header so every request is authenticated automatically.- The
sessionis injected (passed in) rather than created inside — that's the key trick. In production you pass a real network session; in a test you pass a fake one, so the class is testable without any internet. get(self, path, **params)builds the full URL, calls the session'sget, thenr.raise_for_status()turns any error status into an exception before returningr.json()— so callers always get clean data or a clear failure.FakeSessionandFakeRespare stand-ins:FakeSession.getjust prints the URL and returns a canned{"user": "ava"}. That's how you prove the client's logic works with zero network calls.
What the output means: would GET: https://api.example.com/users/1 shows the URL the client built, and result: {'user': 'ava'} is the parsed data — all from the fake, no internet needed.
Try this: Add a post method that mirrors get but calls self.session.post. One class now owns every call's URL, auth, and error handling — the whole point of a reusable client.
9 · Tech-lead — standardize reliability tech-lead
Here's how a real outage starts, and it's almost never dramatic. Ten teams each built their own way to call services. Nine set a timeout, one didn't. A downstream service gets slow one afternoon — not down, just slow. The nine time out and shed load gracefully; the one hangs, its threads pile up waiting forever, it runs out of workers, and it takes down everything upstream of it too. A local slowdown becomes a company-wide cascade, and the post-mortem traces it to a single missing timeout= in one service written eight months ago. The lesson tech leads internalize the hard way: reliability that depends on every engineer remembering to do the right thing is not reliability — it's luck.
The previous section made one client robust. This section is about the leap from "robust in one place" to "robust by default, everywhere," and the insight is almost sociological: a standard that lives in a wiki page gets ignored; a standard baked into a shared library gets enforced automatically. Think of it like the electrical code for a building. You don't trust each electrician to remember to ground every outlet — you sell only pre-wired, code-compliant outlet boxes, so doing it right is the path of least resistance and doing it wrong takes extra effort. A tech lead ships one blessed client (or a make_session factory) with retries, timeouts, backoff, logging, and rate-limit handling already inside. Teams import it instead of hand-rolling their own, and every one of their calls inherits the whole reliability policy for free — not because they read a doc, but because the safe thing is the only thing on offer.
And notice what you actually understand now: this whole chapter has been quietly building the LLM SDK from first principles. That single client.messages.create(...) line you'll write a thousand times is this — an HTTP POST (section 2) with an auth header and a JSON body (section 3), sent over the DNS→TCP→TLS→HTTP chain (section 1), its status checked (section 5), retried with backoff on the right failures (section 6), respecting rate limits and paging (section 7), wrapped in a configured-once client (section 8), standardized across the org (this section). The SDK isn't magic; it's these nine pieces, hardened and hidden. The tech-lead move is to make sure your own integrations get the same treatment — define the reliability once, and let the whole codebase inherit it. The one subtlety a lead must get right, called out below: scope automatic retries to idempotent methods only, so a retried POST can't silently double-write (the very first foot-gun from section 2, now at organizational scale).
This IS the LLM SDK unwrapped: every client.messages.create(...) is an HTTP POST with an auth header + JSON body, status-checked, retried. A lead defines one resilient client (retries, timeouts, logging, rate-limit handling) so every integration inherits the same reliability.
create(...) line sits on the DNS→HTTP trip (§1), a JSON POST (§2–3), status checks (§5), backoff (§6), rate-limit/paging (§7), a configured client (§8), standardized org-wide (§9). No magic — just the whole chapter stacked.
Read it top-down as an unwrapping: the single line of code at the top rests on the layers beneath it, each one a section of this chapter.
- The top blue box is the one line you'll write a thousand times —
client.messages.create(...). - The bottom foundation band (§1) is the DNS→TCP→TLS→HTTP trip — nothing happens without it.
- The middle bands stack the request essentials on top: a JSON body over the right method and status (§2–3), then status handling (§5), retries with backoff (§6), and rate-limit/pagination (§7).
- The two green bands near the top (§8, §9) are the leadership layers: wrapping it all in a configured-once client, then standardizing that client across the whole org so reliability is enforced by shared code, not by memory.
- Read the widening/narrowing bands as "each layer builds on the one below" — the SDK line at the very top is only possible because every layer under it is handled.
In short: The LLM SDK isn't magic — it's this chapter's nine sections stacked and hidden behind one call. The tech-lead move is to give your own integrations the same full stack, defined once.
ApiClient and the team uses it everywhere, every external call gets retries, timeouts, and consistent error handling for free. That's how you make an entire codebase robust with one class.Exercise DF4.1 — Build a resilient client
Context: The capstone of the chapter is a single client that behaves well against a real, flaky network: it retries the right failures, times out, paginates, and — crucially — is fully testable without a network.
Your task: Wrap a public API in an ApiClient with retries + backoff, a timeout, 404→None and 429→retry handling, pagination, and an injectable session so you can unit-test it with a fake.
Requirements:
- Add retries with backoff plus a request timeout
- Map 404 to
Noneand treat 429 as a retryable case - Page through a multi-page response until it's exhausted
- Inject the session/transport so the network seam is swappable for tests (TQ3)
- Prove the retry logic with a fake that fails twice then succeeds, and page through a multi-page fake
💡 Hint: Design against a fake session first: if your client can be fully exercised offline — flaky retries and pagination included — the real requests.Session is just a drop-in.
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Every HTTP response carries a status code, and its first digit tells you the whole story: success, redirect, your mistake, or the server's. Knowing which codes are worth retrying is the foundation of every resilient client.
Your task: Without any network, write a function that maps an HTTP status code to its class (informational / success / redirect / client-error / server-error) and whether a client should retry. Test it on 200, 301, 404, and 503.
Requirements:
- Derive the class from the first digit (e.g.
code // 100) - Return a retryable flag that is true only for transient codes (429, 500, 502, 503, 504)
- Show 404 classed as client-error and not retryable
- Show 503 classed as server-error and retryable
- Print the classification for each of 200, 301, 404, 503
💡 Hint: The mental model: 4xx means you got the request wrong (don't retry, fix it); 5xx means the server failed (retrying may help).
Show solution
Pure Python, no network needed:
def classify(code):
klass = {1: "informational", 2: "success", 3: "redirect",
4: "client-error", 5: "server-error"}[code // 100]
retryable = code in (429, 500, 502, 503, 504)
return klass, retryable
for c in (200, 301, 404, 503):
print(c, classify(c))
# 200 ('success', False)
# 301 ('redirect', False)
# 404 ('client-error', False) -> your bug, do not retry
# 503 ('server-error', True) -> transient, retry with backoff
4xx means you got it wrong (fix the request); 5xx means the server failed (retrying may help).
Context: Data travels between services as JSON text, and JSON only knows a handful of types. Understanding what survives a round-trip — and what doesn't — prevents a whole class of serialization bugs.
Your task: Build a small dict, serialize it to a JSON string, parse it back, and confirm the round-trip preserves types — then show one gotcha: JSON has no tuple or datetime.
Requirements:
- Serialize with
json.dumpsand parse back withjson.loads - Confirm
str/list/bool/floatsurvive the round-trip - Show that a
datetime/dateraisesTypeErroruntil converted - Note that tuples come back as lists
- State the fix: convert dates/sets/bytes to JSON-native types before sending
💡 Hint: Try serializing a value JSON can't represent and catch the TypeError — the error message itself points you at what needs converting first (e.g. isoformat()).
Show solution
import json
from datetime import date
payload = {"user": "ada", "roles": ["admin", "dev"], "active": True, "score": 3.5}
text = json.dumps(payload) # dict -> JSON string (what goes on the wire)
back = json.loads(text) # JSON string -> dict
print(back == payload) # True: str/list/bool/float survive
# gotcha: no native tuple or datetime -- you must convert first
try:
json.dumps({"when": date.today()})
except TypeError as e:
print("needs conversion:", e) # serialize as when.isoformat()
Tuples come back as lists; dates/sets/bytes must be converted to JSON-native types before sending.
Context: Talking to a live API is where theory meets the network's messiness: the call can come back with a bad status, or never come back at all. A real client must handle both without hanging or crashing.
Your task: Call a public API with requests, check the status, and parse the JSON — handling both failure modes: a non-2xx response and a network/timeout error.
Requirements:
- Issue the request with
requests.getand a timeout set - Turn 4xx/5xx into an exception with
r.raise_for_status() - Catch
requests.HTTPErrorfor bad statuses and report the code - Catch
requests.RequestExceptionfor DNS/timeout/connection errors - Return the parsed JSON on success,
Noneon handled failure
💡 Hint: Always pass a timeout — without one a hung server hangs your program forever, which is the single most common production footgun here.
Show solution
Needs network + requests (pip install requests). Uses the free httpbin echo service:
import requests
def get_json(url, timeout=5):
try:
r = requests.get(url, timeout=timeout)
r.raise_for_status() # turns 4xx/5xx into an exception
return r.json()
except requests.HTTPError as e:
print("bad status:", e.response.status_code)
except requests.RequestException as e:
print("network error:", e) # DNS, timeout, connection refused
return None
data = get_json("https://httpbin.org/json")
print(data and list(data.keys()))
Always set a timeout — without it a hung server hangs your program forever.
Context: Transient failures are normal at scale, so a good client retries — but only on the right failures, with growing delays and jitter so a fleet of clients doesn't retry in lockstep. Modelling it offline keeps the logic testable.
Your task: Wrap a flaky call so it retries only on retryable failures (429/5xx and network errors), backing off 1s, 2s, 4s… with jitter, and gives up after N attempts — modelled offline with a fake that fails twice then succeeds.
Requirements:
- Retry up to a configurable number of attempts, then raise after the last one
- Grow the delay exponentially (
base * 2**i) and add random jitter - Only retry retryable failures — never a 4xx, whose request is itself wrong
- Make it runnable offline by injecting/stubbing the
sleepso it prints instead of waiting - Demonstrate with a fake that fails twice then returns success
💡 Hint: Jitter (the small random add on each delay) is what prevents a thundering herd of clients all retrying at the same instant.
Show solution
Runnable offline — the backoff sleep is stubbed so it prints instead of waiting:
import random
def with_retries(call, attempts=4, base=1.0, sleep=print):
for i in range(attempts):
ok, result = call()
if ok:
return result
if i == attempts - 1:
raise RuntimeError(f"gave up after {attempts} attempts")
delay = base * (2 ** i) + random.uniform(0, 0.3) # jitter avoids thundering herd
sleep(f"retry {i+1} in {delay:.2f}s")
# fake service: fails twice (transient), then returns 200
_calls = {"n": 0}
def flaky():
_calls["n"] += 1
if _calls["n"] < 3:
return False, 503
return True, {"data": "ok"}
print(with_retries(flaky)) # prints two backoff lines, then {'data': 'ok'}
Jitter (the random add) prevents many clients retrying in lockstep. Never retry a 4xx — the request itself is wrong.
Context: Once several call sites hit the same API, you want base URL, auth, timeout, and retries configured in one place. Making the network seam swappable is what lets you unit-test the client without touching the network.
Your task: Wrap a base URL, default headers, timeout, and retries into a small ApiClient class so callers just say client.get('/users/1') — keeping the transport swappable so it's testable offline.
Requirements:
- Store base URL, default headers (Accept, optional Bearer auth), and timeout on the instance
- Expose a simple
get(path)that joins path to base and returns parsed JSON - Raise on error statuses (≥ 400) instead of returning a bad body
- Inject the transport (a callable) so a fake can stand in for the network
- Demonstrate the client working against an offline test double
💡 Hint: Centralizing base URL, auth, and timeout means every call is consistent and you change reliability policy in exactly one place — pass a real requests.Session in prod.
Show solution
The transport is injected, so the class is unit-testable offline; in production pass a real requests.Session.
class ApiClient:
def __init__(self, base_url, token=None, timeout=5, transport=None):
self.base = base_url.rstrip("/")
self.headers = {"Accept": "application/json"}
if token:
self.headers["Authorization"] = f"Bearer {token}"
self.timeout = timeout
self.transport = transport # callable(method,url,headers,timeout)
def get(self, path):
url = f"{self.base}{path}"
status, body = self.transport("GET", url, self.headers, self.timeout)
if status >= 400:
raise RuntimeError(f"{status} for {url}")
return body
# offline test double
def fake(method, url, headers, timeout):
assert headers["Authorization"] == "Bearer t0k"
return 200, {"id": 1, "name": "ada"}
c = ApiClient("https://api.example.com", token="t0k", transport=fake)
print(c.get("/users/1")) # {'id': 1, 'name': 'ada'}
Centralizing base URL, auth and timeout means every call is consistent and you change reliability policy in one place.
Context: Ten services each hand-rolling HTTP calls with inconsistent timeouts and no retries is how cascading outages start. As tech lead you ship one blessed way to make a call so reliability is enforced by code, not by a wiki page.
Your task: Ten services hand-roll their own HTTP with inconsistent timeouts and no retries. Define the org standard and show how a shared, configured session enforces it everywhere.
Requirements:
- Ship a single factory (e.g.
make_session) teams import instead of building bare sessions - Mount a
Retry/HTTPAdapterthat retries on 429/5xx with backoff - Enforce a default
timeouton every outbound call - Retry only idempotent methods; do not auto-retry POSTs (double-write risk)
- Make the point that a shared library — not documentation — is what actually enforces the standard
💡 Hint: Wrap the session's request to inject a default timeout, and scope retries to idempotent methods so a retried non-idempotent write can't silently double-apply.
Show solution
Ship one blessed factory that bakes in the org defaults; teams import it instead of constructing bare sessions.
import requests
from requests.adapters import HTTPAdapter
try:
from urllib3.util.retry import Retry
except ImportError: # very old urllib3
from requests.packages.urllib3.util.retry import Retry
def make_session(total=3, backoff=0.5, timeout=5):
s = requests.Session()
retry = Retry(total=total, backoff_factor=backoff,
status_forcelist=(429, 500, 502, 503, 504),
allowed_methods=frozenset(["GET", "PUT", "DELETE"]))
adapter = HTTPAdapter(max_retries=retry)
s.mount("https://", adapter)
s.mount("http://", adapter)
s.request = _with_default_timeout(s.request, timeout) # enforce timeout
return s
def _with_default_timeout(fn, timeout):
def wrapper(method, url, **kw):
kw.setdefault("timeout", timeout)
return fn(method, url, **kw)
return wrapper
The standard: every outbound call has a timeout; idempotent methods retry with backoff on 429/5xx; non-idempotent POSTs do not auto-retry (risk of double-writes). Enforcing it through a shared library — not a wiki page — is what actually stops the next cascade.
✓ Checkpoint — you can move on when you can…
- Trace DNS→TCP→TLS→HTTP; use methods/status codes/JSON.
- Handle statuses, retries+backoff, rate limits, and pagination.
- Build a testable client with a fake session.
- Standardize a reliable client for the whole team.
Knowledge check check yourself
What four steps happen between typing a URL and getting a response, in order?
Show answer
In a resilient client, why use exponential backoff on retries, and which failures should you actually retry?