AI Engineering Course — Build LLM, RAG & Agentic Systems
A complete, free AI engineering curriculum you learn by doing — from Python fundamentals to RAG, AI agents, evaluation, and production deployment with Claude and AWS. Every one of 230+ lessons has objectives, a lab you type out yourself, runnable code, graded exercises with worked solutions, common-mistake warnings, and a checkpoint before you move on. See the full curriculum →
- 🤖 OpenAI / Codex section — the OpenAI counterpart to our Claude section — Codex & GPT models
- 🦊 GitLab automation with Claude & Codex — CI/CD pipelines that plug in either model
claude-opus-4-8, but the concepts transfer to any provider.
Start Here · The Complete Python Curriculum 🐍
🐍 The Complete Python Curriculum →
Everything Python under a single link — fundamentals, data structures & algorithms, and advanced AI engineering, basics to expert.
● ReadyPython basics → expert
Types, data & structures, functions/OOP, advanced, expert, real-world engineering. Start at P1.
● ReadyDSA from scratch
Complexity, stacks/queues/lists, hashing, trees/heaps, graphs, sorting & searching — in Python.
● ReadyThe production layer
Metaprogramming, memory/GIL, async/streaming, numerics/tensors, tokenization, validation, vector DBs, MLOps.
● Ready📐 System Design · SQL, LLD & HLD — from queries to designing Uber
SQL & queries
The relational model, SELECT/JOIN/GROUP BY, subqueries, and window functions — all runnable via sqlite3, no server needed.
Schema design & normalization
1NF→3NF, keys & relationships, indexes, and transactions/ACID — designing a schema that stays correct and fast.
● ReadyOOD, UML & SOLID
Object-oriented design, UML you'll actually use, and the five SOLID principles — each shown as before/after Python.
● ReadyDesign patterns
Creational, structural & behavioral patterns in real Python, plus a machine-coding round (design a rate limiter).
● ReadyHLD: architecture & caching
Client-server, load balancers, API gateways, and caching — with a working LRU cache and load-balancer strategies in Python.
● ReadyScalability & CAP
Vertical vs horizontal scaling, statelessness, replication, and the CAP theorem — the principles behind every big system.
● ReadySharding, hashing & queues
Sharding, a runnable consistent-hashing ring, message queues (Kafka/RabbitMQ), and SQL-vs-NoSQL trade-offs.
● ReadyDesign a system (capstone)
Put it together: design a URL shortener / Uber / WhatsApp-style system end-to-end, with Python components and architecture diagrams.
● Ready🧑💻 Developer Foundations · Git, the command line & how the web works
The command line & Linux basics
Navigate, manipulate files, pipe commands, permissions, and write a first shell script — the terminal you'll live in.
● ReadyGit: track your work
Why version control, the repo/stage/commit model, history, undoing mistakes, and .gitignore — hands-on from init to log.
● ReadyBranching, PRs & collaboration
Branches, merges & conflicts, remotes, the GitHub pull-request workflow, and how teams actually ship together.
● ReadyHTTP, DNS, REST & JSON
What happens when you hit a URL: DNS, TCP, HTTP methods/status codes, REST APIs, and JSON — with runnable Python requests.
● ReadyEnvironments, packaging & dependencies
Virtual environments, pip/requirements, semantic versioning, lockfiles, and publishing a package — reproducible setups.
● ReadySet up a professional repo
Create a real GitHub repo: README, .gitignore, venv, branches, a PR, and a CI check — the setup every project starts from.
● Ready✅ Software Testing & Quality · pytest, TDD & confidence to change code
Why test, and your first pytest
What tests buy you, the test pyramid, and writing + running your first pytest test from zero.
● ReadyUnit tests & assertions
Testing functions and classes, assertions, testing exceptions, edge cases, and structuring a test suite.
● ReadyFixtures, parametrize & mocking
Reusable fixtures, parametrized tests, and mocking external calls (APIs, time, the filesystem) so tests are fast and deterministic.
● ReadyTDD: red → green → refactor
Write the test first: the TDD loop, when it helps, and building a small feature test-first end to end.
● ReadyIntegration tests, coverage & CI
Testing components together, measuring coverage honestly, and running the suite automatically in CI on every push.
● ReadyAdd a test suite to real code
Take an untested module, drive its coverage up with unit + integration tests, add mocking + CI, and gate merges on green.
● Ready🐳 Containers & Deployment · Docker, Compose & shipping to the cloud
Why containers, and your first Dockerfile
The "works on my machine" problem, what a container is vs a VM, and writing + building + running your first Dockerfile.
● ReadyImages, layers & best practices
How layers & the build cache work, small images, multi-stage builds, .dockerignore, and not baking in secrets.
● Readydocker-compose: app + db + cache
Run a multi-container app (API + Postgres + Redis) with one file: services, networks, volumes, and env config.
● ReadyDeploy a container to the cloud
Registries, pushing an image, running it on a managed service, health checks, env/secrets, and rollbacks.
● ReadyCI/CD: build, test, deploy automatically
A GitHub Actions pipeline that tests (TQ), builds the image, and deploys on merge — the automation that ties it together.
● ReadyContainerize & ship an app
Take a FastAPI + DB app, containerize it with Compose, add a CI/CD pipeline, and deploy it live with health checks and rollback.
● ReadyPart 0 · Concepts (start here)
Agentic AI foundations
The mental model with no code: chatbot vs agent, the 7 building blocks, the agent loop, LLMs-vs-agents — each tied to your video-editing project.
● ReadyLearning roadmap & mistakes
The 5-step path mapped onto this course, the beginner mistakes to avoid, and a phased plan for your video-editing agent.
● ReadyPart I · Foundations
Environment & your first API call
Install the SDK, manage keys safely, make a streaming call, and dissect exactly what a request and response contain.
● ReadyPrompting & structured output
Design system prompts, use few-shot examples, control cost with effort, and force schema-valid JSON you can parse safely.
● ReadyPart II · Retrieval
Part III · Agents
Part IV · Production
Evaluation & testing
Build a golden set, write deterministic + LLM-judge evals, and wire a regression gate so quality can't silently drop.
● ReadyProduction hardening
Prompt caching, observability, guardrails, cost control and deployment — assembled into one production-ready service.
● ReadyPart V · Deploy it for real → the Capstone
🧭 The Forward Deployed Engineer method
How to deploy an AI system into a real company: discovery, thin vertical slice, earning trust, the autonomy ladder, handoff & impact.
● Ready🏗️ The AI DevOps Engineer
The full end-to-end case study: architecture, safety model, the complete tool surface, RAG onboarding, infra evals, phased rollout, subscription model — the why behind the build.
● Readyllm-course-starter/devops-agent/.
Setup, schema & mock cluster
Project skeleton, the risk schema, a mock cluster from fixtures, first tests — all offline, no API key.
● ReadyTool registry & agent loop
Tools as data (with risk classes) + the agentic loop → a working read-only diagnostician.
● ReadyRAG runbooks & safety gate
Runbook grounding, autonomy rungs, the policy gate, audit log — and the test that proves it's safe.
● ReadyEvals & going real
Accuracy + hard-fail safety eval, CI gate, then swap the mock for a real kind cluster + sandbox AWS.
● ReadyProject Gallery · 6 in-demand agents to build
★ All 6 projects
The full gallery with a "which should I build first?" guide and the shared blueprint every project follows.
● Ready🎧 Customer Support Agent
Cited answers from your docs, ticketing, and smart escalation. The most-deployed agent type.
● Ready👨💻 Coding & Code-Review Agent
Repo-aware PR review with verified findings and suggested fixes. Fastest-growing use case.
● Ready📄 Document Intelligence Agent
Contracts/invoices → validated structured data, with a review queue. Highest-ROI, most achievable.
● Ready📊 Data Analyst Agent
Plain-English → read-only SQL → charts & explanation. "Talk to your database."
● Ready🔬 Deep Research Agent
Plan → search → verify → cited report. The pattern behind "deep research" everywhere.
● Ready🎓 Specialized Topics · the gaps most courses skip
🔒 LLM Security & Safety
Prompt injection (direct & indirect), jailbreaks, OWASP LLM Top 10, insecure output handling, PII/secrets, guardrails in code & IAM.
● Ready🖼️ Multimodal AI
How vision models see, sending images to Claude, document intelligence (structured extraction), multimodal RAG, audio pipelines.
● Ready🔌 Model Context Protocol
The open standard for tools/data — host/client/server, tools vs resources vs prompts, building a server, when to use it.
● Ready🎯 Fine-tune vs RAG vs Prompt
The decision framework, what each changes, LoRA/adapters, data prep, and why fine-tuning is usually the last resort.
● Ready🗄️ SQL & Databases
Queries, joins, indexing (B-trees), transactions/ACID, connection pooling, SQL injection & safe text-to-SQL for LLMs.
● Ready🅰️ Claude & Anthropic · working directly with the models
Meet Claude & the model family
The current models, how to match a tier to a task, adaptive thinking & the effort dial, and the three surfaces you'll use.
The Anthropic API
Messages, system prompts, streaming, and the manual tool-use loop — the mechanics under every advanced feature and every agent.
● ReadyClaude Code & agentic dev
The agent loop productionized: permissions, CLAUDE.md, slash commands, subagents — and what to steal for your own agents.
Claude with MCP
Connect MCP servers to Claude Code and via the API's MCP connector — plus when to use MCP over inline tools, and the security cautions.
● Ready🧰 Anthropic API in Practice · the day-to-day API features, as runnable recipes
Message Batches API
Submit thousands of requests asynchronously at ~50% cost: batches.create with a custom_id per request, poll processing_status, retrieve & reconcile results. Batch vs realtime, per-item errors, chunking.
Prompt caching, hands-on
The biggest cost/latency lever, in code: where to put the cache_control breakpoint, reading cache_creation/cache_read usage, the ~5-min TTL, invalidation, and the write-1.25× / read-0.1× math.
Files API, vision & PDF
Send images and PDFs to Claude: inline base64 vs the Files API (upload once, reuse by file_id), document Q&A with citations, size/format limits, and cost of image/doc tokens.
Token counting & usage
Pre-flight with count_tokens, read the usage object precisely, compute $ per call, track cost, handle rate limits, and page list endpoints with the SDK's auto-pager.
Server-side tools
Built-in tools Anthropic runs for you — web search and code execution — no manual tool loop. How to enable them, read results & citations, their pricing, and when to prefer them over client-side tools.
● ReadyRun Claude anywhere
The same code on three providers: the direct Anthropic API, AnthropicBedrock, and AnthropicVertex. Auth per provider, model-ID mapping, and when compliance or an existing cloud commit decides the door.
The Claude Agent SDK
Anthropic's official SDK for building agents in code — the engine behind Claude Code. The bridge between the raw message loop and the CLI: it runs the loop, you supply tools, permissions and config.
● ReadyRequest parameters, in full
A precise reference to every messages.create parameter — system, temperature/top_p, stop_sequences, tool_choice, thinking and more: what each does, sane defaults, and the gotchas.
Prompting Claude & migration
Anthropic-specific technique — XML tags, prefilling, long-context ordering, the Console prompt improver — plus the model-migration workflow: pin, re-eval old vs new, adjust prompts, canary, roll out.
● ReadyBatch document pipeline (capstone)
Compose the track: cache a shared instruction, submit a folder of documents as one Batch for ~50% savings, extract structured data, reconcile by custom_id, and produce a token-usage cost report.
🧩 No-Code Agentic AI · ship agents without writing the loop
Agentic workflows with n8n
Open-source, self-hostable workflow canvas — triggers, nodes, and the AI Agent node with tools + memory, mapped to the C2 loop.
● ReadyWorkflow automation with Zapier
The widest integration layer: Zaps, AI steps, AI Agents & the Zapier MCP endpoint — and exactly when to graduate off it.
● ReadyBuilding with Make
Visual scenarios with real control flow — Router, Iterator, Aggregator — plus operations-based pricing and how to design around it.
● ReadyNo-code agents with Flowise
LLM-native builder: assemble a RAG chatflow and a tool-using agent on a canvas, then deploy it as an API — Chapters 3 & 4, visually.
● Ready⛓️ LangChain & LangGraph · agents in code, done properly
Agentic AI foundations & architectures
Workflow vs agent; ReAct, plan-and-execute, reflection; supervisor/network/hierarchical multi-agent; why frameworks exist.
● ReadyLangChain Core — chains, memory, RAG
Runnables & the LCEL pipe, structured output, conversation memory, and Chapter 3's RAG rebuilt as a composable chain.
● ReadyLangChain agents & tool use
The @tool decorator, binding tools, create_react_agent, and why the recommended agent now lives in LangGraph.
LangGraph — state & routing
State, nodes, edges, reducers, and conditional routing — then the ReAct loop rebuilt from scratch as an explicit graph.
● ReadyLangGraph — cycles, HITL & persistence
Bounded cycles, durable checkpointers, and interrupts for human approval — the real safety gate, and the capstone's core.
● Ready👥 Multi-Agent Orchestration · teams, retrieval & depth
Multi-agent orchestration with CrewAI
Agents, Tasks, Crews, Processes — sequential and hierarchical (manager) crews, tools, Crews-vs-Flows, and the "do you need a crew?" gate.
● ReadyMulti-agent systems with Microsoft AutoGen
Conversation-based agents, termination conditions, round-robin vs selector teams, tools + human-in-the-loop, vs CrewAI/LangGraph.
● ReadyAgentic RAG & GraphRAG for agents
Retrieval as a decision: retrieve→grade→rewrite loops, GraphRAG for relational questions, and routing across vector/graph/web sources.
● ReadyDeep Agents — reflection, planning & memory
The four pillars — planning, reflection, long-term memory, sub-agents — composed into a long-horizon agent. The synthesis of the whole arc.
● Ready📦 MLOps & LLMOps · keeping a live system trustworthy
Foundations of MLOps & LLMOps
What MLOps solves, how LLMOps differs (prompts, per-token cost, provider risk), the lifecycle loop, and the versioned-artifact discipline.
● ReadyLLM infrastructure, tooling & the OSS stack
The six-layer stack, hosted-vs-self-hosted at each layer, the LLM gateway/proxy, and when self-hosting an open-weight model is worth it.
● ReadyDeployment, containerization & scaling
Why LLM apps scale differently, containerizing safely, the scaling-lever order, eval-gated rollouts, and graceful degradation.
● ReadyMonitoring, governance & responsible AI
The four signals, agent tracing, drift detection, audit/access/data governance, and the AI incident runbook with a kill switch.
● Ready☁️ AWS AI Automation · Bedrock, agents, ML & AI services, done in code
AWS AI foundations
The three layers of AWS AI, boto3 auth via the credential chain & IAM roles, regions & model access, and the cost model. Where Claude fits.
● ReadyAmazon Bedrock & Claude
The Converse API, calling Claude on Bedrock, streaming, token usage, and the raw invoke_model escape hatch.
Tool use & structured output
Give Claude tools via toolConfig, run the tool loop, and force schema-valid JSON with toolChoice — the seed of an Agent.
Knowledge Bases (managed RAG)
Bedrock Knowledge Bases, OpenSearch Serverless vector store, ingestion, and Retrieve / RetrieveAndGenerate.
Bedrock Agents
Managed agentic loops: action groups, Lambda executors, orchestration, and session state — the manual W3 loop, run for you.
● ReadyBedrock Guardrails
Content filters, PII detection & redaction, denied topics, and contextual grounding checks — safety as a configurable layer.
● ReadyIaC: Terraform & CDK
Provision the KB + Agent + Guardrail + OpenSearch stack with Terraform, then the same with CDK. IAM roles wired correctly.
● ReadySageMaker deploy
JumpStart, real-time endpoints, deploying an open model, batch transform, and invoking from boto3 — plus the delete-the-endpoint discipline.
● ReadySageMaker pipelines
SageMaker Pipelines, training jobs, the Model Registry, and MLOps automation for classic + generative ML.
● ReadyTextract & Comprehend
OCR, forms & tables with Textract; entities, PII & sentiment with Comprehend. A document-AI pipeline, end to end.
● ReadyVision & speech
Rekognition (images/video), Transcribe (speech-to-text), Translate, and Polly (text-to-speech) — media automation.
● ReadyKendra & Amazon Q
Kendra enterprise search and Amazon Q (Developer & Business) — managed search and assistants over your own data.
● ReadyServerless orchestration
Wire AI services into event-driven pipelines with Lambda, Step Functions, and EventBridge. The automation glue.
● ReadyObservability & cost
CloudWatch, model-invocation logging, cost controls, governance, and monitoring for a live AWS AI system.
● ReadyDocument intelligence pipeline
S3 → Textract → Claude-on-Bedrock extraction → DynamoDB, Terraform-provisioned, with Guardrails. Serverless, end to end.
● ReadyBedrock support agent
A Bedrock Agent + Knowledge Base support assistant with action groups, Guardrails, IaC, and evals — production-shaped.
● ReadyProduction RAG platform
KB + OpenSearch + Lambda API provisioned with CDK, plus CI, monitoring, and a cost dashboard. The full platform build.
● Ready✨ AI-Assisted Development · coding with AI, responsibly
Vibe coding fundamentals
Vibe coding vs AI-assisted engineering, the delegation spectrum, the intent→generate→verify→refine loop, and the real risks.
● ReadyAI-powered development with Cursor
The four surfaces (Tab, inline, chat, agent), @-context, a rules file, and running agent mode with a review gate.
AI-native dev with Google Antigravity
The agent-first paradigm, mission-control supervision, verifiable artifacts, and the discipline of managing agents.
● ReadyAccelerating with Amazon Q Developer
AWS-native context, code transformation/upgrades as an agent sweet spot, IAM/infra help, and verifying AWS actions.
● ReadyAI pair programming with GitHub Copilot
Completions→chat→agent→PR review, shaping suggestions, custom instructions, model choice, and secrets/licensing governance.
● Ready🔗 Interoperability & Agent Ops · connecting & operating agents
MCP ecosystem integrations
Reference/vendor/community servers, transports (stdio vs HTTP), connecting to hosts & agents, and vetting third-party servers safely.
● ReadyAgent interoperability — A2A
Agent-to-agent delegation across boundaries: Agent Cards, tasks, artifacts; A2A vs MCP; and the trust risks of remote peers.
● ReadyAgent interoperability — ACP & ANP
The wider protocol landscape mapped by what-connects × scope, ANP's identity-first open network, and a stance on emerging standards.
● ReadyEvaluation & tracing with LangSmith
Traces→datasets→evaluators→version comparison→CI gate, the trace-to-golden-set flywheel, and online monitoring.
● ReadyAI guardrails — NeMo & Guardrails AI
Input/output rails, NeMo's topic/flow control vs Guardrails AI's validators, defense-in-depth, and content-rails vs the action gate.
● Ready📊 Data & App Building · wrangle it, chart it, serve it, show it
Numerical computing & data analysis
The vectorized mindset, DataFrames, the load→clean→transform→aggregate loop, group-by, and shaping tables into LLM-ready records.
● ReadyData visualization
Matplotlib's Figure/Axes model, Seaborn straight from a DataFrame, choosing the right chart, and not lying with axes.
● ReadyAI APIs & backends with FastAPI
Typed endpoints, async for I/O-bound LLM calls, streaming tokens, error/dependency handling, and mapping to O3 deployment.
● ReadyAI-native apps with Streamlit & Gradio
Python-only UIs, the rerun/state model, a streaming chat app in both, and UI-only vs UI + FastAPI backend.
● Ready🎛️ Prompt & Context Engineering · get the most out of any LLM
Advanced prompting techniques
Few-shot, chain-of-thought, self-consistency, decomposition and ReAct — with an escalation ladder so you pick the lowest rung that works.
● ReadyContext engineering: memory, windowing & retrieval
The window as a budget, conversation memory (window/summary/retrieval), prompt caching, and why retrieval beats pasting whole documents.
● ReadyPrompt optimization, evaluation & DSPy
Stop guessing: a minimal eval harness, measuring on held-out data, and DSPy's signature→module→compile idea for prompts you compile, not hand-tune.
● ReadyWorking with LLMs: APIs, SDKs, parameters & OSS
Request/response anatomy, every parameter that matters, using an SDK well, and the real hosted-API-vs-self-hosted-open-source decision.
● Ready🔤 NLP & Transformers · how the model actually works
NLP foundations & text processing
Task families, why language is hard, the rules→statistical→neural→transformer arc, the preprocessing pipeline, and what classic NLP still wins.
● ReadyFeature engineering & text representation
Bag-of-words → TF-IDF → embeddings, cosine similarity, and why meaning-as-vectors powers RAG. Links A4, Ch 3, A7.
● ReadySentiment & text classification
A classic TF-IDF+LogReg classifier, honest evaluation (precision/recall/F1, the accuracy trap), and classic-ML-vs-LLM trade-offs.
● ReadyNeural LMs & sequence modelling
Next-token prediction, n-grams, RNN/LSTM, the two walls (long-range fade + sequential compute), and the seq2seq bottleneck that birthed attention.
● ReadyTransformers → LLMs
Self-attention (Q/K/V), the transformer block, why it beat RNNs, and the path architecture → pretraining → alignment → the LLM you call.
● Ready🅰️ Anthropic Skills · Claude Code, Cowork, Skills, Subagents, MCP & Vertex, in depth
Claude Code in Action
Steering extended sessions, context management, configuration (CLAUDE.md, settings), permissions, and verification — going beyond the basics of C3.
● ReadyClaude Code automation
Headless mode, hooks, slash commands, output styles, CI/CD integration, and scripting Claude Code into your pipelines.
● ReadyClaude Cowork
Task loops, plugins, skills, and file workflows for responsible multi-step project management with Claude.
● ReadyAgent Skills
Building, configuring, and distributing reusable markdown instruction sets (Skills) that extend Claude Code with domain knowledge.
● ReadySubagents
Creating and managing specialized sub-agents to organize workflows, isolate context, and parallelize work.
● ReadyMCP: advanced topics
Sampling, notifications, roots, elicitation, filesystem access, and transports (stdio/HTTP) — production MCP beyond C4.
● ReadyClaude on Vertex / GCP
Running Claude on Google Cloud Vertex AI — auth, the Vertex SDK, streaming, tool use; the GCP parallel to the Bedrock track.
● ReadyAI Fluency: framework
Delegation, description, discernment, diligence — the framework for collaborating with AI effectively, ethically, and safely.
● ReadyAgent workbench (capstone)
Assemble Claude Code + custom Skills + Subagents + an MCP server into one automated, verifiable developer workbench, end to end.
● Ready🎙️ Frontier Agent Capabilities · voice, computer use, reasoning, memory & modern retrieval
Voice & realtime agents
Speech-to-speech vs the STT→LLM→TTS pipeline, streaming transport, turn detection & barge-in, and the ~800ms latency budget that makes a voice agent feel human.
● ReadyComputer use & browser agents
How a model 'sees' a screen and returns click/type actions — the perceive→plan→act loop, grounding, sandboxing, and when a browser agent beats an API.
● ReadyReasoning & test-time compute
Extended/adaptive thinking, why spending compute before answering lifts hard-task accuracy, the effort dial (low→max), and routing fast vs reasoning by cost.
● ReadyLong-term agent memory
Why the context window isn't memory: episodic vs semantic stores, the write/retrieve/consolidate loop, and the Mem0 / Letta (MemGPT) self-editing-memory pattern.
● ReadyRAG evaluation
Faithfulness, answer relevance, context precision & recall — the four metrics that catch retrieval vs generation failures separately, and eval-gating RAG in CI (Ragas).
● ReadyAdvanced retrieval
Beyond naive top-k: contextual retrieval, hybrid search + RRF fusion, cross-encoder reranking, embedding fine-tuning, and structure-aware chunking.
● ReadyModern agent SDKs
The newer typed/lightweight frameworks — OpenAI Agents SDK (handoffs, guardrails, sessions) and Pydantic AI (type-safe agents) — vs the LangChain/CrewAI stack.
● ReadyVoice agent with memory (capstone)
Compose the whole track: a voice agent that remembers, escalates hard turns to a reasoning model, retrieves with an advanced pipeline, and ships behind an eval gate.
● Ready🧬 ML Systems Internals · GPU memory, distributed training, kernels & serving at scale
GPU memory & the model
Where VRAM actually goes — weights, KV cache, activations, optimizer state — and the precision math (fp16/fp8/int4) that decides whether a 70B model fits. OOM debugging, methodically.
● ReadyDistributed training
Data vs tensor vs pipeline parallelism; DDP, FSDP/ZeRO, gradient accumulation. Why one GPU isn't enough, why communication is the bottleneck, and how to choose a strategy by scale.
● ReadyAttention & GPU kernels
The silicon layer: SMs, HBM vs SRAM, the memory-bandwidth wall. Why attention is memory-bound, how FlashAttention tiles + online-softmax to avoid the N×N matrix, and the roofline model.
● ReadyMulti-GPU serving
Serving a model too big for one GPU: tensor parallelism at inference vs replicas, vLLM/TGI multi-GPU, and KV-cache memory as the real concurrency ceiling. Autoscaling & load balancing.
● ReadyThroughput economics
The unit economics of serving: tokens/sec/GPU → $/1M tokens, how batching lowers cost to the knee, utilization as the hidden cost, and the self-host vs API break-even under an SLO.
● ReadyServe a model at scale (capstone)
Given a model, an SLO, and a traffic level: size the VRAM, decide sharding, size the fleet, pick batching, and produce a defensible $/1M-token cost model — with a staff-level grading rubric.
● Ready🧩 Advanced Challenges · diagnose, design, and defend like a staff engineer
Debug a broken RAG
A RAG system returns wrong, empty, or hallucinated answers. Systematically isolate the failing layer — chunking, embedding, retrieval, ranking, or generation — reproduce it, and fix it.
● ReadyDebug a broken agent
An agent loops forever, calls the wrong tool, ignores results, blows its budget, or acts unsafely. Diagnose the loop, tool schemas, context management, and stop conditions — with runnable repros.
● ReadyDesign under constraints
Tight, conflicting constraints — 300ms p99, a cost ceiling, data residency, air-gapped. The skill isn't one right answer; it's finding the binding constraint and defending the tradeoff.
● ReadyIncident post-mortem
Real-shaped LLM incidents — an injection breach, a cost blowout, a silent regression, a cascading outage. Timeline, 5-whys root cause, the systemic fix, and how to write a blameless post-mortem.
● ReadyDefend a tradeoff
RAG vs fine-tune, build vs buy, agent vs pipeline, sync vs batch. State a decision and defend it with weighted criteria — acknowledging what you give up and when the call flips.
● ReadyAudit & harden a system (capstone)
Given a plausible-but-flawed production system, run a full audit across evals, cost, latency, safety, and observability; find the gaps; and produce a severity-ranked, verifiable fix list.
● Ready⚡ Inference & Cost Optimization · make models fast & cheap to serve
The inference cost model
Why LLM serving is hard: I/O-bound, token-metered, memory-bound. Latency vs throughput, the roofline, and where the money goes.
● ReadyQuantization: GPTQ, AWQ, GGUF
FP16→INT8→INT4: how quantization shrinks models and speeds inference, the formats (GPTQ/AWQ/GGUF), and the accuracy tradeoff.
● ReadyThe KV-cache & attention memory
Why the KV-cache dominates memory, PagedAttention, context-length cost, and prompt caching as a first-class lever.
● ReadyContinuous batching & scheduling
Static vs continuous batching, how a scheduler packs requests, and why batching is the biggest throughput win.
● ReadySpeculative & parallel decoding
Draft models, speculative decoding, and other tricks that cut latency without changing the output distribution.
● ReadyServing engines: vLLM & TGI
Standing up vLLM/TGI, the OpenAI-compatible server, tensor/pipeline parallelism, and tuning for your traffic.
● ReadyOptimize a serving stack
Take an open model, quantize it, serve it on vLLM with continuous batching, and cut p95 latency + cost/1k-tokens against a baseline.
● Ready🎚️ Fine-tuning Hands-On · the "yes, and here's how"
When fine-tuning wins
Recap the decision, then the cases where tuning genuinely beats prompting/RAG: style, format, latency, and narrow tasks.
● ReadyDataset preparation
The part that decides success: sourcing, formatting (chat templates), cleaning, dedup, splits, and how much data you actually need.
● ReadyLoRA & QLoRA with PEFT
Parameter-efficient tuning: how LoRA works, QLoRA on a single GPU, and a full PEFT training run with the transformers stack.
● ReadyFull fine-tune vs PEFT
When you need full fine-tuning, the memory math, and how to choose adapter rank, targets, and hyperparameters.
● ReadyDPO, RLHF & alignment
Aligning behavior with preferences: SFT→DPO, how RLHF works conceptually, and building a preference dataset.
● ReadyEvaluate & serve the model
Eval-after-tuning (did it help without regressing?), merging adapters, and serving the tuned model in production.
● ReadyFine-tune a task adapter
Prepare a dataset, QLoRA-tune an open model for a narrow task, eval it against the base, and serve the merged model.
● Ready🖥️ Local & Open-Weight Models · run models on your own hardware
The open-weight landscape
Llama, Mistral/Mixtral, Qwen, Gemma, Phi: licenses, sizes, what runs on what hardware, and how to choose.
● ReadyLocal development with Ollama
Run a model on your laptop in minutes, the OpenAI-compatible API, Modelfiles, and wiring it into the course code.
● ReadyProduction serving with vLLM
From laptop to server: vLLM for high-throughput self-hosting, GPU sizing, and an OpenAI-compatible endpoint your app already speaks.
● ReadyQuantized inference on your hardware
GGUF + llama.cpp on CPU/Apple Silicon, GPU quantized inference, and picking a quant level for your memory budget.
● ReadyAPI vs self-host economics
The honest cost/ops comparison: break-even math, latency/quality tradeoffs, data-residency, and when each wins.
● ReadySelf-host an open model
Serve a quantized open model with vLLM behind an OpenAI-compatible API, point the course code at it, and benchmark vs a hosted API.
● Ready🚨 AI Safety & Red-teaming · break it before someone else does
The LLM threat model
What can go wrong: prompt injection (direct/indirect), jailbreaks, data exfiltration, tool abuse. A taxonomy you can test against.
● ReadyRed-teaming by hand
Systematic manual attacks: jailbreak patterns, injection via retrieved content, and how to run a structured red-team exercise.
● ReadyAutomated red-teaming
Scale it: attack generation, an LLM-as-attacker harness, scoring, and regression-testing safety like you test code.
● ReadyDefenses in depth
Input/output guardrails, isolation, least-privilege tools, human-in-the-loop, and why no single layer is enough.
● ReadyGovernance, compliance & bias
EU AI Act risk tiers, model cards, audit trails, bias/fairness auditing, and the incident runbook for a regulated AI system.
● ReadyRed-team & harden an agent
Take the Ch 8 DevOps agent, red-team it (manual + automated), fix what breaks, and produce a model card + safety report.
● Ready🎯 Career & Interview Prep · turn all this into the job (& the promotion)
Resume & portfolio
A resume that passes the 6-second scan & the ATS, and a portfolio (GitHub + projects) that proves you can build — with a runnable resume/keyword checker.
● ReadyBehavioral & STAR
Tell your story: the STAR method, a bank of answers, handling "tell me about a failure", and reading what interviewers actually assess.
● ReadyThe coding interview
Not the problems (that's DSA) but the process: clarify→approach→code→test→complexity, thinking aloud, and recovering when stuck.
● ReadySystem-design interview
Walk a real prompt with the SD method: requirements→estimation→API→data→architecture→scale, plus what senior vs staff signals look like.
● ReadyOffers & negotiation
Evaluate an offer (comp, equity, growth), negotiate without burning bridges, and choose between offers — with a runnable offer comparator.
● ReadyLand the role & grow to tech-lead
A concrete job-search plan, then the first-90-days playbook and the path from engineer → senior → tech-lead.
● Ready🏛️ Case Studies & Reference Architectures · end-to-end designs, real patterns, honest scenarios
Agents vs workflows
Anthropic's published guidance made practical: most "agent" problems are better solved by composable workflows. The five workflow patterns + the autonomous-agent pattern, each with when-to-use and runnable code — and how to tell which you actually need.
● ReadyCase study · support triage
A representative end-to-end system: classify & route inbound support, answer with RAG, gate on confidence, escalate to a human. Requirements → architecture → code → evals → cost/latency at scale → failure modes.
● ReadyCase study · compliance docs
A regulated-industry design: extract obligations and flag risk over large documents with citations, abstention, human sign-off, and an audit trail — where a wrong answer is a legal liability, so the system abstains, cites, and gates.
● ReadyCase study · clinical-knowledge RAG
Grounded answers over vetted medical literature under strict PII/PHI and data-residency constraints — self-host vs region-locked, redaction, citations, abstention, and clinician-in-the-loop. Decision-support only, never medical advice.
● ReadyScale & cost war-stories
Representative production failures and their fixes: cost blowouts (caching), missed latency SLOs (tiering + streaming), rate-limit outages (backoff + breaker), slow bulk jobs (Batches), silent regressions (eval gates), context blowups (trimming).
● ReadySafety by design & constitutional AI
Anthropic's public safety research explained — Constitutional AI, RLAIF, the Responsible-Scaling mindset — plus how you build safely on top: layered defenses, refusals, guardrails, red-teaming, and the incident→eval loop.
● ReadyWrite a reference design (capstone)
The finale: take a system idea and produce a complete reference design — requirements, the agents-vs-workflows decision, component tradeoffs, data flow, safety, evals, a cost/latency budget, failure modes, and a rollout plan — with a runnable skeleton and a master grading rubric.
● Ready🧭 Find your way · paths & portfolio
Reference · keep these open while you work
pip), a terminal, and basic JSON. No ML background required — we build intuition as we go.