Vector Databases, Embeddings & Frameworks

An embedding is a list of numbers (a vector) that captures the meaning of a piece of text. An embedding model turns "how to scale a deployment" into ~384 numbers; two texts that mean similar things get vectors that point in similar directions. This block measures how similar two vectors are.

1. The commented lines at the top show the real thing: SentenceTransformer loads a small model that runs on your own machine and .encode([...]) turns each sentence into a 384-number vector. They are commented out so the demo runs with no install.
2. cosine(a, b) computes cosine similarity: it takes the dot product a @ b and divides by each vector's length (np.linalg.norm). That cancels out size and leaves only direction — how aligned the two meanings are.
3. q and d1 are stand-in random 384-number vectors so you can run the code today; in a real system they'd come from the model above.
4. print(round(cosine(q, d1), 3)) shows the similarity score for the query vector q against document vector d1.

What the output means: A single number between -1 and 1. Higher = more related (1 = same direction, 0 = unrelated, -1 = opposite). Because these are random vectors, you'll get something near 0; real related sentences score much higher.

Try this: Make d1 = q (identical) and re-run — the score jumps to 1.0, the maximum. That is what "most similar" looks like.

This is the simplest possible search: compare the query to every document and keep the best few. It is 100% accurate, and it's the baseline every faster method is measured against. The catch is speed once you have millions of documents.

1. corpus @ query is one matrix multiply that scores the query against all N documents at once. Because the vectors are normalized, a higher score means more similar (this is the cosine idea from step 1).
2. np.argpartition(-scores, k)[:k] grabs the top-k highest-scoring documents without fully sorting the rest — a fast "find the k best" trick (the heap idea from D4). The minus sign flips it so the biggest scores come first.
3. idx[np.argsort(-scores[idx])] then sorts just those k winners so the very best is first, and returns their positions in the corpus.

What the output means: The function returns the indexes (positions) of the k most similar documents, best first. Retrieval is exact — you always get the true nearest neighbors.

Try this: Note the comment: this is O(N*d) per query — fine up to ~100k documents, but at 10M it scans gigabytes every time. That pain is exactly why the next sections exist.

When exact search gets too slow, you use an ANN (approximate nearest neighbor) index: a clever data structure that finds almost the best matches far faster by not looking at every vector. FAISS is a popular library for this. The real calls are shown as comments; the runnable line just prints a summary.

1. faiss.IndexHNSWFlat(d, 32) builds an HNSW index — a layered graph you hop through toward the query (like zooming in on a map), so query time is roughly O(log N) instead of scanning all N. The 32 is how many links each node keeps.
2. index.add(corpus) loads your document vectors into the index (this is the one-time "build" step). index.search(query, k=5) then returns the top-5 approximate matches.
3. The comments note you can tune the recall-vs-speed dial (HNSW's efSearch or IVF's nprobe): search harder for more-accurate results, or less for more speed.

What the output means: "Approximate" means you might occasionally miss a true top result, but you get answers dramatically faster. FAISS runs in-process (inside your app, no separate server), which is why it's great for embedding search directly into a program.

Try this: Compare mentally to section 2: exact search is always right but O(N); HNSW trades a tiny bit of accuracy for a huge speedup. That trade is the whole point of ANN.

A vector database is an ANN index plus everything a real system needs: it stores your data on disk, lets you add/update/delete, and — crucially — lets you filter by metadata (tags like which customer, environment, or date). Chroma is a beginner-friendly one. The real calls are commented; the last line prints a summary.

1. client.create_collection("runbooks") makes a named bucket to hold documents (like a table). col.add(...) inserts them — notice you pass plain documents=[...] text and Chroma embeds them for you automatically.
2. Each document also gets metadatas=[{"env": "prod"}, ...] — extra labels attached to the vector. These are the fields you can later filter on.
3. col.query(query_texts=["scale up"], n_results=3, where={"env": "prod"}) does the search: it finds the nearest vectors and keeps only those whose env is "prod". The where clause is the metadata filter.

What the output means: You get back the top matching documents that also satisfy the filter — combining meaning-based search with hard scoping rules in one call.

Try this: Read the tip below the code: a lot of "bad retrieval" is really "searched the wrong scope." Filtering by tenant/source/date often helps more than a fancier index.

Meaning-based (dense) search and keyword-based (sparse/BM25) search each miss different things. Hybrid search runs both and merges the two ranked lists into one. Reciprocal Rank Fusion (RRF) is a simple, robust way to merge: a document scores higher when it ranks near the top of either list, and highest when it ranks well in both.

1. For each ranked list, enumerate(ranking) gives every document its position (rank 0 = first). RRF adds 1 / (k + rank + 1) to that document's score — top items get a big boost, lower items get a smaller one. The constant k=60 just softens the difference between adjacent ranks.
2. scores.get(doc_id, 0) + ... accumulates across lists, so a document that appears in several lists collects points from each one.
3. sorted(scores, key=scores.get, reverse=True) returns the document IDs ordered by total fused score, best first.
4. dense comes from vector search and sparse from BM25 keyword search — two different opinions about relevance that RRF blends.

What the output means: The printed order puts d1 and d3 near the top because they appear in both lists — agreement across methods is a strong signal of relevance.

Try this: Notice the final comment: after fusing, you'd re-rank the top ~20 with a cross-encoder (a slower, more accurate model) for the final order. Fuse broadly, then sharpen the top.

Frameworks like LlamaIndex and LangChain are pre-built toolkits that bundle the steps you've seen (chunk → embed → index → retrieve → prompt) so you write far less glue code. This block shows the typical shape of each — as comments, because the point is to read the pattern, not run it.

1. LlamaIndex: SimpleDirectoryReader("runbooks/").load_data() loads your files, VectorStoreIndex.from_documents(docs) does chunk + embed + index in one call, and .as_query_engine().query("...") retrieves relevant chunks and asks the model — a whole RAG pipeline in three lines.
2. LangChain (LCEL): prompt | model | output_parser pipes steps left-to-right with the | symbol. Each piece is a "Runnable"; the data flows through them like a Unix pipeline. chain.invoke({...}) runs it.
3. The closing comments make the key point: both frameworks wrap the same primitives this course builds by hand — chunking, embeddings, the vector index, the agent loop, structured output.

What the output means: "Frameworks = the course's concepts, pre-assembled." There's no magic; they just name and package the building blocks you already understand.

Try this: Map each framework call back to a concept: from_documents = chunk+embed+index; as_query_engine = retrieve+prompt. If you can name the primitive, you can debug the framework.

This isn't runnable code — it's an interview answer sketch in comments. For "design RAG over 10M documents," interviewers grade how you split the system into an offline (prep) stage and an online (per-query) stage, and how you handle scale and failure.

1. OFFLINE (done ahead of time): take documents, split them into overlapping chunks, embed each chunk into a vector, and load them into an ANN index (HNSW/IVF) alongside metadata. This is the expensive build you do once and reuse.
2. ONLINE (per user query): embed the query, retrieve the top-k nearest chunks, re-rank them, stuff them into the prompt as context, and cite the sources in the answer.
3. SCALE: shard (split) the index across machines when it's too big for one, cache answers to popular queries, and combine dense + BM25 with RRF (the hybrid idea from section 5).
4. FAILURE: a stale index or a missed retrieval leads to hallucination — so you measure recall@k (did we fetch the right docs?) and run answer-quality evals.

Try this: In an interview, narrate these four lines out loud. Separating offline-build from online-serving, and naming a failure mode with how you'd measure it, is what earns the marks.

This is the single operation at the heart of every retrieval system, written cleanly for an interview: given a corpus of document vectors and one query vector, return the k most similar documents. It folds the normalization step directly into the function.

1. c = corpus / np.linalg.norm(corpus, axis=1, keepdims=True) normalizes every document vector to length 1. axis=1 means "per row (per document)"; keepdims=True keeps the shape so the division lines up. After this, cosine similarity is just a dot product.
2. q = query / np.linalg.norm(query) normalizes the query the same way.
3. scores = c @ q is one matrix multiply that scores all documents against the query at once — the whole search in a single line.
4. np.argpartition(-scores, k)[:k] pulls the top-k, then argsort orders just those k so the best is first (same top-k pattern as section 2).

What the output means: Returns the positions of the k most similar documents, best first — the exact-search core that an ANN index or vector DB optimizes when N gets huge.

Try this: Be ready to state the cost: O(N*d) for the matmul. If asked to scale it, that's your cue to bring up HNSW/IVF and vector databases.