AI EngineeringZero to ProductionHome·About·Contact
Appendix · Data Structures & Algorithms · Part 5

Graphs

The most general structure: nodes connected by edges, with no rules about shape. Trees and linked lists are just special graphs. Graphs model dependencies, networks, maps, and knowledge — and the agent world is full of them: a DevOps deploy plan, a tool-call chain, a knowledge graph, a citation network. This part covers representation, the two core traversals, and the classic weighted algorithms.

⏱️ ~2.5 hours🎯 Basic → Expert🕸️ networks & depsrunnable

Learning objectives

  • Represent a graph as an adjacency list (and know when a matrix is better).
  • Traverse with BFS (queue) and DFS (stack/recursion), and know what each finds.
  • Order a dependency graph with topological sort and detect cycles.
  • Find shortest paths with Dijkstra (non-negative) and Bellman-Ford (negatives).
  • Build a minimum spanning tree and understand union-find.

1 · What is a graph? basic

A graph is a set of vertices (nodes) and edges (connections). Edges can be directed (one-way, like "task A must finish before B") or undirected (two-way, like "these two services talk"), and weighted (a cost/distance) or not. Almost every real-world relationship is a graph.

TermMeaningAgent example
vertex / nodean entitya microservice, a document, a task
edgea relationship"depends on", "links to", "calls"
directedone-way edgebuild → test → deploy
weightededge has a costlatency, distance, similarity
cyclea path back to starta circular dependency (usually a bug!)
DAGdirected, no cyclesa valid build/deploy pipeline

2 · Representations basic → intermediate

Two standard ways to store a graph. The adjacency list (a dict: node → list of neighbours) is the default — compact for the sparse graphs that occur in practice. The adjacency matrix (an n×n grid) gives O(1) edge lookups but uses O(n²) memory, only worth it for dense graphs.

graph A B C Adjacency list · O(V+E) A: [B, C] B: [A, C] C: [A, B] Matrix · O(V²), O(1) edge test A B C A [ 0 1 1 ] B [ 1 0 1 ] C [ 1 1 0 ] Two representations, same graph. The adjacency list is compact for sparse graphs (most real ones); the matrix answers "is there an edge A–C?" in O(1) but costs O(V²) memory.
🗺️ How to read this diagram

This shows the same tiny graph stored two different ways. A graph is just dots (nodes) joined by lines (edges). Here three nodes — A, B, C — are all connected to each other.

  • On the left are the three circles (nodes) and the lines between them (edges). A line between two nodes means "these two are connected / can reach each other".
  • The adjacency list (top right) stores, for each node, the list of its neighbours. A: [B, C] reads "A is connected to B and C". This is compact because it only records edges that actually exist — cost O(V+E) (V = number of nodes, E = number of edges).
  • The matrix (bottom right) is a grid with one row and one column per node. A 1 at row A, column B means "there is an edge A–B"; a 0 means "no edge". Looking up any single edge is instant (O(1)), but the grid always has V×V cells even when most are 0O(V²) memory.
  • Notice the diagonal is all 0: a node has no edge to itself, and the grid is mirror-symmetric because these edges are undirected (two-way).

In short: Use a list when the graph is sparse (few edges — almost all real graphs). Use a matrix only for tiny or very dense graphs where instant edge look-up matters.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Try it — adjacency list
pythonfrom collections import defaultdict

class Graph:
    def __init__(self, directed=False):
        self.adj = defaultdict(list)     # node -> [neighbours]
        self.directed = directed

    def add_edge(self, u, v, w=1):
        self.adj[u].append((v, w))       # store neighbour + weight
        if not self.directed:
            self.adj[v].append((u, w))

# A DevOps dependency graph (directed): build -> test -> deploy
g = Graph(directed=True)
g.add_edge("build", "test")
g.add_edge("test", "deploy")
g.add_edge("build", "lint")
print(dict(g.adj))
▶ How this works

This builds the adjacency list from the diagram as a real Python class. A Graph object holds a dictionary that maps each node to the list of nodes it points to, plus a flag saying whether edges are one-way (directed) or two-way.

  1. self.adj = defaultdict(list) is the storage. A defaultdict(list) is a dictionary that automatically starts a fresh empty list the first time you touch a new key — so you never have to check "does this node exist yet?" before appending.
  2. add_edge(u, v, w=1) records an edge from u to v with weight w (a cost; default 1). It appends the pair (v, w) to u's neighbour list — storing the neighbour and the edge's cost together.
  3. if not self.directed: — for a two-way graph it also adds the reverse edge v → u, so both nodes list each other. A directed graph skips this, keeping the edge one-way.
  4. The demo builds a directed DevOps graph: build → test, test → deploy, build → lint. print(dict(g.adj)) shows the finished map.

What the output means: You see the adjacency dict, e.g. {'build': [('test', 1), ('lint', 1)], 'test': [('deploy', 1)]} — each node listed with its outgoing neighbours and weights.

Try this: Change Graph(directed=True) to Graph(directed=False) and reprint. Now every edge appears in both directions — that is the only difference between a one-way and a two-way graph.

Adjacency listAdjacency matrix
spaceO(V+E)O(V²)
is edge (u,v)?O(deg u)O(1)
iterate neighboursO(deg u)O(V)
best forsparse (most real graphs)dense / tiny graphs

3 · Breadth-first search (BFS) intermediate

BFS explores the graph in rings outward from the start — all nodes at distance 1, then distance 2, etc. It uses a queue (D2's deque) and a visited set (D3). Its superpower: on an unweighted graph, the first time BFS reaches a node is via a shortest path (fewest edges).

BFS visits by distance from the start — ring by ring S A B D E dist 0: S dist 1: A, B (queue processes these next) dist 2: D, E first arrival = shortest (fewest edges) BFS spreads out in rings. A FIFO queue guarantees every distance-1 node is seen before any distance-2 node — so on an unweighted graph the first time you reach a node is the shortest path to it.
🗺️ How to read this diagram

This shows Breadth-First Search (BFS): starting at node S, it visits the graph in rings, like ripples spreading out on water. First all nodes one step away, then all nodes two steps away, and so on.

  • S (green, centre) is the start. The two dashed circles are the "rings" — everything on the inner ring is distance 1 from S, everything on the outer ring is distance 2.
  • The lines are edges. A and B touch S directly, so they are distance 1. D and E are reached only through A or B, so they are distance 2.
  • The right-hand list is the visit order: dist 0: S, then dist 1: A, B, then dist 2: D, E. BFS always finishes a whole ring before starting the next — that ring-by-ring order is what a queue gives you.
  • The green line is the payoff: because rings are visited in distance order, the first time BFS reaches any node is by the shortest path (fewest edges) to it.

In short: BFS = "nearest first." On a graph with no edge weights, the number of the ring a node sits in is its shortest distance from the start.

Try it — BFS + shortest unweighted path
pythonfrom collections import deque

def bfs(graph, start):
    visited = {start}
    q = deque([start])
    order = []
    while q:
        node = q.popleft()               # FIFO -> ring-by-ring
        order.append(node)
        for nbr, _ in graph.adj[node]:
            if nbr not in visited:
                visited.add(nbr)         # mark on enqueue (not dequeue!)
                q.append(nbr)
    return order

def shortest_path(graph, start, goal):
    q = deque([[start]])                 # queue of PATHS
    visited = {start}
    while q:
        path = q.popleft()
        node = path[-1]
        if node == goal:
            return path                  # first time we reach goal = shortest
        for nbr, _ in graph.adj[node]:
            if nbr not in visited:
                visited.add(nbr)
                q.append(path + [nbr])
    return None
▶ How this works

Two functions here. bfs visits every reachable node in ring order; shortest_path reuses the same idea to return the actual shortest route between two nodes. Both rely on a queue — a line where the first item added is the first removed (FIFO), exactly like a checkout line.

  1. visited = {start} is a set of nodes we've already seen, so we never process one twice. q = deque([start]) is the queue, primed with the start node.
  2. node = q.popleft() takes the node from the front of the queue (FIFO). Because we add nodes in the order we discover them and remove from the front, we naturally drain one ring before the next.
  3. For each neighbour nbr, if it's not already visited we mark it visited and append it to the back of the queue. Marking it now (on enqueue) is critical — see the warning box below.
  4. shortest_path is the same loop, but the queue holds whole paths (lists of nodes) instead of single nodes. It grows each path with path + [nbr]. The first path whose last node equals goal is returned — and because BFS reaches nodes nearest-first, that first path is the shortest one.

What the output means: bfs returns the visit order as a list, e.g. ['S', 'A', 'B', 'D', 'E']. shortest_path returns the node list of the shortest route, e.g. ['S', 'A', 'D'], or None if the goal can't be reached.

Try this: Trace shortest_path for a start with two routes to the goal — a short one and a long one. BFS returns the short one because it fills nearer rings first, so the goal is found via the fewest edges before the long route ever finishes.

The mark-on-enqueue ruleMark a node visited when you add it to the queue, not when you remove it. Otherwise the same node can be enqueued many times before it's processed, and you'll do redundant work (or loop forever on a cycle). This single detail is the most common BFS/DFS bug.

4 · Depth-first search (DFS) intermediate

DFS goes as deep as possible before backtracking — it uses a stack (explicit, or the call stack via recursion). It's the tool for "explore everything," cycle detection, topological sort, connected components, and (as you saw in D3) backtracking.

dive down one branch fully, then backtrack to the next S A B C visit order: S → A → C dead end at C → backtrack …then explore B stack (LIFO) drives the "go deep" order DFS follows one path to the end before trying alternatives. A stack (or recursion's call stack) remembers where to backtrack. Compare with BFS above: swap the queue for a stack and "nearest-first" becomes "deepest-first."
🗺️ How to read this diagram

This shows Depth-First Search (DFS), the opposite strategy to BFS. Instead of spreading out evenly, DFS picks one path and follows it all the way down before trying anything else — like exploring a maze by always taking the next corridor until you hit a dead end.

  • Start at S (green). DFS goes down to A, then deeper to C — the solid arrows show this "dive" straight down one branch.
  • C is a dead end (nothing new below it), so DFS backtracks: it returns up to a node that still has an unexplored neighbour and tries that. Here it then explores B (the dashed sideways arrow).
  • The visit order is therefore S → A → C (dive), then back up and on to B — deep first, siblings later. Contrast BFS, which would have taken A and B together as one ring.
  • A stack drives this (LIFO — last in, first out). The newest node to explore is always tackled next, which is what keeps pushing you deeper before you back out.

In short: BFS vs DFS is one swap: a queue (take the oldest waiting node → nearest-first, spreads in rings) vs a stack (take the newest → deepest-first, dives down one branch).

Try it — recursive and iterative DFS
pythondef dfs_recursive(graph, node, visited=None):
    if visited is None:
        visited = set()
    visited.add(node)
    for nbr, _ in graph.adj[node]:
        if nbr not in visited:
            dfs_recursive(graph, nbr, visited)
    return visited

def dfs_iterative(graph, start):     # explicit stack — no recursion limit
    visited, stack = set(), [start]
    while stack:
        node = stack.pop()               # LIFO -> go deep
        if node in visited:
            continue
        visited.add(node)
        for nbr, _ in graph.adj[node]:
            if nbr not in visited:
                stack.append(nbr)
    return visited
▶ How this works

Two ways to write DFS. dfs_recursive lets Python's own function-call stack do the remembering; dfs_iterative keeps an explicit list as the stack. They visit the same nodes — just differently under the hood.

  1. dfs_recursive(graph, node, visited=None) marks node visited, then calls itself on each unvisited neighbour. Each nested call dives one level deeper; when a call finds no new neighbours it returns (that's the backtrack), and the caller continues with its next neighbour.
  2. if visited is None: visited = set() creates the shared "seen" set on the very first call. (A fresh set each top-level call — never reuse a mutable default.)
  3. dfs_iterative replaces recursion with a plain list stack = [start]. node = stack.pop() removes the last item (LIFO), which is why we go deep: the most recently pushed neighbour is explored next.
  4. if node in visited: continue — because a node can be pushed more than once here, we skip it if already processed. The iterative form avoids Python's recursion depth limit, so it's the safe choice for very deep or large graphs.

What the output means: Both return the set of all nodes reachable from the start, e.g. {'S', 'A', 'C', 'B'}. The set doesn't record order — use a list if you want the exact visit sequence.

Try this: Give the graph a cycle (e.g. an edge that loops back to the start). The visited set stops both versions from looping forever — remove that check and the recursive one will crash with a stack overflow.

BFS vs DFS — the one-line ruleBFS = queue = shortest path / nearest first. DFS = stack = go deep / explore all / detect cycles. Same code skeleton; swap popleft() (queue) for pop() (stack) and the behaviour flips. Both are O(V+E) — they touch every vertex and edge once.

5 · Topological sort — ordering a dependency graph advanced

Given a directed acyclic graph (DAG) of "must happen before" edges, a topological sort produces a linear order respecting all dependencies. This is exactly how a build system, task scheduler, or the DevOps agent decides what to run in what order. Kahn's algorithm uses in-degrees + a queue; a cycle is detected when not all nodes get ordered.

DAG of "must finish before" edges → a safe linear order build lint test deploy order: build → lint → test → deploy (in-degree 0 first) Topological sort orders a dependency graph. Kahn's algorithm repeatedly takes a node with in-degree 0 (no unmet prerequisites), emits it, and removes its edges. If nodes remain but none has in-degree 0, there's a cycle — an impossible ordering.
🗺️ How to read this diagram

This shows a topological sort: turning a graph of "must happen before" arrows into a single safe order to do the tasks. The graph here is a build pipeline where each arrow means "the task at the tail must finish before the task at the head".

  • The arrows are directed (one-way). build → lint and build → test mean build must run first; test → deploy means deploy waits for test. This shape — directed with no loops — is called a DAG.
  • The key idea is in-degree: how many arrows point into a node. build has in-degree 0 (nothing must happen before it), so it can go first.
  • Kahn's algorithm: emit any in-degree-0 node, then "remove" its outgoing arrows, which lowers its neighbours' in-degrees. Do this repeatedly. Here that yields build → lint → test → deploy (bottom, green).
  • If at some point nodes remain but none has in-degree 0, the arrows form a cycle (A waits for B which waits for A) — an impossible order, which the algorithm reports as an error.

In short: "In-degree 0 first" = "do whatever has no unfinished prerequisites next." That single rule, repeated, is exactly how a build system or scheduler picks a safe run order.

Try it — Kahn's algorithm (also detects cycles)
Setup to run this snippet
class _Any:
    '''stands in for any undefined demo value; supports call/attr/index/
    iteration and basic arithmetic (as 0.7) so demo snippets run.'''
    def __call__(self, *a, **k): return _Any()
    def __getattr__(self, k): return _Any()
    def __getitem__(self, k): return _Any()
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __contains__(self, o): return True
    def __enter__(self, *a): return _Any()
    def __exit__(self, *a): return False
    def __float__(self): return 0.7
    def __int__(self): return 1
    def __lt__(self, o): return True
    def __gt__(self, o): return False
    def __le__(self, o): return True
    def __ge__(self, o): return False
    def __add__(self, o): return o
    def __radd__(self, o): return o
    def __bool__(self): return True
    def __repr__(self): return 'demo'
    def __str__(self): return 'demo'
g = _Any()
pythonfrom collections import deque

def topological_sort(graph, nodes):
    indeg = {n: 0 for n in nodes}
    for u in nodes:
        for v, _ in graph.adj[u]:
            indeg[v] += 1                  # count incoming edges
    q = deque([n for n in nodes if indeg[n] == 0])  # no prerequisites
    order = []
    while q:
        u = q.popleft()
        order.append(u)
        for v, _ in graph.adj[u]:
            indeg[v] -= 1                  # "complete" u -> free its dependents
            if indeg[v] == 0:
                q.append(v)
    if len(order) != len(nodes):
        raise ValueError("cycle detected — no valid ordering")
    return order

nodes = ["build", "test", "deploy", "lint"]
print(topological_sort(g, nodes))        # e.g. ['build','test','lint','deploy']
▶ How this works

This is Kahn's algorithm for topological sort in code. It counts how many prerequisites each node has, starts with the ones that have none, and "completes" them one at a time — freeing up their dependents as it goes.

  1. indeg = {n: 0 for n in nodes} then the double loop counts, for every node, how many edges point into it (its in-degree). indeg[v] += 1 runs once per incoming edge.
  2. q = deque([n for n in nodes if indeg[n] == 0]) seeds the queue with every node that has no prerequisites — these are safe to do immediately.
  3. The loop pops a ready node u, appends it to order, then for each neighbour v does indeg[v] -= 1 — treating "u is done" as removing u's outgoing edges. When a neighbour's count hits 0, all its prerequisites are done, so it joins the queue.
  4. if len(order) != len(nodes): — if we couldn't order every node, some were stuck with a permanent prerequisite, i.e. a cycle. We raise an error instead of returning a broken order.

What the output means: A valid ordering list, e.g. ['build', 'test', 'lint', 'deploy']. Several orders can be valid — any order that never puts a task before its prerequisite is correct.

Try this: Add an edge that creates a loop (e.g. make deploy a prerequisite of build). Now no node ever reaches in-degree 0 after the first pass, len(order) falls short, and the cycle error fires — that is the cycle detector.

🔗 Used in the courseThe DevOps agent (Ch 8) plans multi-step infra changes that have ordering constraints — "apply network before compute," "migrate DB before deploy." That plan is a DAG, and executing it safely is a topological sort with a cycle check (a circular dependency is a hard error the agent must refuse).

6 · Dijkstra's shortest path advanced

For a weighted graph with non-negative weights, Dijkstra finds the cheapest path from a source to every other node. It's BFS upgraded with a priority queue (D4's heap): always expand the closest-so-far node. O((V+E) log V) with a heap.

expand the closest node; relax its edges to improve distances A0 B1 C3 1 5 2 A→C direct = 5 A→B→C = 1+2 = 3 ✓ cheaper heap always pops the nearest next Dijkstra = BFS weighted by a heap. It always expands the closest unfinalized node and relaxes its edges (keeps a shorter route if found). The heap makes "closest next" cheap — O((V+E) log V). Needs non-negative weights.
🗺️ How to read this diagram

This shows Dijkstra's algorithm, which finds the cheapest path when edges have weights (costs). BFS counts edges; Dijkstra adds up weights and always follows the cheapest total so far. The number next to each node is its best-known distance from A.

  • The number on an edge is that edge's weight (cost to travel it). A→B costs 1, B→C costs 2, and the direct A→C costs 5.
  • The number inside/under a node is its cheapest total distance from the start: A = 0 (you're already there), B = 1, C = 3.
  • The right side shows the choice: going direct A→C costs 5, but detouring A→B→C costs 1 + 2 = 3 — cheaper! Updating C's distance from 5 to 3 when a cheaper route appears is called relaxing the edge.
  • A priority queue (a min-heap) always hands back the closest unfinished node next, so Dijkstra finalizes nodes in increasing distance — the weighted version of BFS's rings.

In short: Dijkstra = BFS that counts cost instead of steps. "Relax" just means "if I found a cheaper way to reach this node, remember the cheaper number."

Try it — Dijkstra with heapq
Setup to run this snippet
class _Any:
    '''stands in for any undefined demo value; supports call/attr/index/
    iteration and basic arithmetic (as 0.7) so demo snippets run.'''
    def __call__(self, *a, **k): return _Any()
    def __getattr__(self, k): return _Any()
    def __getitem__(self, k): return _Any()
    def __iter__(self): return iter([])
    def __len__(self): return 0
    def __contains__(self, o): return True
    def __enter__(self, *a): return _Any()
    def __exit__(self, *a): return False
    def __float__(self): return 0.7
    def __int__(self): return 1
    def __lt__(self, o): return True
    def __gt__(self, o): return False
    def __le__(self, o): return True
    def __ge__(self, o): return False
    def __add__(self, o): return o
    def __radd__(self, o): return o
    def __bool__(self): return True
    def __repr__(self): return 'demo'
    def __str__(self): return 'demo'
def Graph(*a, **k):  # demo stub
    return _Any()
pythonimport heapq

def dijkstra(graph, start):
    dist = {start: 0}
    pq = [(0, start)]                    # (distance, node) — min-heap
    while pq:
        d, u = heapq.heappop(pq)          # closest unfinalized node
        if d > dist.get(u, float("inf")):
            continue                      # stale entry — skip
        for v, w in graph.adj[u]:
            nd = d + w
            if nd < dist.get(v, float("inf")):
                dist[v] = nd              # found a cheaper route to v
                heapq.heappush(pq, (nd, v))
    return dist

wg = Graph()
wg.add_edge("A", "B", 1); wg.add_edge("B", "C", 2)
wg.add_edge("A", "C", 5)
print(dijkstra(wg, "A"))               # {'A':0,'B':1,'C':3}  (A-B-C beats A-C)
▶ How this works

This is Dijkstra in code. It keeps a dist map of the best-known distance to each node and a min-heap (heapq) that always pops the closest unfinished node next — that ordering is what makes the answer correct.

  1. dist = {start: 0} — the start is 0 away from itself; every other node is unknown (treated as infinity) until we find a route. pq = [(0, start)] is the heap of (distance, node) pairs, ordered by distance.
  2. d, u = heapq.heappop(pq) pulls out the smallest-distance pair — the closest node we haven't finalized. Processing nodes closest-first is the heart of Dijkstra.
  3. if d > dist.get(u, float("inf")): continue skips stale heap entries: we may have pushed u more than once, and if we already have a better distance for it, this older, larger entry is ignored.
  4. For each neighbour, nd = d + w is the cost to reach v through u. If that beats v's current best, we relax: record dist[v] = nd and push (nd, v) so v gets re-examined with its improved distance.

What the output means: A dict of shortest distances from the start, e.g. {'A': 0, 'B': 1, 'C': 3} — confirming the A→B→C detour (3) beats the direct A→C edge (5).

Try this: Add an edge A→C with weight 2. Now the direct route ties/beats the detour and dist['C'] becomes 2 — watch Dijkstra pick whichever total is smallest.

Dijkstra breaks on negative edgesDijkstra assumes that once a node is finalized, no cheaper path exists — true only when weights are non-negative. A negative edge can make a longer route cheaper later, violating that assumption. For negatives you need Bellman-Ford.

7 · Bellman-Ford — handles negatives expert advanced

Bellman-Ford relaxes every edge, V−1 times. Slower than Dijkstra (O(V·E)) but it tolerates negative weights and — a bonus — detects negative cycles (if an edge can still be relaxed after V−1 rounds, a negative loop exists).

Try it
pythondef bellman_ford(edges, nodes, start):   # edges: list of (u, v, w)
    dist = {n: float("inf") for n in nodes}
    dist[start] = 0
    for _ in range(len(nodes) - 1):      # V-1 relaxation passes
        for u, v, w in edges:
            if dist[u] + w < dist[v]:
                dist[v] = dist[u] + w
    for u, v, w in edges:                # one more pass -> still improving?
        if dist[u] + w < dist[v]:
            raise ValueError("negative cycle detected")
    return dist
▶ How this works

Bellman-Ford also finds shortest paths, but unlike Dijkstra it works even when some edge weights are negative. It's simpler and slower: instead of a clever heap order, it just relaxes every edge, over and over.

  1. dist = {n: float("inf") for n in nodes} starts every node at infinity except the start (dist[start] = 0) — nothing is reachable yet.
  2. for _ in range(len(nodes) - 1): repeats the relaxation V−1 times (V = number of nodes). That many passes is provably enough for the shortest distance to "flow" all the way across the graph.
  3. Inside, if dist[u] + w < dist[v]: dist[v] = dist[u] + w is the same relax step as Dijkstra — take a cheaper route to v if going through u is better.
  4. The extra final pass is the clever bit: if any edge can still be relaxed after V−1 passes, distances would shrink forever — a negative cycle — so it raises an error.

What the output means: A dict of shortest distances from the start (like Dijkstra's), or a raised ValueError("negative cycle detected") if a negative loop makes shortest paths meaningless.

Try this: Use Dijkstra when all weights are ≥ 0 (faster). Reach for Bellman-Ford only when negative weights are possible — its one job Dijkstra can't do.

8 · Minimum spanning tree & union-find expert advanced

A minimum spanning tree connects all nodes with the least total edge weight and no cycles — the cheapest way to wire up a network. Kruskal's algorithm sorts edges by weight (D6) and adds each edge unless it would form a cycle. Detecting "would this form a cycle?" efficiently needs the union-find (disjoint-set) structure — near-O(1) per operation with path compression.

sort edges by weight; add each unless it makes a cycle (union-find) A B C 1 ✓ 2 ✓ 3 ✗ cycle add A–B (1) add A–C (2) skip B–C (3): already connected → cycle Kruskal builds the cheapest cycle-free tree. Take edges cheapest-first; union-find answers "are these two already connected?" in near-O(1), so an edge that would close a cycle is skipped. The result spans every node at minimum total weight.
🗺️ How to read this diagram

This shows Kruskal's algorithm building a minimum spanning tree (MST): the cheapest set of edges that connects every node together with no cycles — think "wire up all three sites for the least total cable".

  • Three nodes A, B, C. The number on each edge is its weight (cost): A–B = 1, A–C = 2, B–C = 3.
  • Kruskal sorts edges cheapest first and adds each one unless it would create a loop. It adds A–B (1) ✓, then A–C (2) ✓ — now all three nodes are connected.
  • It then tries B–C (3) but skips it (red ✗): B and C are already connected through A, so adding this edge would just form a cycle and waste cost.
  • Answering "are these two nodes already connected?" fast is the job of union-find, which groups connected nodes into sets. Total weight of the final tree here is 1 + 2 = 3.

In short: MST rule: go cheapest-edge-first, and only keep an edge if it joins two separate groups. An edge inside a group you've already connected is a cycle — skip it.

Try it — union-find + Kruskal's MST
pythonclass UnionFind:
    def __init__(self, nodes):
        self.parent = {n: n for n in nodes}   # each node its own set
    def find(self, x):
        while self.parent[x] != x:
            self.parent[x] = self.parent[self.parent[x]]  # path compression
            x = self.parent[x]
        return x
    def union(self, a, b):
        ra, rb = self.find(a), self.find(b)
        if ra == rb:
            return False                # already connected -> would make a cycle
        self.parent[ra] = rb
        return True

def kruskal(nodes, edges):            # edges: list of (w, u, v)
    uf = UnionFind(nodes)
    mst, total = [], 0
    for w, u, v in sorted(edges):     # cheapest edges first
        if uf.union(u, v):             # add only if it connects two sets
            mst.append((u, v, w)); total += w
    return mst, total

edges = [(1,"A","B"), (3,"B","C"), (2,"A","C")]
print(kruskal(["A","B","C"], edges))   # ([('A','B',1),('A','C',2)], 3)
▶ How this works

This implements union-find (the connectivity tracker) and then Kruskal's MST on top of it. Union-find answers one question very fast: "are these two nodes already in the same connected group?"

  1. self.parent = {n: n for n in nodes} starts with every node in its own group (each node points to itself as its group's representative).
  2. find(x) follows parent links up to the group's root (the node that points to itself). The line self.parent[x] = self.parent[self.parent[x]] is path compression — it flattens the chain as it climbs, making future look-ups nearly instant.
  3. union(a, b) finds both roots. If they're the same, a and b are already connected, so it returns False (joining them would make a cycle). Otherwise it links one root under the other and returns True.
  4. kruskal loops edges via sorted(edges) (cheapest first, since each edge is stored as (w, u, v) — weight leads). It keeps an edge only when uf.union(u, v) returns True, i.e. it connected two separate groups.

What the output means: A pair: the list of chosen MST edges and their total weight, e.g. ([('A','B',1), ('A','C',2)], 3) — the cheapest cycle-free way to connect all nodes. The B–C edge (3) is skipped because it would form a cycle.

Try this: Change A–C from weight 2 to 4. Now B–C (3) is cheaper, so Kruskal picks a different pair of edges — but always the two cheapest that keep the graph cycle-free.

Real tools do this for you — networkxYou rarely hand-code these in production: the networkx library gives shortest_path, topological_sort, minimum_spanning_tree, and cycle detection out of the box. But knowing the algorithms tells you their cost and limits — e.g. why a 100k-node knowledge-graph query is fine but an all-pairs shortest path on it is not.

Exercises expert

Practice
  1. Count the number of connected components in an undirected graph (DFS from each unvisited node).
  2. Detect a cycle in a directed graph using DFS with three colors (white/gray/black).
  3. Modify Dijkstra to also return the actual shortest path, not just distances (store predecessors).
  4. Use BFS to find whether a graph is bipartite (2-colorable).
  5. Model a small "service depends on service" graph and topologically order a safe restart sequence.

🎯 Interview practice interview

The interview questions this topic gets asked — worked, with code. For the full pattern catalog see D8 · Big Tech DSA patterns.

Number of islands (classic) — grid DFS

Scan cells; each unvisited '1' starts a flood-fill that sinks its whole island. Count the floods.

pythondef num_islands(grid):
    rows, cols = len(grid), len(grid[0])
    def sink(r, c):
        if 0 <= r < rows and 0 <= c < cols and grid[r][c] == "1":
            grid[r][c] = "0"
            sink(r+1,c); sink(r-1,c); sink(r,c+1); sink(r,c-1)
    n = 0
    for r in range(rows):
        for c in range(cols):
            if grid[r][c] == "1":
                n += 1; sink(r, c)
    return n
▶ How this works

This is the classic interview problem "Number of Islands". You get a grid of "1" (land) and "0" (water); count how many separate islands of connected land there are. A grid is secretly a graph — each cell is a node, and its up/down/left/right neighbours are its edges.

  1. rows, cols = len(grid), len(grid[0]) reads the grid's size. We'll scan every cell once, left-to-right, top-to-bottom.
  2. sink(r, c) is a DFS "flood fill": if the cell is on the grid and is land ("1"), it turns it to water ("0") so it's never counted again, then recursively sinks its four neighbours — spreading across the whole connected island.
  3. The double for loop walks every cell. Each time it hits a "1" that hasn't been sunk yet, that's a new island: it does n += 1 and then sink(r, c) erases the entire island so its other cells won't be re-counted.
  4. Because sink flattens each island the moment it's found, every remaining "1" the scan meets must belong to a fresh island — so the count is exact.

What the output means: An integer — the number of separate land islands. For a grid with two disconnected land blobs it returns 2.

Try this: This is just DFS run from each unvisited node, which is also how you count "connected components" in any graph (exercise 1 above). Grids are graphs in disguise.

Course schedule (classic) — cycle detection via topo sort

Kahn's algorithm: repeatedly remove in-degree-0 nodes. If any remain, there's a cycle.

pythonfrom collections import deque
def can_finish(n, prereqs):
    adj = {i: [] for i in range(n)}; indeg = [0]*n
    for a, b in prereqs:
        adj[b].append(a); indeg[a] += 1
    q = deque(i for i in range(n) if indeg[i]==0); done = 0
    while q:
        node = q.popleft(); done += 1
        for nxt in adj[node]:
            indeg[nxt] -= 1
            if indeg[nxt]==0: q.append(nxt)
    return done == n
▶ How this works

This is "Course Schedule": given courses and prerequisite pairs, can you finish them all? You can — unless the prerequisites form a cycle (course X needs Y, Y needs X). It's topological sort (Kahn's algorithm) used purely as a cycle detector.

  1. Each prereqs pair [a, b] means "must take b before a". adj[b].append(a) records the edge b → a, and indeg[a] += 1 counts that a now has one more prerequisite.
  2. q = deque(i ... if indeg[i]==0) starts with every course that has no prerequisites — the ones you can take right away.
  3. The loop takes a ready course, counts it done (done += 1), and for each course that depended on it lowers indeg. When a dependent's count hits 0, its prerequisites are all cleared, so it's enqueued.
  4. return done == n — if we managed to "finish" all n courses, there was no cycle (True). If some courses were never freed, they're locked in a prerequisite cycle, so it's impossible (False).

What the output means: True if all courses are completable, False if a circular prerequisite makes it impossible.

Try this: Feed it a cycle like [[0,1],[1,0]] (0 needs 1, 1 needs 0). Neither ever reaches in-degree 0, done stalls below n, and it correctly returns False.

Checkpoint expert

  • Choose an adjacency list vs matrix and justify it by density.
  • Implement BFS and DFS, and say which finds shortest unweighted paths.
  • Topologically order a DAG and detect a cycle.
  • Pick Dijkstra vs Bellman-Ford by whether edges can be negative, and explain union-find's role in Kruskal.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Build an adjacency listBeginner

Context: How you represent a graph decides which algorithms are cheap. Adjacency lists are the default for the sparse graphs that dominate real systems.

Your task: Represent an undirected graph as an adjacency list (dict of neighbour lists), list each node's neighbours, and note why adjacency lists beat matrices for sparse graphs.

Requirements:

  • Build a dict mapping each node to a list of its neighbours
  • Undirected edges appear in both endpoints' lists
  • Enumerate every node's neighbours
  • State that a list uses O(V+E) space vs a matrix's O(V²), which wins when edges are few

💡 Hint: A defaultdict(list) makes edge insertion a two-line append; the space argument is just E-versus-V² for sparse graphs.

Show solution
from collections import defaultdict

def build(edges):
    g = defaultdict(list)
    for u, v in edges:
        g[u].append(v); g[v].append(u)   # undirected -> both directions
    return g

g = build([(1, 2), (1, 3), (2, 4)])
for node in sorted(g):
    print(node, "->", g[node])
# 1 -> [2, 3] ; 2 -> [1, 4] ; 3 -> [1] ; 4 -> [2]

Adjacency list is O(V+E) space; a matrix is O(V^2) and wasteful when edges are few (sparse).

Exercise 2 · BFS shortest path (unweighted)Intermediate

Context: BFS is the shortest-path algorithm for unweighted graphs, and knowing exactly when that guarantee holds is the real lesson.

Your task: Find the fewest hops between two nodes with BFS in O(V+E), and note that BFS gives shortest paths only when all edges weigh the same.

Requirements:

  • Explore level by level using a queue
  • Track visited nodes to avoid re-processing
  • Return the hop count (or path) to the target
  • O(V+E)
  • State the caveat: correct only for uniform edge weights

💡 Hint: The first time BFS reaches a node it has done so in the fewest hops; a visited-set plus a FIFO queue is the whole machine.

Show solution
from collections import deque

def bfs_dist(graph, start, goal):      # O(V+E)
    q = deque([(start, 0)]); seen = {start}
    while q:
        node, d = q.popleft()
        if node == goal:
            return d
        for nxt in graph[node]:
            if nxt not in seen:
                seen.add(nxt); q.append((nxt, d + 1))
    return -1

g = {1: [2, 3], 2: [4], 3: [4], 4: []}
print(bfs_dist(g, 1, 4))               # 2  (1->2->4)

BFS explores in rings of increasing distance, so the first time it reaches the goal is via a shortest path.

Exercise 3 · DFS + cycle detectionAdvanced

Context: Cycle detection in a directed graph with three-colour DFS is the basis of deadlock detection and build-order validation.

Your task: Detect a cycle in a directed graph using DFS with white/gray/black colouring, where a back-edge to a gray node signals a cycle, in O(V+E).

Requirements:

  • White = unvisited, gray = on the current DFS path, black = fully explored
  • Encountering a gray node during DFS means a cycle
  • Finishing a node marks it black
  • O(V+E)
  • Works on disconnected graphs (start DFS from every white node)

💡 Hint: The gray set is exactly the current recursion stack; an edge back into it is the back-edge that closes a cycle.

Show solution
def has_cycle(graph):                  # O(V+E)
    WHITE, GRAY, BLACK = 0, 1, 2
    color = {u: WHITE for u in graph}
    def dfs(u):
        color[u] = GRAY                # on the current recursion stack
        for v in graph[u]:
            if color[v] == GRAY:       # back-edge -> cycle
                return True
            if color[v] == WHITE and dfs(v):
                return True
        color[u] = BLACK               # fully explored
        return False
    return any(color[u] == WHITE and dfs(u) for u in graph)

print(has_cycle({1: [2], 2: [3], 3: [1]}))   # True
print(has_cycle({1: [2], 2: [3], 3: []}))    # False

Gray = "ancestor still being explored"; revisiting one means we looped back — the definition of a directed cycle.

Exercise 4 · Topological sort (Kahn)Expert

Context: Topological sort via Kahn's algorithm orders dependencies and detects impossible ones — the course-schedule / build-graph pattern.

Your task: Order tasks so every dependency precedes its dependents using Kahn's algorithm (repeatedly remove in-degree-0 nodes), which also detects cycles, in O(V+E).

Requirements:

  • Compute each node's in-degree
  • Repeatedly emit a node with in-degree 0 and decrement its successors
  • Produce a valid topological order
  • If some nodes never reach in-degree 0, report a cycle
  • O(V+E)

💡 Hint: Seed a queue with every in-degree-0 node; if you emit fewer nodes than exist, the leftover ones form a cycle.

Show solution
from collections import deque, defaultdict

def topo_sort(graph):                  # O(V+E)
    indeg = defaultdict(int)
    for u in graph:
        indeg.setdefault(u, 0)
        for v in graph[u]:
            indeg[v] += 1
    q = deque([u for u in indeg if indeg[u] == 0])
    order = []
    while q:
        u = q.popleft(); order.append(u)
        for v in graph[u]:
            indeg[v] -= 1
            if indeg[v] == 0:
                q.append(v)
    return order if len(order) == len(indeg) else None  # None = cycle

print(topo_sort({"a": ["b", "c"], "b": ["d"], "c": ["d"], "d": []}))
# ['a', 'b', 'c', 'd']

If fewer nodes come out than went in, some stayed stuck with a nonzero in-degree — a cycle, so no valid ordering exists.

Exercise 5 · Dijkstra shortest pathProfessional

Context: Dijkstra is the standard shortest-path algorithm for non-negative weighted graphs and the reason a priority queue is in every toolkit.

Your task: Find shortest paths from a source in a weighted graph with non-negative weights using a heap, in O((V+E) log V).

Requirements:

  • Use a min-heap keyed by tentative distance
  • Pop the closest unfinalised node and relax its edges
  • Skip stale heap entries whose distance is already beaten
  • Requires non-negative weights
  • O((V+E) log V)

💡 Hint: Always expand the nearest frontier node next; pushing improved distances and ignoring outdated pops keeps the heap correct.

Show solution
import heapq

def dijkstra(graph, src):              # graph: node -> list of (neighbor, weight)
    dist = {src: 0}
    pq = [(0, src)]                    # (distance, node)
    while pq:
        d, u = heapq.heappop(pq)
        if d > dist.get(u, float("inf")):
            continue                   # stale entry, skip
        for v, w in graph[u]:
            nd = d + w
            if nd < dist.get(v, float("inf")):
                dist[v] = nd
                heapq.heappush(pq, (nd, v))
    return dist

g = {"a": [("b", 1), ("c", 4)], "b": [("c", 2), ("d", 5)],
     "c": [("d", 1)], "d": []}
print(dijkstra(g, "a"))   # {'a':0,'b':1,'c':3,'d':4}

The heap always expands the closest unsettled node; because weights are non-negative, once popped a node's distance is final. Negative edges break this (use Bellman-Ford).

Exercise 6 · Number of islands (grid DFS)Industry scenario

Context: Number-of-islands (classic) is the grid flood-fill that appears in countless onsites — connected components on an implicit graph.

Your task: Count connected regions of land ('1') in a 2-D grid via flood-fill, in O(rows·cols).

Requirements:

  • Scan the grid; each unvisited land cell starts a new island
  • Flood-fill its 4-connected land neighbours (DFS or BFS)
  • Mark filled cells so they aren't counted twice
  • O(rows·cols)
  • Return the island count

💡 Hint: Treat each cell as a node with edges to its four orthogonal neighbours; every flood-fill you launch is one connected component.

Show solution
def num_islands(grid):                 # O(R*C)
    if not grid: return 0
    R, C = len(grid), len(grid[0])
    def sink(r, c):
        if 0 <= r < R and 0 <= c < C and grid[r][c] == "1":
            grid[r][c] = "0"           # mark visited in place
            sink(r+1, c); sink(r-1, c); sink(r, c+1); sink(r, c-1)
    count = 0
    for r in range(R):
        for c in range(C):
            if grid[r][c] == "1":
                count += 1; sink(r, c) # flood the whole island
    return count

grid = [list("11000"), list("11000"), list("00100"), list("00011")]
print(num_islands(grid))               # 3

Each unvisited land cell starts a flood-fill that sinks its entire connected component, so the outer loop counts components exactly once.

Knowledge check check yourself

✓ Knowledge check

Why does BFS find shortest paths on an unweighted graph while a plain DFS does not?

Show answer
BFS explores level by level from the source, so the first time it reaches a node it has used the fewest edges possible. DFS dives down one branch first and can reach a node by a longer path before a shorter one.
✓ Knowledge check

What does a topological sort produce, and what kind of graph must it run on?

Show answer
It produces a linear ordering of vertices such that every directed edge points from an earlier to a later vertex — an order that respects all dependencies. It requires a directed acyclic graph (DAG); a cycle makes no valid ordering exist.
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in