Divide & conquer + Master theorem
Divide and conquer is the first great algorithm-design paradigm: split a problem into smaller copies of itself, solve those, and combine. Merge sort, quickselect, binary search, Karatsuba multiplication and the closest-pair algorithm are all the same idea. This lesson makes the recurrence relations concrete and proves their running times with the Master theorem.
Learning objectives
- State the three-step divide-and-conquer schema and write a recurrence for it.
- Apply the Master theorem (all three cases) to solve T(n)=aT(n/b)+f(n).
- Implement merge sort and prove its O(n log n) bound from its recurrence.
- Implement quickselect and explain its O(n) average / O(n²) worst case.
- Write correct binary-search variants (lower/upper bound) and avoid off-by-one bugs.
- Understand why Karatsuba beats schoolbook multiplication (recurrence intuition).
1 · The paradigm and its recurrence advanced
A divide-and-conquer algorithm has three steps. Divide the input of size n into a subproblems, each of size n/b. Conquer each subproblem by recursion (a base case stops the recursion). Combine the sub-answers into the answer, doing f(n) work. The total cost obeys a recurrence relation:
T(n) = a·T(n/b) + f(n), with T(1) = Θ(1).
Everything in this lesson is a special case. Merge sort splits into a=2 halves (b=2) and merges in f(n)=Θ(n). Binary search recurses into a=1 half and does Θ(1) work. The whole game is: write the recurrence, then solve it.
n/b does not shrink toward the base case, the recursion never terminates. Every recurrence below has a base case at n=1.2 · The Master theorem — all three cases expert
The Master theorem solves T(n)=a·T(n/b)+f(n) for constants a≥1, b>1. Compare f(n) against the watershed function n^(log_b a) — the cost of the leaves of the recursion tree. The bigger of {leaf work, combine work} dominates.
| Case | Condition | Result | Who wins |
|---|---|---|---|
| 1 | f(n) = O(n^(log_b a − ε)) | T(n) = Θ(n^(log_b a)) | leaves dominate |
| 2 | f(n) = Θ(n^(log_b a)) | T(n) = Θ(n^(log_b a) · log n) | tie — log factor |
| 3 | f(n) = Ω(n^(log_b a + ε)) + regularity | T(n) = Θ(f(n)) | root/combine dominates |
Worked examples. Merge sort: a=2, b=2 so log_b a = log₂2 = 1, and f(n)=Θ(n)=Θ(n¹) — that is Case 2, giving Θ(n log n). Binary search: a=1, b=2, log₂1 = 0, f(n)=Θ(1)=Θ(n⁰) — Case 2 again, Θ(log n). Naive matrix mult of two n×n via 8 half-size products: a=8, b=2, log₂8 = 3, f(n)=Θ(n²) which is smaller — Case 1, Θ(n³). Strassen shaves a to 7: log₂7 ≈ 2.807, still Case 1, Θ(n^2.807).
pythonimport math
def master_case(a, b, f_exp):
"""a, b as in T(n)=a T(n/b)+f(n); f_exp = exponent p in f(n)=Theta(n^p)."""
watershed = math.log(a, b) # log_b(a)
if abs(f_exp - watershed) < 1e-9:
return f"Case 2: T(n) = Theta(n^{watershed:.3f} * log n)"
if f_exp < watershed:
return f"Case 1: T(n) = Theta(n^{watershed:.3f})"
return f"Case 3: T(n) = Theta(n^{f_exp:.3f})"
print("merge sort ", master_case(2, 2, 1)) # a=2,b=2,f=n^1
print("binary search", master_case(1, 2, 0)) # a=1,b=2,f=n^0
print("naive matmul ", master_case(8, 2, 2)) # a=8,b=2,f=n^2
print("Strassen ", master_case(7, 2, 2)) # a=7,b=2,f=n^2
print("Karatsuba ", master_case(3, 2, 1)) # a=3,b=2,f=n^1
merge sort Case 2: T(n) = Theta(n^1.000 * log n)
binary search Case 2: T(n) = Theta(n^0.000 * log n)
naive matmul Case 1: T(n) = Theta(n^3.000)
Strassen Case 1: T(n) = Theta(n^2.807)
Karatsuba Case 1: T(n) = Theta(n^1.585)
f(n) sits between cases without a polynomial gap (e.g. f(n)=n log n against n^(log_b a)=n). For those, use the recursion-tree method or the Akra–Bazzi generalization.3 · Merge sort — D&C you can prove advanced
Merge sort splits the array in half, sorts each half recursively, and merges two sorted halves in linear time. The merge is the combine step; its linearity is what pins f(n)=Θ(n) and hence Θ(n log n).
pythondef merge(left, right):
out, i, j = [], 0, 0
while i < len(left) and j < len(right): # each comparison consumes one element
if left[i] <= right[j]:
out.append(left[i]); i += 1
else:
out.append(right[j]); j += 1
out.extend(left[i:]); out.extend(right[j:])
return out
def merge_sort(a):
if len(a) <= 1: # base case: T(1)=O(1)
return a
mid = len(a) // 2
left = merge_sort(a[:mid]) # a=2 subproblems
right = merge_sort(a[mid:]) # ...each of size n/2
return merge(left, right) # combine in Theta(n)
data = [5, 2, 9, 1, 7, 3, 8, 4, 6, 0]
print(merge_sort(data))
print("stable + correct:", merge_sort(data) == sorted(data))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
stable + correct: True
Proof sketch. T(n)=2T(n/2)+cn. Expand the recursion tree: level i has 2^i nodes each doing c·n/2^i work — so every level costs cn. There are log₂n + 1 levels, giving cn·(log₂n+1)=Θ(n log n). The merge's linearity is the load-bearing fact: each of the n elements is copied once per level.
4 · Quickselect — expected linear time advanced
To find the k-th smallest element you do not need to sort. Quickselect partitions around a pivot (like quicksort) but recurses into only one side — the side containing rank k. Because it discards half the work on average, the recurrence is T(n)=T(n/2)+Θ(n), which sums to Θ(n) (a geometric series, not n log n).
pythonimport random
def quickselect(a, k):
"""Return the k-th smallest (0-indexed) element of list a."""
if len(a) == 1:
return a[0]
pivot = random.choice(a)
lo = [x for x in a if x < pivot]
eq = [x for x in a if x == pivot]
hi = [x for x in a if x > pivot]
if k < len(lo):
return quickselect(lo, k) # recurse ONE side only
if k < len(lo) + len(eq):
return pivot # k lands in the pivot block
return quickselect(hi, k - len(lo) - len(eq))
random.seed(7)
data = [5, 2, 9, 1, 7, 3, 8, 4, 6, 0]
got = [quickselect(data[:], k) for k in range(len(data))]
print("selected in rank order:", got)
print("matches sorted:", got == sorted(data))
selected in rank order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
matches sorted: True
| Algorithm | Recurrence | Average | Worst |
|---|---|---|---|
| merge sort | 2T(n/2)+Θ(n) | Θ(n log n) | Θ(n log n) |
| quickselect | T(n/2)+Θ(n) avg | Θ(n) | Θ(n²) — adversarial pivots |
| quicksort | 2T(n/2)+Θ(n) avg | Θ(n log n) | Θ(n²) |
Θ(n²) worst case needs an adversary who always picks the min/max as pivot. A random pivot makes that vanishingly unlikely; the deterministic median-of-medians pivot guarantees Θ(n) worst-case but with a worse constant.5 · Binary search variants — the off-by-one minefield intermediate
Binary search is D&C with a=1: halve the search space each step, Θ(log n). The subtlety is not the idea but the boundaries. Two workhorse variants: lower_bound (first index with a[i] ≥ target) and upper_bound (first index with a[i] > target). Together they give the range of equal elements and an insertion point.
pythondef lower_bound(a, target):
lo, hi = 0, len(a) # half-open interval [lo, hi)
while lo < hi:
mid = (lo + hi) // 2
if a[mid] < target:
lo = mid + 1 # mid too small -> discard left half
else:
hi = mid # keep mid as a candidate
return lo
def upper_bound(a, target):
lo, hi = 0, len(a)
while lo < hi:
mid = (lo + hi) // 2
if a[mid] <= target:
lo = mid + 1
else:
hi = mid
return lo
a = [1, 2, 2, 2, 4, 6, 6, 9]
lb, ub = lower_bound(a, 2), upper_bound(a, 2)
print("first >= 2 at index", lb, "; first > 2 at index", ub)
print("count of 2s:", ub - lb)
print("insertion point for 5:", lower_bound(a, 5))
first >= 2 at index 1 ; first > 2 at index 4
count of 2s: 3
insertion point for 5: 5
[lo, hi) and the loop condition lo < hi. Never write mid-1/mid+1 inconsistently: if mid could be the answer, move hi=mid (not mid-1). Python's bisect_left/bisect_right are exactly these two functions.6 · Karatsuba & closest-pair — beating the obvious bound expert
Schoolbook multiplication of two n-digit numbers is Θ(n²). Karatsuba splits each number into high/low halves and observes that the three products it needs can be computed with three half-size multiplications instead of four (using (a+b)(c+d) = ac + bd + (ad+bc) to recover the cross term). That drops a from 4 to 3: T(n)=3T(n/2)+Θ(n) = Θ(n^log₂3) ≈ Θ(n^1.585).
pythondef karatsuba(x, y):
if x < 10 or y < 10: # base case: single digit
return x * y
n = max(len(str(x)), len(str(y)))
half = n // 2
high_x, low_x = divmod(x, 10 ** half)
high_y, low_y = divmod(y, 10 ** half)
z0 = karatsuba(low_x, low_y) # bd
z2 = karatsuba(high_x, high_y) # ac
z1 = karatsuba(low_x + high_x, low_y + high_y) - z2 - z0 # ad+bc via ONE mult
return z2 * 10 ** (2 * half) + z1 * 10 ** half + z0
import random
random.seed(1)
ok = all(karatsuba(a, b) == a * b
for a, b in ((random.randint(0, 10**12), random.randint(0, 10**12))
for _ in range(1000)))
print("1000 random products all correct:", ok)
print("example:", karatsuba(1234567, 89012345), "==", 1234567 * 89012345)
1000 random products all correct: True
example: 109891703729615 == 109891703729615
Closest pair of points is the geometric classic: sort by x, split at the median x, recursively find the closest pair in each half (distance δ), then check only points within a 2δ-wide strip around the split line. A packing argument shows each strip point compares against at most 7 others, so the combine is Θ(n) — giving T(n)=2T(n/2)+Θ(n)=Θ(n log n), beating the Θ(n²) all-pairs check.
a; closest-pair caps the strip work. Beating a bound almost always means beating the combine, not the recursion shape.Checkpoint expert
✓ Checkpoint — you can move on when you can…
- Write the recurrence T(n)=aT(n/b)+f(n) for a given divide-and-conquer algorithm.
- Pick the correct Master-theorem case and state the resulting Θ bound.
- Explain why merge sort is Θ(n log n) using the recursion-tree per-level argument.
- Explain why quickselect is Θ(n) average but Θ(n²) worst.
- Implement lower_bound/upper_bound without an off-by-one error.
For T(n) = 2T(n/2) + Θ(n²), which Master-theorem case applies and what is the bound?
Show answer
log_b a = log₂2 = 1, so the watershed is n¹. Here f(n)=Θ(n²) is polynomially larger (n^(1+ε) with ε=1) and satisfies the regularity condition, so it is Case 3: T(n)=Θ(n²) — the combine step dominates.Why does quickselect recurse into only one side while quicksort recurses into both?
Show answer
2T(n/2). Quickselect only needs the element of a specific rank k; after partitioning it knows which side contains rank k and discards the other. One-sided recursion turns 2T(n/2)+Θ(n) into T(n/2)+Θ(n), whose sum is a geometric series bounded by Θ(n).🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Computing x^n by multiplying n times is Θ(n). Divide and conquer does it in Θ(log n) — the trick behind modular exponentiation in cryptography.
Your task: Write power(x, n) using the identity x^n = (x^(n/2))² (times an extra x when n is odd).
Requirements:
- Handle n = 0 (return 1) and odd/even n.
- Must be O(log n) multiplications, not O(n).
- Verify against Python's
x ** nfor several inputs.
💡 Hint: Recurse on n // 2 once and square the result; don't recompute the half twice.
Show solution
pythondef power(x, n):
if n == 0:
return 1
half = power(x, n // 2) # ONE recursive call -> O(log n)
half_sq = half * half
return half_sq * x if n % 2 else half_sq
print([power(2, k) for k in range(11)])
print("2^10 =", power(2, 10), "check:", power(2, 10) == 2 ** 10)
print("3^13 =", power(3, 13), "check:", power(3, 13) == 3 ** 13)
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
2^10 = 1024 check: True
3^13 = 1594323 check: True
Context: An inversion is a pair (i, j) with i<j but a[i]>a[j] — a measure of how unsorted an array is, used in rank correlation and collaborative filtering.
Your task: Modify merge sort to count inversions in Θ(n log n).
Requirements:
- Return (sorted_list, inversion_count).
- During merge, when you take from the right half, every remaining left element is an inversion.
- Verify against the O(n²) brute-force count.
💡 Hint: The count is additive: inversions in left + inversions in right + cross-inversions found during merge.
Show solution
pythondef sort_count(a):
if len(a) <= 1:
return a, 0
mid = len(a) // 2
left, cl = sort_count(a[:mid])
right, cr = sort_count(a[mid:])
merged, i, j, cross = [], 0, 0, 0
while i < len(left) and j < len(right):
if left[i] <= right[j]:
merged.append(left[i]); i += 1
else:
merged.append(right[j]); j += 1
cross += len(left) - i # all remaining left > right[j]
merged.extend(left[i:]); merged.extend(right[j:])
return merged, cl + cr + cross
def brute(a):
return sum(1 for i in range(len(a)) for j in range(i + 1, len(a)) if a[i] > a[j])
data = [2, 4, 1, 3, 5, 0]
sorted_a, inv = sort_count(data)
print("sorted:", sorted_a, "inversions:", inv)
print("brute matches:", inv == brute(data))
sorted: [0, 1, 2, 3, 4, 5] inversions: 8
brute matches: True
Context: A sorted array rotated at an unknown pivot (e.g. [4,5,6,7,0,1,2]) is the classic 'binary search on a twisted invariant' interview question.
Your task: Find the index of target in Θ(log n), or -1 if absent.
Requirements:
- No linear scan; must be O(log n).
- At each step one half is guaranteed sorted — decide which, then whether target lies in it.
- Handle duplicates-free input; verify on several rotations.
💡 Hint: Compare a[mid] with a[lo] to detect which half is sorted, then range-test the target.
Show solution
pythondef search_rotated(a, target):
lo, hi = 0, len(a) - 1
while lo <= hi:
mid = (lo + hi) // 2
if a[mid] == target:
return mid
if a[lo] <= a[mid]: # left half [lo..mid] is sorted
if a[lo] <= target < a[mid]:
hi = mid - 1
else:
lo = mid + 1
else: # right half [mid..hi] is sorted
if a[mid] < target <= a[hi]:
lo = mid + 1
else:
hi = mid - 1
return -1
arr = [4, 5, 6, 7, 0, 1, 2]
print([search_rotated(arr, t) for t in (0, 4, 2, 7, 3)])
# indices of 0,4,2,7 and -1 for missing 3
[4, 0, 6, 3, -1]
Context: The maximum-subarray problem (largest contiguous sum) has a famous O(n) DP solution (Kadane), but the divide-and-conquer version is the textbook D&C exercise and generalizes to 2-D.
Your task: Solve it with divide and conquer in Θ(n log n).
Requirements:
- Split at the midpoint; the best subarray is entirely left, entirely right, or crosses the middle.
- Compute the crossing max in Θ(n) by extending outward from the center.
- Verify against Kadane's O(n) algorithm.
💡 Hint: The crossing sum = best suffix of the left half + best prefix of the right half.
Show solution
pythondef max_subarray(a):
def solve(lo, hi):
if lo == hi:
return a[lo]
mid = (lo + hi) // 2
left = solve(lo, mid)
right = solve(mid + 1, hi)
# best suffix ending at mid
s, best_l = 0, float("-inf")
for i in range(mid, lo - 1, -1):
s += a[i]; best_l = max(best_l, s)
# best prefix starting at mid+1
s, best_r = 0, float("-inf")
for i in range(mid + 1, hi + 1):
s += a[i]; best_r = max(best_r, s)
return max(left, right, best_l + best_r)
return solve(0, len(a) - 1)
def kadane(a):
best = cur = a[0]
for x in a[1:]:
cur = max(x, cur + x); best = max(best, cur)
return best
data = [-2, 1, -3, 4, -1, 2, 1, -5, 4]
print("D&C:", max_subarray(data), "Kadane:", kadane(data))
print("match:", max_subarray(data) == kadane(data))
D&C: 6 Kadane: 6
match: True
Context: Merging two sorted arrays to find the median is O(m+n); the interview-hard version demands O(log(min(m,n))) via binary search on the partition — a real Big Tech screening question.
Your task: Find the median of two sorted arrays without merging them.
Requirements:
- Binary-search the split point in the smaller array so the left halves have (m+n+1)//2 elements.
- Correct partition: max(left) ≤ min(right) across both arrays.
- Handle even/odd total length; verify against sorting the concatenation.
💡 Hint: Binary-search a cut i in A; the cut j in B is forced by the half-size requirement. Adjust i left/right by comparing the four boundary elements.
Show solution
pythondef median_two(a, b):
if len(a) > len(b):
a, b = b, a # ensure a is smaller
m, n = len(a), len(b)
lo, hi, half = 0, m, (m + n + 1) // 2
while lo <= hi:
i = (lo + hi) // 2 # cut in a
j = half - i # cut in b
a_left = a[i - 1] if i > 0 else float("-inf")
a_right = a[i] if i < m else float("inf")
b_left = b[j - 1] if j > 0 else float("-inf")
b_right = b[j] if j < n else float("inf")
if a_left <= b_right and b_left <= a_right:
if (m + n) % 2:
return float(max(a_left, b_left))
return (max(a_left, b_left) + min(a_right, b_right)) / 2
if a_left > b_right:
hi = i - 1
else:
lo = i + 1
import statistics
tests = [([1, 3], [2]), ([1, 2], [3, 4]), ([1, 5, 9], [2, 3, 4, 6, 7])]
for a, b in tests:
got = median_two(a, b)
exp = statistics.median(sorted(a + b))
print(a, b, "-> median", got, "ok:", got == exp)
[1, 3] [2] -> median 2.0 ok: True
[1, 2] [3, 4] -> median 2.5 ok: True
[1, 5, 9] [2, 3, 4, 6, 7] -> median 4.5 ok: True
Context: The skyline problem (merge building silhouettes into the outline of a city) is used in rendering, computational geometry, and range-aggregation systems. Its D&C solution mirrors merge sort: split buildings, solve halves, merge two skylines.
Your task: Given buildings as (left, right, height), produce the skyline as a list of (x, height) key points.
Requirements:
- Divide buildings into two halves, compute each skyline recursively.
- Merge two skylines by sweeping x left-to-right, tracking the max of the two current heights.
- Only emit a key point when the running max height actually changes.
- Runs in Θ(n log n); verify the output outline on a small example.
💡 Hint: The merge is the crux: walk both skylines by x, keep h1/h2 as the current height of each, and emit (x, max(h1,h2)) whenever that max differs from the last emitted height.
Show solution
pythondef get_skyline(buildings):
if not buildings:
return []
if len(buildings) == 1:
l, r, h = buildings[0]
return [(l, h), (r, 0)]
mid = len(buildings) // 2
left = get_skyline(buildings[:mid])
right = get_skyline(buildings[mid:])
return merge_skyline(left, right)
def merge_skyline(left, right):
i = j = 0
h1 = h2 = 0
result = []
while i < len(left) and j < len(right):
if left[i][0] < right[j][0]:
x, h1 = left[i]; i += 1
elif left[i][0] > right[j][0]:
x, h2 = right[j]; j += 1
else:
x, h1 = left[i]; h2 = right[j][1]; i += 1; j += 1
cur = max(h1, h2)
if not result or result[-1][1] != cur:
result.append((x, cur))
result.extend(left[i:])
result.extend(right[j:])
return result
buildings = [(2, 9, 10), (3, 7, 15), (5, 12, 12), (15, 20, 10), (19, 24, 8)]
print(get_skyline(buildings))
[(2, 10), (3, 15), (7, 12), (12, 0), (15, 10), (20, 8), (24, 0)]