AI EngineeringZero to ProductionHome·About·Contact
Appendix · Data Structures & Algorithms · Part 9

Divide & conquer + Master theorem

Divide and conquer is the first great algorithm-design paradigm: split a problem into smaller copies of itself, solve those, and combine. Merge sort, quickselect, binary search, Karatsuba multiplication and the closest-pair algorithm are all the same idea. This lesson makes the recurrence relations concrete and proves their running times with the Master theorem.

⏱️ ~2.5 hours🎯 Advanced → Expert➗ recurrences & proofsrunnable

Learning objectives

  • State the three-step divide-and-conquer schema and write a recurrence for it.
  • Apply the Master theorem (all three cases) to solve T(n)=aT(n/b)+f(n).
  • Implement merge sort and prove its O(n log n) bound from its recurrence.
  • Implement quickselect and explain its O(n) average / O(n²) worst case.
  • Write correct binary-search variants (lower/upper bound) and avoid off-by-one bugs.
  • Understand why Karatsuba beats schoolbook multiplication (recurrence intuition).

1 · The paradigm and its recurrence advanced

A divide-and-conquer algorithm has three steps. Divide the input of size n into a subproblems, each of size n/b. Conquer each subproblem by recursion (a base case stops the recursion). Combine the sub-answers into the answer, doing f(n) work. The total cost obeys a recurrence relation:

T(n) = a·T(n/b) + f(n), with T(1) = Θ(1).

Everything in this lesson is a special case. Merge sort splits into a=2 halves (b=2) and merges in f(n)=Θ(n). Binary search recurses into a=1 half and does Θ(1) work. The whole game is: write the recurrence, then solve it.

Divide a subproblems of size n/b Conquer (recurse) solve each by recursion Combine merge in f(n) work
Why it worksRecursion is only correct when subproblems are strictly smaller and a base case is reachable. If n/b does not shrink toward the base case, the recursion never terminates. Every recurrence below has a base case at n=1.

2 · The Master theorem — all three cases expert

The Master theorem solves T(n)=a·T(n/b)+f(n) for constants a≥1, b>1. Compare f(n) against the watershed function n^(log_b a) — the cost of the leaves of the recursion tree. The bigger of {leaf work, combine work} dominates.

CaseConditionResultWho wins
1f(n) = O(n^(log_b a − ε))T(n) = Θ(n^(log_b a))leaves dominate
2f(n) = Θ(n^(log_b a))T(n) = Θ(n^(log_b a) · log n)tie — log factor
3f(n) = Ω(n^(log_b a + ε)) + regularityT(n) = Θ(f(n))root/combine dominates

Worked examples. Merge sort: a=2, b=2 so log_b a = log₂2 = 1, and f(n)=Θ(n)=Θ(n¹) — that is Case 2, giving Θ(n log n). Binary search: a=1, b=2, log₂1 = 0, f(n)=Θ(1)=Θ(n⁰)Case 2 again, Θ(log n). Naive matrix mult of two n×n via 8 half-size products: a=8, b=2, log₂8 = 3, f(n)=Θ(n²) which is smaller — Case 1, Θ(n³). Strassen shaves a to 7: log₂7 ≈ 2.807, still Case 1, Θ(n^2.807).

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Try it — check the Master theorem numerically
pythonimport math

def master_case(a, b, f_exp):
    """a, b as in T(n)=a T(n/b)+f(n); f_exp = exponent p in f(n)=Theta(n^p)."""
    watershed = math.log(a, b)              # log_b(a)
    if abs(f_exp - watershed) < 1e-9:
        return f"Case 2: T(n) = Theta(n^{watershed:.3f} * log n)"
    if f_exp < watershed:
        return f"Case 1: T(n) = Theta(n^{watershed:.3f})"
    return f"Case 3: T(n) = Theta(n^{f_exp:.3f})"

print("merge sort   ", master_case(2, 2, 1))    # a=2,b=2,f=n^1
print("binary search", master_case(1, 2, 0))    # a=1,b=2,f=n^0
print("naive matmul ", master_case(8, 2, 2))    # a=8,b=2,f=n^2
print("Strassen     ", master_case(7, 2, 2))    # a=7,b=2,f=n^2
print("Karatsuba    ", master_case(3, 2, 1))    # a=3,b=2,f=n^1
merge sort    Case 2: T(n) = Theta(n^1.000 * log n)
binary search Case 2: T(n) = Theta(n^0.000 * log n)
naive matmul  Case 1: T(n) = Theta(n^3.000)
Strassen      Case 1: T(n) = Theta(n^2.807)
Karatsuba     Case 1: T(n) = Theta(n^1.585)
The gap the theorem cannot seeThe Master theorem fails when f(n) sits between cases without a polynomial gap (e.g. f(n)=n log n against n^(log_b a)=n). For those, use the recursion-tree method or the Akra–Bazzi generalization.

3 · Merge sort — D&C you can prove advanced

Merge sort splits the array in half, sorts each half recursively, and merges two sorted halves in linear time. The merge is the combine step; its linearity is what pins f(n)=Θ(n) and hence Θ(n log n).

Try it — merge sort with a merge that proves the bound
pythondef merge(left, right):
    out, i, j = [], 0, 0
    while i < len(left) and j < len(right):     # each comparison consumes one element
        if left[i] <= right[j]:
            out.append(left[i]); i += 1
        else:
            out.append(right[j]); j += 1
    out.extend(left[i:]); out.extend(right[j:])
    return out

def merge_sort(a):
    if len(a) <= 1:                             # base case: T(1)=O(1)
        return a
    mid = len(a) // 2
    left = merge_sort(a[:mid])                  # a=2 subproblems
    right = merge_sort(a[mid:])                 # ...each of size n/2
    return merge(left, right)                   # combine in Theta(n)

data = [5, 2, 9, 1, 7, 3, 8, 4, 6, 0]
print(merge_sort(data))
print("stable + correct:", merge_sort(data) == sorted(data))
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
stable + correct: True

Proof sketch. T(n)=2T(n/2)+cn. Expand the recursion tree: level i has 2^i nodes each doing c·n/2^i work — so every level costs cn. There are log₂n + 1 levels, giving cn·(log₂n+1)=Θ(n log n). The merge's linearity is the load-bearing fact: each of the n elements is copied once per level.

4 · Quickselect — expected linear time advanced

To find the k-th smallest element you do not need to sort. Quickselect partitions around a pivot (like quicksort) but recurses into only one side — the side containing rank k. Because it discards half the work on average, the recurrence is T(n)=T(n/2)+Θ(n), which sums to Θ(n) (a geometric series, not n log n).

Try it — quickselect (random pivot for good average case)
pythonimport random

def quickselect(a, k):
    """Return the k-th smallest (0-indexed) element of list a."""
    if len(a) == 1:
        return a[0]
    pivot = random.choice(a)
    lo = [x for x in a if x < pivot]
    eq = [x for x in a if x == pivot]
    hi = [x for x in a if x > pivot]
    if k < len(lo):
        return quickselect(lo, k)               # recurse ONE side only
    if k < len(lo) + len(eq):
        return pivot                            # k lands in the pivot block
    return quickselect(hi, k - len(lo) - len(eq))

random.seed(7)
data = [5, 2, 9, 1, 7, 3, 8, 4, 6, 0]
got = [quickselect(data[:], k) for k in range(len(data))]
print("selected in rank order:", got)
print("matches sorted:", got == sorted(data))
selected in rank order: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
matches sorted: True
AlgorithmRecurrenceAverageWorst
merge sort2T(n/2)+Θ(n)Θ(n log n)Θ(n log n)
quickselectT(n/2)+Θ(n) avgΘ(n)Θ(n²) — adversarial pivots
quicksort2T(n/2)+Θ(n) avgΘ(n log n)Θ(n²)
Random pivots kill the worst case in practiceThe Θ(n²) worst case needs an adversary who always picks the min/max as pivot. A random pivot makes that vanishingly unlikely; the deterministic median-of-medians pivot guarantees Θ(n) worst-case but with a worse constant.

5 · Binary search variants — the off-by-one minefield intermediate

Binary search is D&C with a=1: halve the search space each step, Θ(log n). The subtlety is not the idea but the boundaries. Two workhorse variants: lower_bound (first index with a[i] ≥ target) and upper_bound (first index with a[i] > target). Together they give the range of equal elements and an insertion point.

Try it — lower_bound / upper_bound (half-open [lo, hi))
pythondef lower_bound(a, target):
    lo, hi = 0, len(a)                          # half-open interval [lo, hi)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] < target:
            lo = mid + 1                        # mid too small -> discard left half
        else:
            hi = mid                            # keep mid as a candidate
    return lo

def upper_bound(a, target):
    lo, hi = 0, len(a)
    while lo < hi:
        mid = (lo + hi) // 2
        if a[mid] <= target:
            lo = mid + 1
        else:
            hi = mid
    return lo

a = [1, 2, 2, 2, 4, 6, 6, 9]
lb, ub = lower_bound(a, 2), upper_bound(a, 2)
print("first >= 2 at index", lb, "; first > 2 at index", ub)
print("count of 2s:", ub - lb)
print("insertion point for 5:", lower_bound(a, 5))
first >= 2 at index 1 ; first > 2 at index 4
count of 2s: 3
insertion point for 5: 5
Two invariants keep it correctUse a half-open interval [lo, hi) and the loop condition lo < hi. Never write mid-1/mid+1 inconsistently: if mid could be the answer, move hi=mid (not mid-1). Python's bisect_left/bisect_right are exactly these two functions.

6 · Karatsuba & closest-pair — beating the obvious bound expert

Schoolbook multiplication of two n-digit numbers is Θ(n²). Karatsuba splits each number into high/low halves and observes that the three products it needs can be computed with three half-size multiplications instead of four (using (a+b)(c+d) = ac + bd + (ad+bc) to recover the cross term). That drops a from 4 to 3: T(n)=3T(n/2)+Θ(n) = Θ(n^log₂3) ≈ Θ(n^1.585).

Try it — Karatsuba multiplication (verified against int *)
pythondef karatsuba(x, y):
    if x < 10 or y < 10:                         # base case: single digit
        return x * y
    n = max(len(str(x)), len(str(y)))
    half = n // 2
    high_x, low_x = divmod(x, 10 ** half)
    high_y, low_y = divmod(y, 10 ** half)
    z0 = karatsuba(low_x, low_y)                 # bd
    z2 = karatsuba(high_x, high_y)               # ac
    z1 = karatsuba(low_x + high_x, low_y + high_y) - z2 - z0   # ad+bc via ONE mult
    return z2 * 10 ** (2 * half) + z1 * 10 ** half + z0

import random
random.seed(1)
ok = all(karatsuba(a, b) == a * b
         for a, b in ((random.randint(0, 10**12), random.randint(0, 10**12))
                      for _ in range(1000)))
print("1000 random products all correct:", ok)
print("example:", karatsuba(1234567, 89012345), "==", 1234567 * 89012345)
1000 random products all correct: True
example: 109891703729615 == 109891703729615

Closest pair of points is the geometric classic: sort by x, split at the median x, recursively find the closest pair in each half (distance δ), then check only points within a -wide strip around the split line. A packing argument shows each strip point compares against at most 7 others, so the combine is Θ(n) — giving T(n)=2T(n/2)+Θ(n)=Θ(n log n), beating the Θ(n²) all-pairs check.

The recurring trickEvery speedup here is the same move: find structure so the combine step is cheaper than the naive one. Karatsuba shrinks a; closest-pair caps the strip work. Beating a bound almost always means beating the combine, not the recursion shape.

Checkpoint expert

✓ Checkpoint — you can move on when you can…

  • Write the recurrence T(n)=aT(n/b)+f(n) for a given divide-and-conquer algorithm.
  • Pick the correct Master-theorem case and state the resulting Θ bound.
  • Explain why merge sort is Θ(n log n) using the recursion-tree per-level argument.
  • Explain why quickselect is Θ(n) average but Θ(n²) worst.
  • Implement lower_bound/upper_bound without an off-by-one error.
✓ Knowledge check

For T(n) = 2T(n/2) + Θ(n²), which Master-theorem case applies and what is the bound?

Show answer
log_b a = log₂2 = 1, so the watershed is . Here f(n)=Θ(n²) is polynomially larger (n^(1+ε) with ε=1) and satisfies the regularity condition, so it is Case 3: T(n)=Θ(n²) — the combine step dominates.
✓ Knowledge check

Why does quickselect recurse into only one side while quicksort recurses into both?

Show answer
Quicksort must fully order both partitions, so it recurses left and right — 2T(n/2). Quickselect only needs the element of a specific rank k; after partitioning it knows which side contains rank k and discards the other. One-sided recursion turns 2T(n/2)+Θ(n) into T(n/2)+Θ(n), whose sum is a geometric series bounded by Θ(n).

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Power via fast exponentiationBeginner

Context: Computing x^n by multiplying n times is Θ(n). Divide and conquer does it in Θ(log n) — the trick behind modular exponentiation in cryptography.

Your task: Write power(x, n) using the identity x^n = (x^(n/2))² (times an extra x when n is odd).

Requirements:

  • Handle n = 0 (return 1) and odd/even n.
  • Must be O(log n) multiplications, not O(n).
  • Verify against Python's x ** n for several inputs.

💡 Hint: Recurse on n // 2 once and square the result; don't recompute the half twice.

Show solution
Solution
pythondef power(x, n):
    if n == 0:
        return 1
    half = power(x, n // 2)          # ONE recursive call -> O(log n)
    half_sq = half * half
    return half_sq * x if n % 2 else half_sq

print([power(2, k) for k in range(11)])
print("2^10 =", power(2, 10), "check:", power(2, 10) == 2 ** 10)
print("3^13 =", power(3, 13), "check:", power(3, 13) == 3 ** 13)
[1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024]
2^10 = 1024 check: True
3^13 = 1594323 check: True
Exercise 2 · Count inversions while sortingIntermediate

Context: An inversion is a pair (i, j) with i<j but a[i]>a[j] — a measure of how unsorted an array is, used in rank correlation and collaborative filtering.

Your task: Modify merge sort to count inversions in Θ(n log n).

Requirements:

  • Return (sorted_list, inversion_count).
  • During merge, when you take from the right half, every remaining left element is an inversion.
  • Verify against the O(n²) brute-force count.

💡 Hint: The count is additive: inversions in left + inversions in right + cross-inversions found during merge.

Show solution
Solution
pythondef sort_count(a):
    if len(a) <= 1:
        return a, 0
    mid = len(a) // 2
    left, cl = sort_count(a[:mid])
    right, cr = sort_count(a[mid:])
    merged, i, j, cross = [], 0, 0, 0
    while i < len(left) and j < len(right):
        if left[i] <= right[j]:
            merged.append(left[i]); i += 1
        else:
            merged.append(right[j]); j += 1
            cross += len(left) - i         # all remaining left > right[j]
    merged.extend(left[i:]); merged.extend(right[j:])
    return merged, cl + cr + cross

def brute(a):
    return sum(1 for i in range(len(a)) for j in range(i + 1, len(a)) if a[i] > a[j])

data = [2, 4, 1, 3, 5, 0]
sorted_a, inv = sort_count(data)
print("sorted:", sorted_a, "inversions:", inv)
print("brute matches:", inv == brute(data))
sorted: [0, 1, 2, 3, 4, 5] inversions: 8
brute matches: True
Exercise 3 · Search a rotated sorted arrayAdvanced

Context: A sorted array rotated at an unknown pivot (e.g. [4,5,6,7,0,1,2]) is the classic 'binary search on a twisted invariant' interview question.

Your task: Find the index of target in Θ(log n), or -1 if absent.

Requirements:

  • No linear scan; must be O(log n).
  • At each step one half is guaranteed sorted — decide which, then whether target lies in it.
  • Handle duplicates-free input; verify on several rotations.

💡 Hint: Compare a[mid] with a[lo] to detect which half is sorted, then range-test the target.

Show solution
Solution
pythondef search_rotated(a, target):
    lo, hi = 0, len(a) - 1
    while lo <= hi:
        mid = (lo + hi) // 2
        if a[mid] == target:
            return mid
        if a[lo] <= a[mid]:                 # left half [lo..mid] is sorted
            if a[lo] <= target < a[mid]:
                hi = mid - 1
            else:
                lo = mid + 1
        else:                               # right half [mid..hi] is sorted
            if a[mid] < target <= a[hi]:
                lo = mid + 1
            else:
                hi = mid - 1
    return -1

arr = [4, 5, 6, 7, 0, 1, 2]
print([search_rotated(arr, t) for t in (0, 4, 2, 7, 3)])
# indices of 0,4,2,7 and -1 for missing 3
[4, 0, 6, 3, -1]
Exercise 4 · Maximum subarray by divide and conquerExpert

Context: The maximum-subarray problem (largest contiguous sum) has a famous O(n) DP solution (Kadane), but the divide-and-conquer version is the textbook D&C exercise and generalizes to 2-D.

Your task: Solve it with divide and conquer in Θ(n log n).

Requirements:

  • Split at the midpoint; the best subarray is entirely left, entirely right, or crosses the middle.
  • Compute the crossing max in Θ(n) by extending outward from the center.
  • Verify against Kadane's O(n) algorithm.

💡 Hint: The crossing sum = best suffix of the left half + best prefix of the right half.

Show solution
Solution
pythondef max_subarray(a):
    def solve(lo, hi):
        if lo == hi:
            return a[lo]
        mid = (lo + hi) // 2
        left = solve(lo, mid)
        right = solve(mid + 1, hi)
        # best suffix ending at mid
        s, best_l = 0, float("-inf")
        for i in range(mid, lo - 1, -1):
            s += a[i]; best_l = max(best_l, s)
        # best prefix starting at mid+1
        s, best_r = 0, float("-inf")
        for i in range(mid + 1, hi + 1):
            s += a[i]; best_r = max(best_r, s)
        return max(left, right, best_l + best_r)
    return solve(0, len(a) - 1)

def kadane(a):
    best = cur = a[0]
    for x in a[1:]:
        cur = max(x, cur + x); best = max(best, cur)
    return best

data = [-2, 1, -3, 4, -1, 2, 1, -5, 4]
print("D&C:", max_subarray(data), "Kadane:", kadane(data))
print("match:", max_subarray(data) == kadane(data))
D&C: 6 Kadane: 6
match: True
Exercise 5 · Median of two sorted arrays in O(log(m+n))Professional

Context: Merging two sorted arrays to find the median is O(m+n); the interview-hard version demands O(log(min(m,n))) via binary search on the partition — a real Big Tech screening question.

Your task: Find the median of two sorted arrays without merging them.

Requirements:

  • Binary-search the split point in the smaller array so the left halves have (m+n+1)//2 elements.
  • Correct partition: max(left) ≤ min(right) across both arrays.
  • Handle even/odd total length; verify against sorting the concatenation.

💡 Hint: Binary-search a cut i in A; the cut j in B is forced by the half-size requirement. Adjust i left/right by comparing the four boundary elements.

Show solution
Solution
pythondef median_two(a, b):
    if len(a) > len(b):
        a, b = b, a                          # ensure a is smaller
    m, n = len(a), len(b)
    lo, hi, half = 0, m, (m + n + 1) // 2
    while lo <= hi:
        i = (lo + hi) // 2                   # cut in a
        j = half - i                         # cut in b
        a_left = a[i - 1] if i > 0 else float("-inf")
        a_right = a[i] if i < m else float("inf")
        b_left = b[j - 1] if j > 0 else float("-inf")
        b_right = b[j] if j < n else float("inf")
        if a_left <= b_right and b_left <= a_right:
            if (m + n) % 2:
                return float(max(a_left, b_left))
            return (max(a_left, b_left) + min(a_right, b_right)) / 2
        if a_left > b_right:
            hi = i - 1
        else:
            lo = i + 1

import statistics
tests = [([1, 3], [2]), ([1, 2], [3, 4]), ([1, 5, 9], [2, 3, 4, 6, 7])]
for a, b in tests:
    got = median_two(a, b)
    exp = statistics.median(sorted(a + b))
    print(a, b, "-> median", got, "ok:", got == exp)
[1, 3] [2] -> median 2.0 ok: True
[1, 2] [3, 4] -> median 2.5 ok: True
[1, 5, 9] [2, 3, 4, 6, 7] -> median 4.5 ok: True
Exercise 6 · Skyline problem — divide and conquer over intervalsIndustry scenario

Context: The skyline problem (merge building silhouettes into the outline of a city) is used in rendering, computational geometry, and range-aggregation systems. Its D&C solution mirrors merge sort: split buildings, solve halves, merge two skylines.

Your task: Given buildings as (left, right, height), produce the skyline as a list of (x, height) key points.

Requirements:

  • Divide buildings into two halves, compute each skyline recursively.
  • Merge two skylines by sweeping x left-to-right, tracking the max of the two current heights.
  • Only emit a key point when the running max height actually changes.
  • Runs in Θ(n log n); verify the output outline on a small example.

💡 Hint: The merge is the crux: walk both skylines by x, keep h1/h2 as the current height of each, and emit (x, max(h1,h2)) whenever that max differs from the last emitted height.

Show solution
Solution
pythondef get_skyline(buildings):
    if not buildings:
        return []
    if len(buildings) == 1:
        l, r, h = buildings[0]
        return [(l, h), (r, 0)]
    mid = len(buildings) // 2
    left = get_skyline(buildings[:mid])
    right = get_skyline(buildings[mid:])
    return merge_skyline(left, right)

def merge_skyline(left, right):
    i = j = 0
    h1 = h2 = 0
    result = []
    while i < len(left) and j < len(right):
        if left[i][0] < right[j][0]:
            x, h1 = left[i]; i += 1
        elif left[i][0] > right[j][0]:
            x, h2 = right[j]; j += 1
        else:
            x, h1 = left[i]; h2 = right[j][1]; i += 1; j += 1
        cur = max(h1, h2)
        if not result or result[-1][1] != cur:
            result.append((x, cur))
    result.extend(left[i:])
    result.extend(right[j:])
    return result

buildings = [(2, 9, 10), (3, 7, 15), (5, 12, 12), (15, 20, 10), (19, 24, 8)]
print(get_skyline(buildings))
[(2, 10), (3, 15), (7, 12), (12, 0), (15, 10), (20, 8), (24, 0)]
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in