AI EngineeringZero to ProductionHome·About·Contact
Appendix · Data Structures & Algorithms · Part 12

Greedy algorithms

A greedy algorithm builds a solution by repeatedly taking the locally best choice and never looking back. When that is provably optimal it is the fastest paradigm of all — but it is correct only when the problem has the greedy-choice property. This lesson proves correctness with exchange arguments, works the classics (activity selection, interval scheduling, Huffman coding, fractional knapsack), and shows exactly where greedy fails and DP is required.

⏱️ ~2.5 hours🎯 Advanced → Expert✂️ proofs & counterexamplesrunnable

Learning objectives

  • State the greedy-choice property and optimal substructure — the two proof obligations.
  • Prove a greedy algorithm optimal with an exchange (swap) argument.
  • Solve activity selection / interval scheduling and justify the 'earliest finish' rule.
  • Build a Huffman code and explain why merging the two rarest symbols is optimal.
  • Solve fractional knapsack greedily and contrast with 0/1 (which needs DP).
  • Recognize when greedy fails and produce a concrete counterexample.

1 · The greedy-choice property advanced

A greedy algorithm is correct only if two things hold. Optimal substructure (shared with DP): an optimal solution contains optimal solutions to subproblems. Greedy-choice property: a globally optimal solution can be reached by making the locally optimal choice at each step — you never need to reconsider. DP tries all choices; greedy proves that one choice is always safe, so it skips the search entirely.

Make greedy choice locally optimal Reduce to 1 subproblem the rest of the input Repeat no backtracking
How to prove a greedy algorithm (exchange argument)Assume an optimal solution OPT that differs from the greedy one. Find the first place they diverge and swap OPT's choice for greedy's. Show the swap does not make OPT worse (and keeps it feasible). Repeating the swap turns OPT into the greedy solution without losing optimality — so greedy is optimal too. This 'greedy stays ahead / exchange' template proves almost every result below.

2 · Activity selection — earliest finish wins intermediate

Given activities with start/finish times, pick the largest set that don't overlap. The greedy rule: always take the activity that finishes earliest among those still compatible. Sorting by finish time and sweeping is Θ(n log n).

Exchange proof. Let greedy pick activity g (earliest finish) and suppose some optimal set OPT picks a different first activity o. Since g finishes no later than o, replacing o with g in OPT leaves all later activities still compatible (they started after o finished, hence after g finished). The swapped set is the same size and still valid — so greedy's first choice is safe. Induct on the remainder.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Try it — activity selection by earliest finish
pythondef select_activities(activities):
    """activities: list of (start, finish). Return a max non-overlapping subset."""
    chosen = []
    last_finish = float("-inf")
    for start, finish in sorted(activities, key=lambda a: a[1]):  # earliest finish first
        if start >= last_finish:            # compatible with the last chosen
            chosen.append((start, finish))
            last_finish = finish
    return chosen

acts = [(1, 4), (3, 5), (0, 6), (5, 7), (3, 9), (5, 9), (6, 10), (8, 11), (8, 12), (2, 14), (12, 16)]
picked = select_activities(acts)
print("selected", len(picked), "activities:", picked)
selected 4 activities: [(1, 4), (5, 7), (8, 11), (12, 16)]
Sort by finish, not start or durationSorting by earliest start or shortest duration both give wrong answers on simple inputs (a long early-finishing job vs two short late ones). Only 'earliest finish' has an exchange-argument proof. The choice of greedy criterion is the whole ballgame.

3 · Interval partitioning — minimum resources advanced

A dual problem: schedule all intervals using the fewest resources (rooms, machines) so overlapping intervals get different resources. The greedy: sort by start time; assign each interval to any free resource, opening a new one only when none is free. The number of resources equals the maximum overlap ('depth') — and you cannot beat the depth, so greedy is optimal.

Try it — minimum rooms via a min-heap of end times
pythonimport heapq

def min_rooms(intervals):
    if not intervals:
        return 0
    intervals.sort(key=lambda x: x[0])      # by start time
    heap = []                               # end times of rooms in use (min at top)
    for start, end in intervals:
        if heap and heap[0] <= start:
            heapq.heappop(heap)             # a room freed up -> reuse it
        heapq.heappush(heap, end)
    return len(heap)                        # rooms never freed = peak overlap

meetings = [(0, 30), (5, 10), (15, 20)]
print("rooms needed:", min_rooms(meetings))            # 2
print("rooms:", min_rooms([(1, 5), (2, 6), (3, 7), (8, 9)]))  # 3
rooms needed: 2
rooms: 3

4 · Huffman coding — optimal prefix codes expert

Huffman builds an optimal prefix code (no codeword is a prefix of another) that minimizes total encoded length given symbol frequencies. The greedy: repeatedly merge the two lowest-frequency nodes into a parent whose frequency is their sum, until one tree remains. Rarer symbols end up deeper (longer codes). Θ(n log n) with a heap. It is the backbone of DEFLATE/gzip and JPEG.

Why merging the two rarest is safe. In an optimal tree the two deepest leaves are siblings (else move one up and shorten the code). An exchange argument shows the two lowest-frequency symbols can be made those deepest siblings without increasing cost — so greedily pairing them is optimal, and the rest follows by induction on the merged 'super-symbol'.

Try it — build a Huffman code and check it compresses
pythonimport heapq
from collections import Counter

def huffman_codes(text):
    freq = Counter(text)
    if len(freq) == 1:                      # single-symbol edge case
        return {next(iter(freq)): "0"}
    # nodes as (freq, tie_id, node); node is a symbol str or a (left, right) tuple
    nodes = [(f, i, sym) for i, (sym, f) in enumerate(freq.items())]
    heapq.heapify(nodes)
    counter = len(nodes)
    while len(nodes) > 1:
        f1, _, n1 = heapq.heappop(nodes)    # two rarest
        f2, _, n2 = heapq.heappop(nodes)
        heapq.heappush(nodes, (f1 + f2, counter, (n1, n2)))
        counter += 1
    root = nodes[0][2]

    codes = {}
    def walk(node, prefix):  # noqa
        if isinstance(node, str):
            codes[node] = prefix or "0"
        else:
            walk(node[0], prefix + "0")
            walk(node[1], prefix + "1")
    walk(root, "")
    return codes

text = "abracadabra"
codes = huffman_codes(text)
encoded_bits = sum(len(codes[c]) for c in text)
print("codes:", {k: codes[k] for k in sorted(codes)})
print("Huffman bits:", encoded_bits, "vs fixed 3-bit:", 3 * len(text))
codes: {'a': '0', 'b': '110', 'c': '100', 'd': '101', 'r': '111'}
Huffman bits: 23 vs fixed 3-bit: 33

5 · Fractional knapsack — where greedy beats DP advanced

Unlike 0/1 knapsack, in the fractional version you may take a fraction of an item. Now greedy is optimal: sort by value-per-weight and take as much of the densest item as fits, breaking the last item fractionally. Θ(n log n). The fractionality is what unlocks the exchange argument — any suboptimal packing can be improved by swapping in a denser fraction.

Try it — fractional knapsack
pythondef fractional_knapsack(items, W):
    """items: list of (value, weight). Return max value for capacity W."""
    items = sorted(items, key=lambda it: it[0] / it[1], reverse=True)  # densest first
    total = 0.0
    for value, weight in items:
        if W <= 0:
            break
        take = min(weight, W)               # take a fraction of the last item
        total += value * (take / weight)
        W -= take
    return round(total, 2)

items = [(60, 10), (100, 20), (120, 30)]    # (value, weight)
print("fractional, W=50:", fractional_knapsack(items, 50))   # 240.0
print("fractional, W=25:", fractional_knapsack(items, 25))
fractional, W=50: 240.0
fractional, W=25: 135.0

6 · When greedy FAILS — and DP saves you expert

The most important skill is knowing when not to be greedy. If a locally optimal choice can foreclose a better global solution, greedy is wrong and you need DP (try all choices, keep the best). Two canonical failures: 0/1 knapsack (taking the densest item first can waste capacity) and min-coin change with non-canonical denominations.

Try it — greedy vs DP on 0/1 knapsack (greedy is wrong)
pythondef greedy_01(items, W):
    items = sorted(items, key=lambda it: it[0] / it[1], reverse=True)
    total, w = 0, 0
    for value, weight in items:
        if w + weight <= W:                 # 0/1: take whole item or none
            total += value; w += weight
    return total

def dp_01(items, W):
    dp = [0] * (W + 1)
    for value, weight in items:
        for cap in range(W, weight - 1, -1):
            dp[cap] = max(dp[cap], value + dp[cap - weight])
    return dp[W]

items = [(60, 10), (100, 20), (120, 30)]    # (value, weight)
W = 50
print("greedy 0/1:", greedy_01(items, W), " DP 0/1:", dp_01(items, W))
greedy 0/1: 160  DP 0/1: 220
The tell-tale signGreedy fails when a choice is irrevocable but interacts with future choices — committing to the densest 0/1 item (value 60, weight 10) blocks a better pair (100+120 for weight 50). Whenever a locally best move can leave you unable to reach the global best, reach for DP. Proving the greedy-choice property is how you rule this out before shipping.
ProblemGreedy correct?Why
fractional knapsackyesfractions let you always improve by density swap
0/1 knapsacknoindivisible items — needs DP (Θ(nW))
activity selectionyesearliest-finish exchange argument
min coins, canonicalyese.g. standard currency
min coins, arbitrarynoe.g. {1,3,4} for 6 — needs DP

Checkpoint expert

✓ Checkpoint — you can move on when you can…

  • State the greedy-choice property and give the exchange-argument proof template.
  • Justify why activity selection sorts by finish time (not start or duration).
  • Explain why Huffman merges the two lowest-frequency nodes.
  • Contrast fractional knapsack (greedy) with 0/1 knapsack (DP) and say why.
  • Produce a counterexample where greedy min-coin change is suboptimal.
✓ Knowledge check

Sketch the exchange argument that 'earliest finish time' is optimal for activity selection.

Show answer
Take any optimal solution OPT and let g be greedy's first pick (earliest finish). If OPT's first activity o differs, then since g finishes no later than o, swapping og keeps every later OPT activity compatible (each started after o finished ≥ after g finished). The swapped set has the same size and is still valid, so greedy's choice is safe. Recurse on the activities that start after g finishes.
✓ Knowledge check

Your teammate insists greedy 'take the highest value-per-weight item' solves 0/1 knapsack because it works for fractional. Give the one-line reason they are wrong.

Show answer
In 0/1 you cannot take a fraction of the last item, so the densest item can consume capacity that would have been better spent on a combination — e.g. items (60,10),(100,20),(120,30) with W=50: greedy takes 60+100=160 but the optimum is 100+120=220. Indivisibility breaks the density exchange argument, so 0/1 needs DP.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Assign cookies to greedy childrenBeginner

Context: A gentle greedy: match cookies to children so as many children as possible are content — the pattern of 'sort both sides, sweep with two pointers'.

Your task: Each child i has greed g[i] (min cookie size to be content); each cookie has size s[j]. Maximize the number of content children (one cookie per child).

Requirements:

  • Sort both greed and cookie lists.
  • Give the smallest adequate cookie to the least greedy remaining child.
  • Return how many children are satisfied.

💡 Hint: Two pointers over sorted arrays: advance the child pointer only when a cookie satisfies them.

Show solution
Solution
pythondef find_content_children(g, s):
    g.sort(); s.sort()
    child = cookie = 0
    while child < len(g) and cookie < len(s):
        if s[cookie] >= g[child]:           # this cookie contents this child
            child += 1
        cookie += 1
    return child

print(find_content_children([1, 2, 3], [1, 1]))    # 1
print(find_content_children([1, 2], [1, 2, 3]))    # 2
1
2
Exercise 2 · Non-overlapping intervals (min removals)Intermediate

Context: The complement of activity selection: how many intervals must you delete so the rest do not overlap? A staple scheduling/calendar problem.

Your task: Given intervals, return the minimum number to remove so no two overlap.

Requirements:

  • Keep the max non-overlapping set greedily (earliest end), remove the rest.
  • Answer = total - kept.
  • Verify on overlapping and disjoint inputs.

💡 Hint: Sort by end time; count how many you can keep with start >= last kept end.

Show solution
Solution
pythondef erase_overlap(intervals):
    if not intervals:
        return 0
    intervals.sort(key=lambda x: x[1])      # earliest end first
    kept, last_end = 0, float("-inf")
    for start, end in intervals:
        if start >= last_end:
            kept += 1; last_end = end
    return len(intervals) - kept

print(erase_overlap([(1, 2), (2, 3), (3, 4), (1, 3)]))   # 1
print(erase_overlap([(1, 2), (1, 2), (1, 2)]))           # 2
print(erase_overlap([(1, 2), (2, 3)]))                   # 0
1
2
0
Exercise 3 · Jump game — can you reach the end?Advanced

Context: A deceptively simple greedy: track the farthest index reachable so far. Appears in reachability and packet-forwarding style problems.

Your task: Given an array where a[i] is the max jump length from i, determine if you can reach the last index starting from index 0.

Requirements:

  • Track the farthest reachable index in one pass.
  • If the current index exceeds the farthest reachable, you're stuck -> False.
  • Return True if farthest >= last index.

💡 Hint: Greedy invariant: at each i, extend reach = max(reach, i + a[i]); fail the moment i > reach.

Show solution
Solution
pythondef can_jump(nums):
    farthest = 0
    for i, jump in enumerate(nums):
        if i > farthest:                    # unreachable gap
            return False
        farthest = max(farthest, i + jump)
    return True

print(can_jump([2, 3, 1, 1, 4]))   # True
print(can_jump([3, 2, 1, 0, 4]))   # False
print(can_jump([0]))               # True (already at end)
True
False
True
Exercise 4 · Minimum jumps to reach the endExpert

Context: The optimization version of jump game: not just 'can you', but the fewest jumps. A greedy BFS-by-levels over reachable ranges.

Your task: Return the minimum number of jumps to reach the last index (assume it is reachable).

Requirements:

  • Treat each 'jump' as a BFS level: expand the current reachable window.
  • When you pass the end of the current window, increment jumps and set the new window edge to the farthest seen.
  • Achieve O(n); verify on known cases.

💡 Hint: Maintain cur_end (edge of the current jump's reach) and farthest; bump the jump count when i hits cur_end.

Show solution
Solution
pythondef min_jumps(nums):
    jumps = cur_end = farthest = 0
    for i in range(len(nums) - 1):
        farthest = max(farthest, i + nums[i])
        if i == cur_end:                    # must jump now to go further
            jumps += 1
            cur_end = farthest
    return jumps

print(min_jumps([2, 3, 1, 1, 4]))          # 2
print(min_jumps([2, 3, 0, 1, 4]))          # 2
print(min_jumps([1, 1, 1, 1]))             # 3
2
2
3
Exercise 5 · Task scheduler with cooldownProfessional

Context: A CPU/task-runner problem: identical tasks need n idle cycles between runs. The greedy insight (schedule the most frequent task first) has a clean closed-form, but a heap-based simulation is the robust general solution.

Your task: Given task labels and a cooldown n, return the least number of time units to run all tasks (idle cycles allowed).

Requirements:

  • Greedy: always run the most frequent remaining task that is off cooldown.
  • Simulate with a max-heap of counts and a cooldown queue.
  • Verify against the known formula answer on examples.

💡 Hint: Pop the highest-count tasks for a window of n+1 slots, decrement, and re-queue after the cooldown.

Show solution
Solution
pythonimport heapq
from collections import Counter, deque

def least_interval(tasks, n):
    counts = Counter(tasks)
    heap = [-c for c in counts.values()]    # max-heap via negation
    heapq.heapify(heap)
    time = 0
    cooldown = deque()                      # (ready_time, remaining_count)
    while heap or cooldown:
        time += 1
        if heap:
            remaining = heapq.heappop(heap) + 1   # ran one instance
            if remaining < 0:
                cooldown.append((time + n, remaining))
        if cooldown and cooldown[0][0] == time:
            heapq.heappush(heap, cooldown.popleft()[1])
    return time

print(least_interval(["A","A","A","B","B","B"], 2))   # 8
print(least_interval(["A","A","A","B","B","B"], 0))   # 6
print(least_interval(["A","A","A","A","B","C"], 2))   # 10
8
6
10
Exercise 6 · Gas station circuitIndustry scenario

Context: A real routing/logistics greedy: can a delivery truck complete a circular route, and from where? The one-pass greedy proof (if total gas >= total cost, a unique valid start exists) is a classic whiteboard result.

Your task: Given gas[i] at station i and cost[i] to travel to i+1 (circular), return the starting index from which you can complete the loop, or -1 if impossible.

Requirements:

  • If total gas < total cost, return -1 (impossible).
  • Track a running tank; whenever it goes negative, no start in [current_start..i] works, so reset start to i+1.
  • Prove: the surviving start completes the loop. Verify on examples.

💡 Hint: The key lemma: if the tank drops below zero at station i, none of the stations from the current candidate start up to i can be the answer — jump the start past i.

Show solution
Solution
pythondef can_complete_circuit(gas, cost):
    if sum(gas) < sum(cost):
        return -1                           # not enough fuel overall
    start = 0
    tank = 0
    for i in range(len(gas)):
        tank += gas[i] - cost[i]
        if tank < 0:                        # can't reach i+1 from `start`
            start = i + 1                   # ...nor from anything in between
            tank = 0
    return start

print(can_complete_circuit([1, 2, 3, 4, 5], [3, 4, 5, 1, 2]))   # 3
print(can_complete_circuit([2, 3, 4], [3, 4, 3]))               # -1
print(can_complete_circuit([5, 1, 2, 3, 4], [4, 4, 1, 5, 1]))   # 4
3
-1
4
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in