AI EngineeringZero to ProductionHome·About·Contact
Appendix · Python for AI Agents · Part 9

Functional & recursion theory

Functions in Python are first-class values, which makes a whole functional toolkit available: closures that capture state, pure functions you can cache and parallelize safely, and higher-order functions that take or return other functions. This part makes the theory precise. You will see how a closure stores free variables in cells (and the famous late-binding trap that follows), why immutability makes reasoning and memoization sound, how recursion really uses the call stack — including that CPython does no tail-call optimization, so deep recursion raises RecursionError — and how to convert any recursion into an explicit stack. We close with map/filter/reduce vs comprehensions and partial application, each with complexity notes.

⏱️ ~2.5 hours🎯 Advanced → Industryλ functional + recursionrunnable CPython 3.13

Learning objectives

  • Explain closures in terms of free variables and cells, and avoid the late-binding trap.
  • Define pure functions and immutability, and see why the mutable-default-argument bug happens.
  • Use higher-order functions: passing functions in and returning them out.
  • Reason about recursion via the call stack, the recursion limit, and CPython's lack of tail-call optimization.
  • Convert a recursive algorithm to an explicit-stack iterative one to remove the depth limit.
  • Apply memoization, choose between map/filter/reduce and comprehensions, and use partial/currying — with complexity in mind.

1 · Closures — free variables & cells

A closure is a nested function together with the enclosing variables it references — its free variables. CPython stores each captured variable in a cell shared between the outer and inner functions, so the inner function keeps that state alive after the outer returns. Each call to the factory creates fresh cells, so closures have independent state. nonlocal lets the inner function rebind the captured name rather than shadow it with a new local.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Try it
closures.py# A closure is a function plus the enclosing variables it captures ("free vars").
def make_counter():
    count = 0                       # a free variable for the inner function
    def inc():
        nonlocal count              # rebind the enclosing name, don't shadow it
        count += 1
        return count
    return inc

c1, c2 = make_counter(), make_counter()
print(c1(), c1(), c1())             # 1 2 3   -> each closure has its OWN cell
print(c2())                         # 1       -> independent state

# Introspection: free variables live in code.co_freevars and in __closure__ cells
print("free vars:", make_counter().__code__.co_freevars)   # ('count',)
print("has closure cells:", make_counter().__closure__ is not None)
1 2 3
1
free vars: ('count',)
has closure cells: True

2 · The late-binding gotcha

Closures capture the variable, not the value it had when the closure was created. So building lambdas in a loop that all reference the loop variable makes them all see its final value — a bug that bites everyone once. The idiomatic fix is to bind the current value as a default argument (defaults are evaluated at definition time), or to close over a parameter via a factory function.

Try it
late_binding.py# The late-binding gotcha: closures capture the VARIABLE, not its value-at-creation.
funcs = [lambda: i for i in range(3)]
print("late binding:", [f() for f in funcs])   # [2, 2, 2]  (all see final i)

# Fix 1: bind the current value as a default argument (evaluated at def time)
fixed = [lambda i=i: i for i in range(3)]
print("default-arg fix:", [f() for f in fixed])# [0, 1, 2]

# Fix 2: a factory that closes over its own parameter
def make(i):
    return lambda: i
fixed2 = [make(i) for i in range(3)]
print("factory fix:", [f() for f in fixed2])   # [0, 1, 2]
late binding: [2, 2, 2]
default-arg fix: [0, 1, 2]
factory fix: [0, 1, 2]
This is not a Python quirk to memorize — it follows from the modelBecause a closure holds a reference to the enclosing variable (a cell), and the loop keeps updating that one variable, every lambda reads the same final cell. The default-argument fix works precisely because default values are computed once, at def time, and stored on the function.

3 · Pure functions & immutability

A pure function returns the same output for the same inputs and has no side effects — it does not mutate arguments, globals, or the outside world. Purity is what makes memoization correct, testing trivial, and parallelism safe. Its opposite shows up in the notorious mutable default argument: a default list is created once and shared across calls, so it accumulates state across invocations.

Try it
pure.py# A PURE function: output depends only on inputs, and it mutates nothing external.
def pure_add_tax(price, rate):
    return round(price * (1 + rate), 2)      # no side effects, deterministic

# An IMPURE function: mutates its argument (a side effect) -> surprises callers.
def impure_append(item, bucket=[]):          # classic mutable-default bug!
    bucket.append(item)
    return bucket

print(pure_add_tax(100, 0.2))                # 120.0  (same input -> same output)
print(pure_add_tax(100, 0.2))                # 120.0
print(impure_append(1))                      # [1]
print(impure_append(2))                      # [1, 2]  <- shared default persists!

# The fix for the mutable default:
def safe_append(item, bucket=None):
    bucket = [] if bucket is None else bucket
    bucket.append(item)
    return bucket
print(safe_append(1), safe_append(2))        # [1] [2]
120.0
120.0
[1]
[1, 2]
[1] [2]
The None sentinel patternNever use a mutable object ([], {}) as a default argument. Use None and build the fresh object inside the function. This single habit prevents a whole class of shared-state bugs.

4 · Higher-order functions

Functions are first-class values: you can store them, pass them as arguments, and return them. A higher-order function does one of the last two. map and filter take a function and an iterable and return lazy iterators; reduce folds a sequence to one value. Returning a function is how decorators, adders, and configurable callbacks are built.

Try it
hof.pyfrom functools import reduce

nums = [1, 2, 3, 4, 5]

# map / filter are lazy iterators; wrap in list() to materialize.
print("map    :", list(map(lambda x: x * x, nums)))        # [1, 4, 9, 16, 25]
print("filter :", list(filter(lambda x: x % 2 == 0, nums)))# [2, 4]
print("reduce :", reduce(lambda a, b: a + b, nums, 0))     # 15

# The Pythonic equivalents are comprehensions (usually clearer & as fast):
print("comp   :", [x * x for x in nums if x % 2])          # [1, 9, 25]

# Higher-order: a function that RETURNS a function (a decorator-style adder).
def adder(n):
    def add(x): return x + n
    return add
add10 = adder(10)
print("hof    :", add10(5))                                # 15
map    : [1, 4, 9, 16, 25]
filter : [2, 4]
reduce : 15
comp   : [1, 9, 25]
hof    : 15
map/filter vs comprehensionsFor a simple transform-and-filter, a comprehension [x*x for x in nums if x%2] is usually clearer than map+filter+lambda and is at least as fast. Reach for map/filter when you already have a named function, or want the lazy iterator for streaming.

5 · Recursion, the call stack & no TCO

Each function call pushes a frame onto the call stack (arguments, locals, return address). Recursion just calls the same function again, stacking frames until a base case unwinds them. Python caps recursion depth (default 1000) to catch runaway recursion before it exhausts the C stack. Crucially, CPython performs no tail-call optimization: even a function whose recursive call is the last thing it does still allocates a new frame, so deep recursion raises RecursionError.

Try it
recursion.pyimport sys

# Recursion uses the call stack: each call adds a frame. Python does NOT optimize
# tail calls, so recursion depth is bounded by the recursion limit.
print("recursion limit:", sys.getrecursionlimit())   # 1000 by default

def factorial(n):
    return 1 if n == 0 else n * factorial(n - 1)      # not tail-optimized either
print("factorial(10):", factorial(10))                # 3628800

# Even a perfectly tail-recursive function blows the stack — no TCO in CPython:
def count_down(n):
    if n == 0:
        return "done"
    return count_down(n - 1)                          # tail call, still a new frame

try:
    count_down(100_000)
except RecursionError:
    print("RecursionError: CPython has no tail-call optimization")
recursion limit: 1000
factorial(10): 3628800
RecursionError: CPython has no tail-call optimization
count_down(3) frame 1 count_down(2) frame 2 count_down(1) frame 3 count_down(0) base case unwind return up
No TCO is a deliberate CPython choiceGuido van Rossum has explained the omission as intentional: tail-call elimination would erase frames from tracebacks, hurting debuggability. So in Python, prefer iteration or an explicit stack for deep problems; don't rely on the interpreter to flatten tail recursion the way Scheme does.

6 · Recursion → explicit stack

Any recursion can be rewritten with an explicit stack (a list) that you push and pop yourself. The work then lives on the heap instead of the call stack, so it is bounded only by memory — no RecursionError, and you can pause/resume or change traversal order at will. This is the standard fix when a tree or nested structure can be arbitrarily deep.

Try it
explicit_stack.py# Convert recursion to an EXPLICIT stack -> no depth limit, iterative control.
def sum_nested_recursive(x):
    if isinstance(x, int):
        return x
    return sum(sum_nested_recursive(c) for c in x)

def sum_nested_iterative(root):
    total, stack = 0, [root]
    while stack:                    # our own stack on the heap, not the call stack
        node = stack.pop()
        if isinstance(node, int):
            total += node
        else:
            stack.extend(node)      # push children
    return total

data = [1, [2, 3, [4, 5]], 6]
print("recursive:", sum_nested_recursive(data))   # 21
print("iterative:", sum_nested_iterative(data))   # 21

# The iterative version handles arbitrarily deep nesting without RecursionError.
deep = 1
for _ in range(50_000):
    deep = [deep]                    # 50k-deep nesting
print("deep sum:", sum_nested_iterative(deep))     # 1
recursive: 21
iterative: 21
deep sum: 1

7 · Memoization & complexity

Memoization caches a pure function's results so repeated inputs are returned instantly. On naive recursive Fibonacci it collapses an exponential O(φn) call tree into linear O(n) time (each n computed once) at O(n) space. functools.lru_cache does this in one decorator — but only correctly for pure functions with hashable arguments.

Try it
memoization.pyfrom functools import lru_cache
import time

# Naive Fibonacci is exponential: O(phi^n) calls (recomputes the same subproblems).
def fib_slow(n):
    return n if n < 2 else fib_slow(n - 1) + fib_slow(n - 2)

# Memoized: each n computed once -> O(n) time, O(n) space. Requires a PURE function.
@lru_cache(maxsize=None)
def fib_fast(n):
    return n if n < 2 else fib_fast(n - 1) + fib_fast(n - 2)

print("fib_fast(30):", fib_fast(30))      # 832040
print("cache:", fib_fast.cache_info())    # hits/misses show the reuse

# fib_slow(35) is already noticeably slow; fib_fast(100) is instant:
print("fib_fast(100):", fib_fast(100))    # 354224848179261915075
fib_fast(30): 832040
cache: CacheInfo(hits=28, misses=31, maxsize=None, currsize=31)
fib_fast(100): 354224848179261915075
ApproachTimeSpaceNotes
Naive recursion (fib)O(φn) ≈ exponentialO(n) stackRecomputes subproblems
Memoized recursionO(n)O(n) cache + stackPure fn required; hashable args
Bottom-up iterationO(n)O(1) with two varsNo recursion limit at all

8 · Partial application & currying

Partial application fixes some of a function's arguments to produce a new, more specialized function. functools.partial is the standard tool — great for pre-configuring callbacks and API wrappers. Currying is the related idea of turning an n-argument function into a chain of one-argument functions, which you can do by hand with nested closures.

Try it
partial.pyfrom functools import partial

def power(base, exponent):
    return base ** exponent

# partial application: pre-fill some arguments to make a specialized function.
square = partial(power, exponent=2)
cube   = partial(power, exponent=3)
print("square(5):", square(5))     # 25
print("cube(2):", cube(2))         # 8

# "Currying" by hand: turn f(a, b) into f(a)(b) via nested closures.
def curry_add(a):
    return lambda b: a + b
print("curried:", curry_add(3)(4)) # 7

# partial is common for pre-configuring callbacks / API wrappers:
log_error = partial(print, "[ERROR]")
log_error("disk full")             # [ERROR] disk full
square(5): 25
cube(2): 8
curried: 7
[ERROR] disk full

✓ Checkpoint — you can move on when you can…

  • Explain why [lambda: i for i in range(3)] all return 2, and fix it two ways.
  • Identify a pure vs impure function and rewrite the mutable-default bug safely.
  • Predict whether a tail-recursive function will overflow in CPython, and why.
  • Convert a recursive tree sum into an explicit-stack loop that survives 50k-deep nesting.
  • State the time/space complexity of naive vs memoized Fibonacci and when memoization is valid.
✓ Knowledge check

You register per-item callbacks in a loop: for name in names: register(lambda: handle(name)). At runtime every callback processes the last name. What's wrong and how do you fix it?

Show answer
Late binding: each lambda captures the variable name (a shared cell), not its value at creation time. By the time the callbacks run, the loop has finished and name holds the last value, so all callbacks see it. Fix by binding the current value at definition time: register(lambda name=name: handle(name)), or use a factory def make(n): return lambda: handle(n) and call register(make(name)). Both give each closure its own captured value.
✓ Knowledge check

You memoize a function with @lru_cache and it returns stale or wrong results for some callers. What properties must the function have for memoization to be correct, and what breaks it?

Show answer
Memoization is only sound for pure functions with hashable arguments: same inputs must always mean same output, and results must not depend on time, randomness, external state, or mutable arguments. It breaks when (a) the function reads changing global/DB state, (b) an argument is mutable and later mutated (the cache key no longer reflects the value — and unhashable args like lists raise TypeError), or (c) the result is per-user/per-request. Fixes: only cache the pure core, pass hashable/immutable args (tuples, frozensets), and bound the cache with maxsize. For time/user-varying data, don't cache — or add an explicit expiry.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Build a stateful counter with a closureBeginner

Context: Closures are the simplest way to bundle behavior with private, persistent state — no class needed.

Your task: Write a make_counter() that returns a function; each call increments and returns a private count.

Requirements:

  • Use nonlocal to update the captured count
  • Show two independent counters have separate state
  • Print a short sequence from each

💡 Hint: The count lives in the enclosing scope, not a global.

Show solution
Solution
def make_counter():
    count = 0
    def inc():
        nonlocal count
        count += 1
        return count
    return inc

a, b = make_counter(), make_counter()
print(a(), a(), a())   # 1 2 3
print(b())             # 1  (independent)
1 2 3
1
Exercise 2 · Fix the mutable-default bugIntermediate

Context: The mutable-default-argument bug silently corrupts data across calls and is a favorite interview trap.

Your task: Demonstrate the bug with a function that uses bucket=[], then write the correct version.

Requirements:

  • Show the buggy version accumulating across calls
  • Rewrite using the None sentinel
  • Show the fixed version gives a fresh list each call

💡 Hint: Defaults are created once, at function-definition time.

Show solution
Solution
def buggy(item, bucket=[]):
    bucket.append(item); return bucket
print(buggy(1))   # [1]
print(buggy(2))   # [1, 2]  <- shared!

def fixed(item, bucket=None):
    bucket = [] if bucket is None else bucket
    bucket.append(item); return bucket
print(fixed(1))   # [1]
print(fixed(2))   # [2]
[1]
[1, 2]
[1]
[2]
Exercise 3 · Show CPython has no tail-call optimizationAdvanced

Context: Engineers from Lisp/Scheme backgrounds assume tail recursion is free in every language. In CPython it is not, and proving it prevents a real class of production crashes.

Your task: Write a tail-recursive function and show it raises RecursionError at depth, then give an iterative equivalent that doesn't.

Requirements:

  • A function whose only recursive call is in tail position
  • Call it deep enough to raise RecursionError; catch and report
  • Provide a loop version that returns the correct result for the same depth

💡 Hint: The tail call still allocates a frame — no reuse.

Show solution
Solution
def down(n):
    if n == 0: return 'done'
    return down(n - 1)          # tail call, but a NEW frame

try:
    down(100_000)
except RecursionError:
    print('overflowed: no TCO')

def down_iter(n):
    while n:                     # same logic, constant stack
        n -= 1
    return 'done'
print(down_iter(100_000))        # done
overflowed: no TCO
done
Exercise 4 · Iterative DFS with an explicit stackExpert

Context: Deep or user-supplied tree structures can exceed the recursion limit; production traversals use an explicit stack.

Your task: Convert a recursive depth-first traversal of a nested list into an explicit-stack iterative version that collects leaves in DFS order.

Requirements:

  • Recursive version for reference
  • Iterative version using a list as a stack
  • Show both agree, and that the iterative one handles very deep nesting

💡 Hint: Push children so the pop order matches DFS.

Show solution
Solution
def leaves_rec(x, out=None):
    out = [] if out is None else out
    if isinstance(x, int): out.append(x)
    else:
        for c in x: leaves_rec(c, out)
    return out

def leaves_iter(root):
    out, stack = [], [root]
    while stack:
        node = stack.pop()
        if isinstance(node, int): out.append(node)
        else: stack.extend(reversed(node))   # preserve left-to-right DFS
    return out

t = [1, [2, [3, 4]], 5]
print(leaves_rec(t))    # [1, 2, 3, 4, 5]
print(leaves_iter(t))   # [1, 2, 3, 4, 5]
deep = 1
for _ in range(50_000): deep = [deep]
print(leaves_iter(deep))# [1]  (no RecursionError)
[1, 2, 3, 4, 5]
[1, 2, 3, 4, 5]
[1]
Exercise 5 · Memoize an expensive pure function correctlyProfessional

Context: A recommendation service recomputes an expensive similarity score for repeated (user, item) pairs. Caching helps — but only if done safely.

Your task: Wrap a pure scoring function with lru_cache, prove reuse via cache_info(), and state the conditions that keep it correct.

Requirements:

  • A pure function of two hashable args
  • Bound the cache with maxsize
  • Call with repeated args and print cache_info() showing hits

💡 Hint: Bounded cache + hashable args + purity = safe.

Show solution
Solution
from functools import lru_cache

@lru_cache(maxsize=1024)
def score(user, item):           # pure: depends only on its args
    return (hash((user, item)) % 1000) / 1000.0

for _ in range(3):
    score('u1', 'i1'); score('u1', 'i2')
print(score.cache_info())        # hits=4, misses=2, currsize=2
CacheInfo(hits=4, misses=2, maxsize=1024, currsize=2)
When NOT to cacheIf the score depends on freshness, per-request context, or mutable inputs, caching returns stale/wrong results. Cache only the pure core, use hashable args, bound the size, and add an expiry for time-varying data.
Exercise 6 · Choose functional tools for a data pipeline stageIndustry scenario

Context: You maintain an ingestion stage that (1) normalizes each record, (2) drops invalid ones, (3) combines per-record scores into a total, and (4) is called across many worker processes. A colleague wrote it as one deeply nested recursive function with a shared module-level accumulator list and it crashes on large batches.

Your task: Redesign the stage using pure functions, the right higher-order tools, and safe state, and explain why each choice fixes a specific failure.

Requirements:

  • Replace the shared accumulator with pure functions + a fold
  • Use map/filter (or comprehensions) for transform/validate
  • Explain why this makes the stage parallel-safe and why recursion was the wrong shape

💡 Hint: Purity removes the shared-state crash; a fold replaces the accumulator; iteration removes the depth limit.

Show solution

Why the original failed. A shared module-level accumulator makes the function impure and unsafe across worker processes/threads (races, stale state), and deep recursion over a large batch hits CPython's recursion limit (no TCO) — two independent crashes.

Redesign. Make each step a pure function of one record, compose them with map/filter (or comprehensions) for a flat iterative pass, and combine scores with an explicit reduce fold instead of mutating a shared list. Pure + local state means each worker is independent, so the stage parallelizes safely.

Solution
from functools import reduce

def normalize(rec):            # pure
    return {**rec, 'name': rec['name'].strip().lower()}

def is_valid(rec):             # pure predicate
    return rec.get('score') is not None

def process(batch):
    cleaned = (normalize(r) for r in batch)          # map, lazy
    valid = [r for r in cleaned if is_valid(r)]      # filter
    total = reduce(lambda acc, r: acc + r['score'], valid, 0)  # fold
    return valid, total

batch = [{'name': ' Ada ', 'score': 3}, {'name': 'Bo', 'score': None},
         {'name': 'CE', 'score': 5}]
records, total = process(batch)
print(len(records), total)     # 2 8
2 8
The through-linePure functions are the unit of safe parallelism; a fold (reduce) is the pure way to combine; iteration/comprehensions replace deep recursion. Reach for recursion only when the problem is genuinely tree-shaped and shallow, or convert it to an explicit stack.
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in