AI EngineeringZero to ProductionHome·About·Contact
Appendix · Python for AI Agents · Part 7

CPython internals & the GIL

P1–P6 taught you to use Python. This part opens the hood. You will see that in CPython every value is a heap-allocated object with a type and a reference count, that your functions compile to a stack-based bytecode you can print with dis, that a name is just a label bound to an object, and that memory is reclaimed by reference counting plus a cyclic collector for the cycles refcounting cannot see. Then the headline act: the Global Interpreter Lock — what it protects, why CPU-bound threads do not speed up while processes do. Every claim here is checked against real CPython output, and the CPython-specific parts are flagged as such: the language spec does not mandate refcounting or a GIL — PyPy and others differ.

⏱️ ~2.5 hours🎯 Advanced → Industry🧠 language internalsrunnable CPython 3.13

Learning objectives

  • Explain the CPython object model: every value is a typed, refcounted PyObject on the heap.
  • Read Python bytecode with dis.dis and describe the stack-based evaluation loop.
  • Distinguish names from objects, and is/id (identity) from == (value).
  • Reason about reference counting, reference cycles, and the cyclic garbage collector (gc, weakref).
  • State what the GIL protects and demonstrate why CPU-bound threads don't speed up but processes do.
  • Separate CPython implementation details (refcount, GIL, interning) from the Python language spec.

1 · Everything is an object

In CPython, there are no primitives hiding underneath — an int, a function, a class, a module are all objects: a chunk of heap memory carrying (at least) a type pointer and a reference count. The C struct behind them is PyObject; every concrete type embeds it. Because the type itself is an object (of type type), you can pass classes and functions around like any other value — the foundation for decorators, registries, and the object model in P8.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Try it
objects.py# Everything is an object: a value, a type, and (in CPython) a refcount.
import sys

x = 42
print("value :", x)
print("type  :", type(x))                 # <class 'int'>
print("id    :", type(id(x)))             # id() -> an int (the address in CPython)
print("is int an object? ", isinstance(int, object))   # True: the TYPE is an object
print("type of int       ", type(int))                 # <class 'type'>

# A function is an object too — you can attach attributes to it.
def greet(name):
    return f"hi {name}"

greet.category = "io"                       # functions carry a __dict__
print("function attr:", greet.category)
print("function type:", type(greet))        # <class 'function'>

# 'everything is an object' means even modules and classes are values you can pass.
print("module is object:", isinstance(sys, object))
value : 42
type  : <class 'int'>
id    : <class 'int'>
is int an object?  True
type of int        <class 'type'>
function attr: io
function type: <class 'function'>
module is object: True
Spec vs implementation“Everything is an object” is part of the language model. That objects carry a refcount and that id() returns a memory address are CPython implementation facts — the spec only promises id() is a unique, constant integer for the object's lifetime.

2 · The interpreter loop & bytecode

CPython does not execute your source directly. It compiles each function to bytecode — a sequence of instructions for a stack machine — stored on the function's __code__ object. The evaluation loop (historically the giant switch in ceval.c) fetches one instruction at a time and mutates a value stack. dis.dis shows you exactly what the compiler produced.

Try it
disassemble.pyimport dis

def add(a, b):
    total = a + b
    return total

dis.dis(add)
  2           RESUME                   0

  3           LOAD_FAST_LOAD_FAST      1 (a, b)
              BINARY_OP                0 (+)
              STORE_FAST               2 (total)

  4           LOAD_FAST                2 (total)
              RETURN_VALUE

Read it top to bottom: LOAD_FAST_LOAD_FAST pushes the two locals a and b onto the stack (a 3.13 fused opcode), BINARY_OP 0 (+) pops both and pushes their sum, STORE_FAST pops it into total, then the last two lines reload total and RETURN_VALUE pops it as the result. That is the whole machine: push operands, apply an op, store or return.

source .py def add compile() AST → code bytecode co_code eval loop stack machine result value
Exact opcodes are version-specificBytecode is not a stable interface — LOAD_FAST_LOAD_FAST and the BINARY_OP arg-encoding are 3.11+/3.13 specifics. Never depend on exact opcodes across versions; use dis to understand, not to build on.

3 · Names vs objects

A variable in Python is a name (a binding in a namespace) that points at an object; it is not a box holding bytes. Assignment b = a copies the reference, not the object, so both names see the same mutable list. Rebinding a to a new object leaves b pointing at the old one. Getting this right explains nearly every “why did my list change?” bug.

Try it
names.py# A name is a label; the object lives on the heap. Assignment rebinds the label.
a = [1, 2, 3]
b = a                 # b labels the SAME list object (no copy)
b.append(4)
print("a:", a)        # a sees the change: [1, 2, 3, 4]
print("same object:", a is b, " id equal:", id(a) == id(b))

a = a + [5]           # + builds a NEW list; a now labels it, b still the old one
print("a:", a)        # [1, 2, 3, 4, 5]
print("b:", b)        # [1, 2, 3, 4]
print("still same?", a is b)   # False
a: [1, 2, 3, 4]
same object: True  id equal: True
a: [1, 2, 3, 4, 5]
b: [1, 2, 3, 4]
still same? False

4 · Identity vs equality: is / id / ==

== asks “equal value?” and calls __eq__. is asks “the same object?” and compares id(). They usually agree — but not always, because CPython caches small integers (−5…256) and interns some strings, so equal values sometimes share one object and sometimes don't. The rule for your code: use is only for singletons like None; use == for values.

Try it
identity.py# is  -> same object (identity, compares id()).   ==  -> equal value.
c = int("1000")       # built at runtime
d = int("1000")       # a DIFFERENT object with the same value
print("c == d:", c == d)        # True  (equal value)
print("c is d:", c is d)        # False (two separate objects)

# CPython caches the small integers -5..256, so these ARE the same object:
m = 100
n = 100
print("100 is 100:", m is n)    # True — cached (an implementation detail!)

# Strings: identical literals are interned by the compiler...
s1 = "hello"
s2 = "hello"
print("literal intern:", s1 is s2)          # True

# ...but strings BUILT at runtime are not automatically interned:
r1 = "".join(["h", "e", "l", "l", "o"])
r2 = "hel" + "lo"
print("runtime is:", r1 is r2, " runtime ==:", r1 == r2)   # False True
c == d: True
c is d: False
100 is 100: True
literal intern: True
runtime is: False  runtime ==: True
Never use `is` to compare valuesx is 1000 may be True or False depending on how the int was produced and which literals the compiler folded — an implementation detail. Comparing values with is is a real, hard-to-find bug. Use ==.

5 · Reference counting

CPython's primary memory manager is reference counting: each object tracks how many references point at it; when that count drops to zero the object is freed immediately. sys.getrefcount lets you watch it — remembering that the call itself holds one temporary reference while it runs, so the number is always one higher than the references you created.

Try it
refcount.pyimport sys

data = ["a", "b", "c"]
# getrefcount reports one MORE than you might expect: the argument passed to
# getrefcount is itself a temporary reference while the call runs.
print("refs to data:", sys.getrefcount(data))   # 2  (the name + the temp arg)

alias = data                                     # a second name -> +1
print("after alias  :", sys.getrefcount(data))   # 3

del alias                                        # drop that reference -> -1
print("after del     :", sys.getrefcount(data))  # 2
refs to data: 2
after alias  : 3
after del     : 2
Refcounting is a CPython choiceThe language does not require reference counting. PyPy and Jython use tracing garbage collectors with no per-object count, so sys.getrefcount is CPython-only and __del__ timing differs. Do not rely on prompt, deterministic destruction across implementations — use with/context managers for cleanup.

6 · Cycles & the garbage collector

Reference counting has one blind spot: a cycle (A refers to B, B refers to A) keeps both counts above zero forever even when nothing outside can reach them. CPython adds a separate cyclic garbage collector (the gc module) that periodically finds and frees such unreachable cycles. A weakref lets you observe an object without keeping it alive, so you can prove the cycle survives refcounting and dies only when gc.collect() runs.

Try it
cycles.pyimport gc, weakref

class Node:
    def __init__(self, name):
        self.name = name
        self.ref = None

gc.collect()          # start clean
gc.disable()          # turn OFF the cyclic collector to see refcounting alone

a = Node("a"); b = Node("b")
a.ref = b; b.ref = a   # a <-> b : a reference CYCLE
watch = weakref.ref(a) # a weak ref does NOT keep the object alive

del a, b               # drop our names, but a and b still reference each other
print("alive after del?      ", watch() is not None)  # True: refcount never hit 0

collected = gc.collect()   # the cyclic collector finds and frees the cycle
print("objects reclaimed >=2:", collected >= 2)
print("alive after gc.collect?", watch() is not None) # False
gc.enable()
alive after del?       True
objects reclaimed >=2: True
alive after gc.collect? False
ReclaimsReference countingCyclic GC (gc)
Acyclic garbageYes, immediately at refcount 0Not needed
Reference cyclesNo — counts never reach 0Yes, on collection
CostTiny, constant per ref changePeriodic scan of tracked objects
DeterminismPrompt (freed at once)Deferred to a collection pass

7 · The GIL: threads vs processes

The Global Interpreter Lock is a single mutex that lets only one thread execute Python bytecode at a time inside one interpreter. It exists because CPython's memory management (that refcount on every object) is not thread-safe; the GIL makes refcount updates safe cheaply, at the cost of true multi-core parallelism for pure-Python code. The consequence is stark and measurable: two CPU-bound threads take about as long as doing the work twice in a row, while two processes — each with its own interpreter and its own GIL — run genuinely in parallel.

Try it
gil_demo.pyimport time, threading
from multiprocessing import Pool

def cpu_bound(n):
    total = 0
    for i in range(n):
        total += i * i
    return total

N = 20_000_000

# 1) sequential: do the work twice, one after the other
t0 = time.perf_counter()
cpu_bound(N); cpu_bound(N)
seq = time.perf_counter() - t0

# 2) two THREADS: they cannot run Python bytecode at the same time — the GIL
#    lets only one thread hold the interpreter at a time.
t0 = time.perf_counter()
ts = [threading.Thread(target=cpu_bound, args=(N,)) for _ in range(2)]
for t in ts: t.start()
for t in ts: t.join()
thr = time.perf_counter() - t0

# 3) two PROCESSES: each has its OWN interpreter and its OWN GIL -> real parallelism
if __name__ == "__main__":
    t0 = time.perf_counter()
    with Pool(2) as pool:
        pool.map(cpu_bound, [N, N])
    proc = time.perf_counter() - t0

    print(f"sequential   : {seq:.2f}s")
    print(f"2 threads    : {thr:.2f}s   speedup x{seq/thr:.2f}")
    print(f"2 processes  : {proc:.2f}s   speedup x{seq/proc:.2f}")
sequential   : 1.64s
2 threads    : 1.77s   speedup x0.93
2 processes  : 0.90s   speedup x1.82
Numbers are machine-dependent — the ratio is the pointThe exact seconds vary by CPU; what is reproducible is the shape: threads give ≈1x speedup (no gain, sometimes slightly worse from lock contention), processes give ≈2x on two cores. Threads still help for I/O-bound work (network, disk) because a thread releases the GIL while it waits — which is exactly why asyncio and threads win for API calls (P5 §5).
The GIL is CPython-specific and evolvingJython and IronPython have no GIL; PyPy has one. CPython 3.13 ships an experimental free-threaded build (PEP 703) that removes the GIL behind a build flag. For standard CPython today the rule holds: CPU-bound → multiprocessing; I/O-bound → threads or async.

✓ Checkpoint — you can move on when you can…

  • Explain why a = a + [5] leaves an aliased b unchanged but a.append(5) would not.
  • Predict is vs == for cached small ints, interned literals, and runtime-built strings.
  • Describe what reference counting frees immediately and what only the cyclic gc can reclaim.
  • State what the GIL protects and why CPU-bound threads don't scale but processes do.
  • Name three things in this lesson that are CPython implementation details, not language spec.
✓ Knowledge check

A colleague writes if status is 'active': and it works on their machine but fails intermittently in production. What is happening, and what is the fix?

Show answer
String literals are often interned, so at authoring time 'active' is 'active' can be True — but a status value built at runtime (read from a request, sliced, concatenated, or decoded) is a different object even when equal, so is returns False. Identity coincidence is an implementation detail you must never rely on. Fix: compare values with == (if status == 'active':); reserve is for singletons like None.
✓ Knowledge check

Your data pipeline is CPU-bound (heavy numeric loops in pure Python). You move it to a ThreadPoolExecutor with 8 workers on an 8-core box and see no speedup. Why, and what should you do?

Show answer
The GIL serializes Python bytecode: only one of the 8 threads runs the interpreter at a time, so CPU-bound pure-Python work does not parallelize with threads — you get ≈1x. Options, best first: (1) use multiprocessing/ProcessPoolExecutor so each worker has its own interpreter and GIL (real parallelism); (2) push the hot loop into a C/NumPy/native extension that releases the GIL while it computes; (3) on 3.13, experiment with the free-threaded build. Threads would only have helped if the work were I/O-bound.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · Prove names are referencesBeginner

Context: New Python programmers are surprised when mutating a list through one name changes it through another. Proving it to yourself cements the model.

Your task: Write a function that shows a list mutated via one name is visible via an alias, and that rebinding one name does not affect the other.

Requirements:

  • Create a, alias it to b, mutate through b, show a changed
  • Then rebind a with + and show b is unchanged
  • Print a is b before and after the rebind

💡 Hint: Mutation (.append) changes the shared object; + builds a new one.

Show solution
Solution
a = [1, 2, 3]
b = a
b.append(99)
print(a, a is b)        # [1, 2, 3, 99] True  -> same object
a = a + [0]             # NEW list bound to a
print(a)                # [1, 2, 3, 99, 0]
print(b)                # [1, 2, 3, 99]  -> b unchanged
print(a is b)           # False
[1, 2, 3, 99] True
[1, 2, 3, 99, 0]
[1, 2, 3, 99]
False
Exercise 2 · Disassemble a comprehensionIntermediate

Context: Reading bytecode turns “magic” into mechanism. A list comprehension compiles to its own code object with a loop — dis reveals it.

Your task: Disassemble a function containing a list comprehension and identify the instruction that appends each element.

Requirements:

  • Define def squares(n): return [i*i for i in range(n)]
  • Call dis.dis(squares)
  • In your answer, name the opcode that builds/extends the result list

💡 Hint: Look for LIST_APPEND in the nested comprehension code object.

Show solution
Solution
import dis
def squares(n):
    return [i*i for i in range(n)]
dis.dis(squares)
# The inner comprehension is its own code object; the element is built with
# BINARY_OP (i*i) and appended with LIST_APPEND, looped by FOR_ITER.

Key opcodes: FOR_ITER drives the loop, BINARY_OP 5 (*) squares each i, and LIST_APPEND pushes it onto the result list. Exact listing varies by CPython version — run it on yours to see the current form.

Exercise 3 · Watch a refcount rise and fallAdvanced

Context: Understanding when an object is freed is the difference between confident and superstitious resource handling.

Your task: Track an object's reference count as you add and drop references, accounting for the temporary reference getrefcount itself holds.

Requirements:

  • Create an object, print sys.getrefcount
  • Add an alias and a container reference; print again after each
  • Delete them one at a time; print after each and explain the +1 offset

💡 Hint: Every extra name or container that holds the object adds exactly one to the count.

Show solution
Solution
import sys
obj = object()
print(sys.getrefcount(obj))   # 2  (name obj + temp arg to getrefcount)
box = [obj]
print(sys.getrefcount(obj))   # 3  (list holds one)
alias = obj
print(sys.getrefcount(obj))   # 4
del alias
print(sys.getrefcount(obj))   # 3
box.clear()
print(sys.getrefcount(obj))   # 2
2
3
4
3
2
Why +1getrefcount receives the object as an argument, so while it runs there is one extra reference. Subtract 1 to get the count your code created.
Exercise 4 · Force and observe a cycle collectionExpert

Context: Long-running services leak when cycles pile up between collections. Reproducing and observing a cycle is the first step to diagnosing that.

Your task: Build a reference cycle, disable gc to show refcounting alone cannot free it, then collect it and confirm reclamation with a weakref.

Requirements:

  • Two objects referring to each other; a weakref to one
  • gc.disable(), del both names, show the weakref is still alive
  • gc.collect(), show the weakref is now dead and report objects collected

💡 Hint: A weakref's callable returns None once the referent is gone.

Show solution
Solution
import gc, weakref
class N:
    def __init__(self): self.other = None
gc.collect(); gc.disable()
x = N(); y = N()
x.other = y; y.other = x          # cycle
w = weakref.ref(x)
del x, y
print('alive:', w() is not None)  # True: refcount stuck at 1 each
n = gc.collect()
print('collected>=2:', n >= 2)    # True
print('alive:', w() is not None)  # False
gc.enable()
alive: True
collected>=2: True
alive: False
Exercise 5 · Measure the GIL: threads vs processesProfessional

Context: Before choosing a concurrency model for a CPU-bound stage, you must be able to show the GIL's effect, not just assert it.

Your task: Benchmark the same CPU-bound function run sequentially, across two threads, and across two processes, and report the speedups.

Requirements:

  • A pure-Python CPU-bound function (a big arithmetic loop)
  • Time: sequential ×2, two threading.Threads, two-worker Pool
  • Print each time and the speedup vs sequential; guard the process code with __main__

💡 Hint: Threads should give ≈1x; processes ≈number-of-cores (up to 2 here).

Show solution
Solution
import time, threading
from multiprocessing import Pool

def work(n):
    s = 0
    for i in range(n): s += i * i
    return s

N = 20_000_000
if __name__ == '__main__':
    t = time.perf_counter(); work(N); work(N)
    seq = time.perf_counter() - t
    t = time.perf_counter()
    th = [threading.Thread(target=work, args=(N,)) for _ in range(2)]
    [x.start() for x in th]; [x.join() for x in th]
    thr = time.perf_counter() - t
    t = time.perf_counter()
    with Pool(2) as p: p.map(work, [N, N])
    pr = time.perf_counter() - t
    print(f'seq {seq:.2f}s  thread x{seq/thr:.2f}  proc x{seq/pr:.2f}')
seq 1.64s  thread x0.93  proc x1.82
Interpretation, not exact secondsAbsolute times depend on your CPU. The durable result is thread ≈ x1 (no parallel speedup) and process ≈ x(cores). If you see thread > x1, the work wasn't purely CPU-bound.
Exercise 6 · Diagnose a latency regression from the object modelIndustry scenario

Context: A production LLM gateway shows rising p99 latency and memory after a refactor that added a shared in-process cache holding response objects that also reference their request context. You are on call.

Your task: Explain, using the internals from this lesson, the two most likely root causes and how you would confirm and fix each.

Requirements:

  • Reference the GIL for the latency symptom under CPU-bound post-processing
  • Reference cycles + deferred gc for the memory growth
  • Give a concrete confirmation step and fix for each

💡 Hint: One symptom is about parallelism under a lock; the other is about what refcounting can't free.

Show solution

Root cause 1 — latency under the GIL. If the refactor added CPU-heavy pure-Python post-processing (token counting, JSON re-serialization) on the request threads, the GIL serializes it: adding worker threads won't help and p99 climbs under load. Confirm: profile with py-spy/cProfile; check whether wall-time scales with concurrency. Fix: move the CPU stage to a ProcessPoolExecutor, or replace the hot loop with a native library (NumPy/orjson) that releases the GIL.

Root cause 2 — cycles + deferred collection. Cached response objects that reference their request context (and vice versa) form cycles; refcounting alone never frees them, so memory grows until the periodic cyclic gc runs — and if objects have __del__, older CPythons could even refuse to collect them. Confirm: gc.set_debug(gc.DEBUG_SAVEALL) or objgraph to find the cycle; watch gc.get_count(). Fix: break the cycle with a weakref from response→context, or drop the back-reference so refcounting reclaims promptly. Bound the cache (e.g. lru_cache(maxsize=...)) so it can't grow without limit.

© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in