CPython internals & the GIL

1 · Everything is an object

Start with a moment of confusion you may have already had. You came from a language where int is a primitive — a naked machine word, cheap, no frills. Then in Python you type (42).bit_length() and it works. A plain number has methods? You attach an attribute to a function and it sticks. You pass a class into another function as if it were data. Nothing in your old mental model explains any of this, and the temptation is to file it under "Python is weird." It isn't weird — it's consistent, and this section is the one idea that makes all of it click.

Here's the analogy that unlocks it. Imagine a warehouse where every single thing — the products, the shelves, the price tags, even the catalogue that lists the products — is stored in an identical kind of labelled box. Same box, same label format, whether it holds a screw or the master inventory. That uniformity is the whole trick: one forklift, one labelling scheme, one set of rules handles everything. CPython is that warehouse. An int, a string, a function, a class, a module — all live in the same kind of box (a heap object) with the same standard header on the front. There is no secret second category of "primitives" that plays by different rules underneath.

Concretely: in CPython there are no primitives hiding underneath — an int, a function, a class, a module are all objects: a chunk of heap memory carrying (at least) a type pointer (what am I?) and a reference count (how many names point at me?). The C struct behind them is PyObject; every concrete type embeds it as its first field, which is exactly why the same machinery can inspect any value at all. Because the type itself is an object (of type type — yes, types have a type), you can pass classes and functions around like any other value — the foundation for decorators, registries, and the object model in P8.

The common mistake this section inoculates you against is reasoning about performance and memory as if Python had C's primitives. A million-integer Python list is not a million machine words packed tight; it's a million pointers to a million boxed objects, each with a header — which is precisely why NumPy exists (it stores raw machine numbers, skipping the boxes) and why "everything is an object" is the first thing you must internalise before the refcounting and GIL sections make sense. Uniformity buys you enormous flexibility; you pay for it in indirection and memory. Keep that trade in mind — it is the thread running through this entire lesson.

objects.py# Everything is an object: a value, a type, and (in CPython) a refcount.
import sys

x = 42
print("value :", x)
print("type  :", type(x))                 # <class 'int'>
print("id    :", type(id(x)))             # id() -> an int (the address in CPython)
print("is int an object? ", isinstance(int, object))   # True: the TYPE is an object
print("type of int       ", type(int))                 # <class 'type'>

# A function is an object too — you can attach attributes to it.
def greet(name):
    return f"hi {name}"

greet.category = "io"                       # functions carry a __dict__
print("function attr:", greet.category)
print("function type:", type(greet))        # <class 'function'>

# 'everything is an object' means even modules and classes are values you can pass.
print("module is object:", isinstance(sys, object))

value : 42
type  : <class 'int'>
id    : <class 'int'>
is int an object?  True
type of int        <class 'type'>
function attr: io
function type: <class 'function'>
module is object: True

2 · The interpreter loop & bytecode

Here's a belief worth killing early: "Python is interpreted, so it reads my source line by line as it runs." Almost everyone starts there, and it's wrong in a way that quietly blocks understanding. When you run a Python file, your source code is compiled first — into an intermediate instruction set — and it's that compiled form, not your text, that actually executes. Realising this is the difference between "Python is a black box that runs my strings" and "Python is a small, knowable machine I can watch operate."

The analogy: think of a recipe. Your Python source is the recipe written in prose — "cream the butter and sugar, then fold in the flour." A CPU can't follow prose; it needs a numbered checklist of tiny mechanical steps: pick up bowl, add butter, add sugar, stir 40 times. CPython's compiler is the sous-chef who reads your prose once up front and rewrites it as that checklist. The checklist is called bytecode, and the machine that mindlessly executes it one step at a time is the evaluation loop. The beautiful part: this machine is a stack machine — it has no complicated registers, just one pile (a stack) where it puts operands, applies an operation, and leaves the result. Add two numbers? Push both onto the pile, run "add", and the pile now holds the sum. Every Python expression you have ever written reduces to that.

Concretely: CPython does not execute your source directly. It compiles each function to bytecode — a sequence of instructions for that stack machine — stored on the function's __code__ object (specifically its co_code). The evaluation loop (historically the giant switch in ceval.c) fetches one instruction at a time and mutates a value stack. The tool that lets you see the checklist your prose became is dis.dis — it prints exactly what the compiler produced, turning the black box into glass.

Why any of this matters to you, and the trap to avoid. Reading bytecode is how you settle "which of these two ways is faster?" arguments with evidence instead of folklore — you disassemble both and count the work. It also demystifies whole features: a list comprehension, a generator, an f-string all become obvious once you see their opcodes. But here is the mistake to sidestep from day one: never depend on the exact opcodes. Bytecode is a private implementation detail, not a public contract; the instructions you see below are specific to a CPython version and the compiler rewrites them freely between releases. Use dis as an X-ray to understand what runs — never as an interface to build on.

disassemble.pyimport dis

def add(a, b):
    total = a + b
    return total

dis.dis(add)

  2           RESUME                   0

  3           LOAD_FAST_LOAD_FAST      1 (a, b)
              BINARY_OP                0 (+)
              STORE_FAST               2 (total)

  4           LOAD_FAST                2 (total)
              RETURN_VALUE

Read it top to bottom: LOAD_FAST_LOAD_FAST pushes the two locals a and b onto the stack (a 3.13 fused opcode), BINARY_OP 0 (+) pops both and pushes their sum, STORE_FAST pops it into total, then the last two lines reload total and RETURN_VALUE pops it as the result. That is the whole machine: push operands, apply an op, store or return.

3 · Names vs objects

Almost every Python programmer has lost an afternoon to this exact bug. You copy a list — backup = config — tweak config, and then discover to your horror that backup changed too. You "backed nothing up." You stare at the code, re-read it five times, and it looks perfectly correct. It is correct — for what Python actually did, which is not what you thought it did. That gap is one of the most consequential misunderstandings in the language, and this section closes it for good.

The fix is a single change of picture. Stop imagining a variable as a box that holds a value. In C that's true — int x = 5 carves out a box and puts the bytes 5 inside. Python does not work that way. A Python variable is a name tag — a luggage label — tied to an object that floats out on the heap. a = [1, 2, 3] creates one list object somewhere in memory and ties the label a to it. When you then write b = a, you do not photocopy the suitcase; you tie a second label, b, onto the very same suitcase. Now two labels, one object. Mutate the object through either label — b.append(4) — and of course both labels see it, because there is only one suitcase. It was never copied.

The precise statement: a variable in Python is a name (a binding in a namespace) that points at an object; it is not a box holding bytes. Assignment b = a copies the reference, not the object, so both names see the same mutable list. The other half of the rule is what happens on rebinding: a = a + [5] builds a brand-new list and re-ties the label a to that, while b's label stays on the original — so they diverge. This is the single distinction between mutating the object a name points at (everyone sees it) and rebinding the name to a different object (only that name moves).

The mistake, named plainly, and the cure. The trap is assuming = copies. It never does — it only ever binds a label. So the moment you actually want an independent copy, you must ask for one explicitly: b = a.copy() (or list(a), or copy.deepcopy for nested structures). Getting names-vs-objects right explains nearly every "why did my list change?" bug, the classic mutable-default-argument footgun, and — since a reference is exactly a "point at" — it is also the mental model you carry directly into reference counting two sections from now. Names point; objects live; copying is a thing you request, never a thing = gives you for free.

names.py# A name is a label; the object lives on the heap. Assignment rebinds the label.
a = [1, 2, 3]
b = a                 # b labels the SAME list object (no copy)
b.append(4)
print("a:", a)        # a sees the change: [1, 2, 3, 4]
print("same object:", a is b, " id equal:", id(a) == id(b))

a = a + [5]           # + builds a NEW list; a now labels it, b still the old one
print("a:", a)        # [1, 2, 3, 4, 5]
print("b:", b)        # [1, 2, 3, 4]
print("still same?", a is b)   # False

a: [1, 2, 3, 4]
same object: True  id equal: True
a: [1, 2, 3, 4, 5]
b: [1, 2, 3, 4]
still same? False

4 · Identity vs equality: is / id / ==

This is the bug that survives code review, passes every test, ships to production, and then fails only on Thursdays. Someone writes if status is "active":. It works. It works on their laptop, works in CI, works in the demo. Then in production, under real traffic, it starts returning False for statuses that are plainly "active" — intermittently, unreproducibly, maddeningly. The author swears the code hasn't changed. It hasn't. They just used the wrong one of two operators that usually agree and occasionally, catastrophically, don't.

The analogy that keeps them straight: think of two people. == asks "are these twins?" — do they look the same, same value, same everything a photo would capture. is asks a completely different question: "are these literally the same person?" — one body, standing in one place. Identical twins are equal (== True) but they are not the same person (is False). The trouble is that Python, to save memory, sometimes reuses one object for equal values — so now and then your "twins" really are one person, and is accidentally returns True, lulling you into thinking it always will. It won't.

Precisely: == asks "equal value?" and calls __eq__. is asks "the same object?" and compares id() (the object's identity — in CPython, its address). They usually agree — but not always, because CPython caches small integers (−5…256) and interns some strings, so equal values sometimes share one object and sometimes don't. A literal "active" written in your source may be interned to the same object as another literal "active"; but a status built at runtime — sliced from a request, decoded from JSON, concatenated — is a fresh, separate object with an equal value. Same value, different identity: == True, is False. That is the whole Thursday bug.

The rule, and it has exactly one exception worth memorising. Use == for values — always, every time you're comparing what something is worth. Reserve is for singletons: None, True, False, and sentinel objects, where there is by design only ever one object in the whole program, so identity and equality genuinely coincide (which is why if x is None: is not just allowed but preferred). The caching of small ints and interning of literals are implementation details you must never build on — treat any is comparison against a number or string as a latent bug, no matter how reliably it seems to work today.

identity.py# is  -> same object (identity, compares id()).   ==  -> equal value.
c = int("1000")       # built at runtime
d = int("1000")       # a DIFFERENT object with the same value
print("c == d:", c == d)        # True  (equal value)
print("c is d:", c is d)        # False (two separate objects)

# CPython caches the small integers -5..256, so these ARE the same object:
m = 100
n = 100
print("100 is 100:", m is n)    # True — cached (an implementation detail!)

# Strings: identical literals are interned by the compiler...
s1 = "hello"
s2 = "hello"
print("literal intern:", s1 is s2)          # True

# ...but strings BUILT at runtime are not automatically interned:
r1 = "".join(["h", "e", "l", "l", "o"])
r2 = "hel" + "lo"
print("runtime is:", r1 is r2, " runtime ==:", r1 == r2)   # False True

c == d: True
c is d: False
100 is 100: True
literal intern: True
runtime is: False  runtime ==: True

5 · Reference counting

Every language that hides memory from you has to answer one question: when is it safe to throw an object away? Get it wrong in one direction and you free something still in use — a crash, or worse, a security hole. Get it wrong the other way and you never free anything — the process bloats until it dies. In C you answer this by hand and pay for every mistake. Python answers it for you, silently, thousands of times a second, and this section is how. Once you see the mechanism you'll stop treating memory as magic and start reasoning about exactly when your objects die.

The analogy is a library book with a checkout card. Every time someone borrows the book, the librarian adds a tally mark to the card; every time someone returns it, she crosses one off. The rule is dead simple: the moment the tally hits zero — nobody has it checked out — the book can go back on the shelf (or be pulped). CPython does exactly this with every object. The "tally" is the reference count: how many names, list slots, and attributes currently point at the object. Bind a new name to it, the count goes up; let a name go out of scope or del it, the count goes down. When it reaches zero, the object is freed immediately — not eventually, not at some sweep later, but right then. That immediacy is a genuine feature: memory is reclaimed the instant it's provably dead.

The precise statement: CPython's primary memory manager is reference counting. Each object tracks how many references point at it; when that count drops to zero the object is freed at once. The tool for watching it is sys.getrefcount — with one quirk that trips up everyone who first meets it. To ask an object its count, you must pass that object as an argument to getrefcount, and passing it creates one more reference that exists only while the call runs. So the number you get back is always one higher than the references your own code created. See 2? That means one — your name — plus the temporary argument. It is not a bug; it's the observer effect, baked in.

The mistake to guard against is over-trusting this determinism across the whole Python world. On CPython, prompt freeing at refcount-zero is real, and it's tempting to lean on it — to assume a file closes or a lock releases the instant its object's last name disappears. But reference counting is a CPython choice, not a language guarantee: other implementations (PyPy, Jython) use tracing collectors with no per-object count, where an object may linger long after your last reference drops. So use refcounting to understand when things die, but never rely on __del__ firing at a precise moment for cleanup — reach for with / context managers, which promise deterministic release everywhere. And hold onto this section: the one case where refcounting quietly fails — the reference cycle — is the entire reason the next section exists.

refcount.pyimport sys

data = ["a", "b", "c"]
# getrefcount reports one MORE than you might expect: the argument passed to
# getrefcount is itself a temporary reference while the call runs.
print("refs to data:", sys.getrefcount(data))   # 2  (the name + the temp arg)

alias = data                                     # a second name -> +1
print("after alias  :", sys.getrefcount(data))   # 3

del alias                                        # drop that reference -> -1
print("after del     :", sys.getrefcount(data))  # 2

refs to data: 2
after alias  : 3
after del     : 2

6 · Cycles & the garbage collector

Picture a slow memory leak in a long-running service — an LLM gateway, say — that only shows up after hours of uptime. Memory climbs, gently, relentlessly, and none of your usual tools point at anything. You checked: there are no obvious lingering references, objects go out of scope like they should, refcounting is doing its job. And yet RAM keeps rising until the box tips over at 3 a.m. This is the failure mode the previous section's library-card scheme cannot catch on its own, and understanding why is the difference between a two-minute fix and a week of confusion.

The trap is a reference cycle. Two objects that point at each other — a parent holding its child, the child holding a back-reference to its parent — keep each other's tallies above zero forever, even after every outside name to them is gone. Picture two drowning swimmers each gripping the other to stay afloat: from the outside no one is holding either of them up, yet neither sinks, because each is counting on the other. The library card never reaches zero. Refcounting, which frees an object only at zero, is structurally blind to this — the counts are "correct," they're just permanently stuck at one apiece. Cycles are not exotic, either: doubly linked lists, trees with parent pointers, and caches that store objects which reference their own context all create them routinely.

So CPython bolts on a second, independent mechanism: a cyclic garbage collector (the gc module). Rather than counting references, it periodically walks the tracked objects, works out which groups are unreachable from the outside — including our two drowning swimmers — and frees them as a batch. This is why it's "garbage collection" in the classic sense: a deferred sweep, not the instant, per-object freeing of refcounting. The two systems divide the labour: refcounting handles the overwhelming common case promptly and cheaply, and the cyclic collector mops up the cycles refcounting can't, at the cost of running only now and then. To prove all this to yourself, the lab uses a weakref — a reference that deliberately does not add a tally mark, so it lets you peek at whether an object is still alive without being the very thing keeping it alive.

Where this bites in practice, and the fix. The mistake is assuming "Python has GC, so I never think about memory" — true right up until a cycle full of heavy objects (large tensors, cached responses, request contexts) sits around between collection passes and your footprint balloons. The cure once you spot it is to break the cycle: make one direction a weakref (typically the back-reference, child→parent), so refcounting can reclaim promptly and the cyclic collector never has to. This is exactly the diagnosis you'll be asked to make in the industry exercise at the end — a service leaking memory because cached objects reference their own context and back again.

cycles.pyimport gc, weakref

class Node:
    def __init__(self, name):
        self.name = name
        self.ref = None

gc.collect()          # start clean
gc.disable()          # turn OFF the cyclic collector to see refcounting alone

a = Node("a"); b = Node("b")
a.ref = b; b.ref = a   # a <-> b : a reference CYCLE
watch = weakref.ref(a) # a weak ref does NOT keep the object alive

del a, b               # drop our names, but a and b still reference each other
print("alive after del?      ", watch() is not None)  # True: refcount never hit 0

collected = gc.collect()   # the cyclic collector finds and frees the cycle
print("objects reclaimed >=2:", collected >= 2)
print("alive after gc.collect?", watch() is not None) # False
gc.enable()

alive after del?       True
objects reclaimed >=2: True
alive after gc.collect? False

7 · The GIL: threads vs processes

This is the one that ends up in a postmortem. A team has a CPU-bound job — a heavy numeric loop in pure Python — chewing through work on a shiny 8-core machine. The obvious move: spread it across 8 threads, expect roughly 8× the throughput. They ship it. The result? No speedup. None. Sometimes it's slightly slower than one thread. Eight cores sitting there, seven of them idle, and the code that "should" parallelise flatly refuses to. Nobody made a mistake in the threading code; they made a mistake about what a Python thread is. That mistake is so common, and so expensive, that it earns the headline slot of this lesson.

The culprit is the Global Interpreter Lock — the GIL. The analogy that makes it stick: imagine a workshop with eight skilled carpenters (your threads) and exactly one set of tools, chained to a single bench, with a rule that only the person holding the tool-key may work. You can hire all eight carpenters you like; only one can actually build at any instant. The others stand around waiting for the key. That single key is the GIL: a lock that lets only one thread execute Python bytecode at a time inside one interpreter. More threads doesn't mean more building — it means more people queuing for the same key, which is why you sometimes go backwards (the overhead of passing the key around).

Why would a language do something so apparently self-defeating? Because of the refcount from section 5. Every object's tally is bumped and dropped constantly, and if two threads did that to the same object simultaneously the count would corrupt — freeing live objects or leaking dead ones. Making every refcount update individually thread-safe would be slow for the 99% single-threaded case. The GIL is the pragmatic bargain: one coarse lock makes all those refcount updates safe cheaply, and single-threaded code (almost everyone's) pays nothing. The bill comes due only when you want true multi-core parallelism for pure-Python CPU work — which you don't get.

The escape is to stop sharing one tool-set: run separate processes. Each process has its own interpreter, its own memory, and — crucially — its own GIL and its own key, so two processes genuinely run in parallel on two cores. That is the whole rule, and it's measurable in the lab below: two CPU-bound threads take about as long as doing the work twice in a row (≈1×), while two processes finish in roughly half the time (≈2× on two cores). The distinction that saves you from the postmortem: is your slow stage CPU-bound (crunching numbers) or I/O-bound (waiting on the network, disk, an API)? For CPU-bound work, reach for multiprocessing, or push the hot loop into a native library (NumPy, orjson) that releases the GIL while it computes in C. For I/O-bound work, threads and asyncio shine — because a thread drops the key while it waits, letting the others run. And note the honest caveat the notes below expand on: the GIL is a CPython trait, not a Python law — 3.13 ships an experimental free-threaded build (PEP 703) that removes it — but for standard CPython today, this rule holds.

gil_demo.pyimport time, threading
from multiprocessing import Pool

def cpu_bound(n):
    total = 0
    for i in range(n):
        total += i * i
    return total

N = 20_000_000

# 1) sequential: do the work twice, one after the other
t0 = time.perf_counter()
cpu_bound(N); cpu_bound(N)
seq = time.perf_counter() - t0

# 2) two THREADS: they cannot run Python bytecode at the same time — the GIL
#    lets only one thread hold the interpreter at a time.
t0 = time.perf_counter()
ts = [threading.Thread(target=cpu_bound, args=(N,)) for _ in range(2)]
for t in ts: t.start()
for t in ts: t.join()
thr = time.perf_counter() - t0

# 3) two PROCESSES: each has its OWN interpreter and its OWN GIL -> real parallelism
if __name__ == "__main__":
    t0 = time.perf_counter()
    with Pool(2) as pool:
        pool.map(cpu_bound, [N, N])
    proc = time.perf_counter() - t0

    print(f"sequential   : {seq:.2f}s")
    print(f"2 threads    : {thr:.2f}s   speedup x{seq/thr:.2f}")
    print(f"2 processes  : {proc:.2f}s   speedup x{seq/proc:.2f}")

sequential   : 1.64s
2 threads    : 1.77s   speedup x0.93
2 processes  : 0.90s   speedup x1.82

a = [1, 2, 3]
b = a
b.append(99)
print(a, a is b)        # [1, 2, 3, 99] True  -> same object
a = a + [0]             # NEW list bound to a
print(a)                # [1, 2, 3, 99, 0]
print(b)                # [1, 2, 3, 99]  -> b unchanged
print(a is b)           # False

[1, 2, 3, 99] True
[1, 2, 3, 99, 0]
[1, 2, 3, 99]
False

import dis
def squares(n):
    return [i*i for i in range(n)]
dis.dis(squares)
# The inner comprehension is its own code object; the element is built with
# BINARY_OP (i*i) and appended with LIST_APPEND, looped by FOR_ITER.

import sys
obj = object()
print(sys.getrefcount(obj))   # 2  (name obj + temp arg to getrefcount)
box = [obj]
print(sys.getrefcount(obj))   # 3  (list holds one)
alias = obj
print(sys.getrefcount(obj))   # 4
del alias
print(sys.getrefcount(obj))   # 3
box.clear()
print(sys.getrefcount(obj))   # 2

2
3
4
3
2

import gc, weakref
class N:
    def __init__(self): self.other = None
gc.collect(); gc.disable()
x = N(); y = N()
x.other = y; y.other = x          # cycle
w = weakref.ref(x)
del x, y
print('alive:', w() is not None)  # True: refcount stuck at 1 each
n = gc.collect()
print('collected>=2:', n >= 2)    # True
print('alive:', w() is not None)  # False
gc.enable()

alive: True
collected>=2: True
alive: False

import time, threading
from multiprocessing import Pool

def work(n):
    s = 0
    for i in range(n): s += i * i
    return s

N = 20_000_000
if __name__ == '__main__':
    t = time.perf_counter(); work(N); work(N)
    seq = time.perf_counter() - t
    t = time.perf_counter()
    th = [threading.Thread(target=work, args=(N,)) for _ in range(2)]
    [x.start() for x in th]; [x.join() for x in th]
    thr = time.perf_counter() - t
    t = time.perf_counter()
    with Pool(2) as p: p.map(work, [N, N])
    pr = time.perf_counter() - t
    print(f'seq {seq:.2f}s  thread x{seq/thr:.2f}  proc x{seq/pr:.2f}')

seq 1.64s  thread x0.93  proc x1.82