Memory Optimization & Concurrency / the GIL

This measures how many bytes each tiny Python object actually takes. The surprise: nothing is free. Even the number 0 or an empty list is a full object carrying bookkeeping — a reference count, a type pointer, and length info — so "small" values are surprisingly heavy.

1. sys.getsizeof(x) returns the size of one object in bytes. It's the tool you use to measure instead of guessing.
2. The comments show the results on a typical machine: an int is ~28 bytes, an empty string ~49, an empty list ~56, an empty dict ~64. Compare that to a language where an int is 8 bytes — Python trades size for flexibility.
3. The last comment is the real lesson: a list of a million ints stores a million 8-byte pointers plus the million int objects they point to. That's why homogeneous numeric data should live in an array or numpy, not a list.

What the output means: Four numbers roughly like 28, 49, 56, 64. Exact values vary by Python version and OS, but the order of magnitude — tens of bytes for "empty" — is the point.

Try this: Run sys.getsizeof([1, 2, 3]) and then sys.getsizeof([1, 2, 3, 4]). Each extra element adds ~8 bytes — that's the pointer per slot the last comment warns about.

Two classes that look identical, but store their attributes differently. By default every object gets a hidden __dict__ — a full hash table — to hold its attributes. That's flexible but heavy. __slots__ tells Python "these are the only attributes," so it uses a compact fixed array instead and skips the dict entirely.

1. ChunkA is a normal class: each instance carries its own __dict__.
2. ChunkB declares __slots__ = ("text", "vec"). Same two attributes, but now there is no per-instance dict — the values sit in a small slot array baked into the object.
3. sys.getsizeof(a.__dict__) measures just the dictionary that ChunkA instances carry (~100+ bytes each). ChunkB has no such dict, so that cost disappears entirely.

What the output means: A number in the ~100+ range — the size of one instance's __dict__. Multiply that by millions of chunks and you see why __slots__ is a big win at scale.

Try this: Add a line a.extra = 5 — it works, because ChunkA has a dict. Then try b.extra = 5 on the slotted object: you get an AttributeError. That's the trade-off — smaller and faster, but the attribute set is locked.

The core idea here is copy vs. view. Slicing a normal bytes or list makes a brand-new copy of the data — fine for small things, disastrous for a 10 MB blob you slice often. A memoryview gives you a window onto the same underlying bytes, so no data is duplicated.

1. bytearray(10 * 1024 * 1024) allocates a 10 MB editable buffer. memoryview(data) wraps it without copying anything.
2. view[1024:2048] is the zero-copy slice — it points into the same memory. It costs O(1) time and 0 extra bytes, unlike data[1024:2048] which would copy 1 KB. Proof that they share memory: chunk[0] = 255 writes straight through and changes data too.
3. array.array("f", ...) stores a million floats as raw 4-byte numbers (~4 MB) instead of a million float objects (~30+ MB in a list) — the "f" means "float".
4. struct.pack(">IHf", ...) packs values into a compact binary record and unpack reads them back — the format string ">IHf" means big-endian uint32, uint16, float. This is how wire protocols and file headers are read.

What the output means: No printed output — the payoff is invisible: chunk is created with zero copying, and after chunk[0] = 255 the first byte of the big data buffer is now 255. The struct round-trip gives back 42, 7, 3.14.

Try this: After chunk[0] = 255, print data[1024] — it's 255. You mutated the big buffer through the small window, proving the slice never copied.

This shows how Python decides an object is garbage. Every object counts how many references point to it; when that count hits zero, it's freed immediately. A weak reference is the exception — it lets you point at an object without bumping that count, so it won't keep the object alive. That's exactly what a cache wants: cache a thing, but don't be the reason it can never be freed.

1. weakref.WeakValueDictionary() is a dict whose values are held weakly. Putting m in it does not, by itself, keep the model alive.
2. del m removes the last strong reference. Now the only thing pointing at the model is the weak cache entry — which doesn't count — so the object is freed and the entry vanishes on its own.
3. print(list(cache)) shows []: the cache emptied itself, so it caused no memory leak.
4. gc.collect() handles the harder case — reference cycles (A points to B, B points back to A). Their counts never reach zero on their own, so the cyclic garbage collector must find and free them.

What the output means: [] — an empty list, meaning the cached model disappeared the moment its last real reference was deleted. gc.collect() returns a count of objects it reclaimed.

Try this: Comment out del m and re-run: now list(cache) prints ['opus'], because m is still a live strong reference keeping the model — and its cache entry — alive.

This is the payoff of the GIL discussion: threads do help when the work is waiting (network, disk, an LLM API). While a thread waits, Python releases the GIL so other threads run — so ten one-second waits can overlap and finish in about one second instead of ten.

1. fetch(doc_id) uses time.sleep(1) as a fake network call. Sleeping releases the GIL, just like a real I/O wait does — so multiple fetch calls can wait at the same time.
2. ThreadPoolExecutor(max_workers=10) creates a pool of 10 worker threads. The with block cleans them up automatically when done.
3. ex.submit(fetch, i) hands each job to the pool and returns a future — a placeholder for a result that isn't ready yet.
4. as_completed(futures) yields each future as it finishes, and f.result() pulls out its return value. Because all ten waited in parallel, the whole batch takes ~1 second.

What the output means: 10 — all ten fetches completed. The headline is the time: ~1 second total instead of ~10, because the waits overlapped.

Try this: Change max_workers to 2 and re-run. Now only two can wait at once, so 10 jobs take ~5 seconds — you can watch the concurrency limit control the speed.

When the work is CPU-bound pure Python (real number-crunching, no waiting), threads can't help — the GIL lets only one run at a time. The fix is separate processes: each has its own Python interpreter and its own GIL, so they truly run in parallel on different CPU cores.

1. heavy(n) is deliberately CPU-heavy — it just adds up squares with no I/O. This is the exact case where threads would give no speedup at all.
2. if __name__ == "__main__": is required when using processes on macOS and Windows. They start new processes by re-importing this file, and this guard stops that import from launching the pool again (which would loop forever).
3. ProcessPoolExecutor() with no argument uses one worker per CPU core. ex.map(heavy, [10_000_000] * 4) runs four heavy jobs, spreading them across cores.
4. The catch, noted in the section text: inputs and results must be pickled (serialized) and copied between processes. That overhead only pays off when each job does a lot of compute — as these do.

What the output means: A list of four identical big sums. On a 4-core machine it finishes in roughly a quarter of the sequential time — real parallelism, which threads could not deliver here.

Try this: Swap ProcessPoolExecutor for ThreadPoolExecutor and time both. The thread version stays as slow as running one after another — a hands-on demonstration of what the GIL blocks.

The interview-ready version of the whole lesson: threads help I/O, not CPU. This snippet is the I/O case — network calls that spend their time waiting — so a thread pool overlaps the waits and wins.

1. fetch(url) stands for a network call (its body is elided with ...). The key fact is that during a network wait, CPython releases the GIL, letting other threads proceed.
2. ThreadPoolExecutor(max_workers=10) gives ten workers; ex.map(fetch, urls) runs fetch once per URL and returns results in order.
3. The comment ~1x latency, not 10x is the punchline: ten calls that each wait finish in about the time of one, because the waits happen simultaneously.

Try this: In an interview, say the rule out loud: I/O-bound → threads (or async) because the GIL is released while waiting; CPU-bound pure Python → processes because each gets its own GIL. That single sentence answers most GIL questions.

Two independent memory tricks in one snippet: use __slots__ to shrink each object, and use a generator to avoid building a giant list at all.

1. class Point with __slots__ = ("x", "y") drops the per-instance __dict__ — a big saving when you create millions of points.
2. squares(n) uses yield instead of return, which makes it a generator: it produces one value at a time on demand rather than building a full list of n numbers up front.
3. So a normal function would need O(n) memory to hold every square at once; the generator needs only O(1) — it remembers just where it is and hands back the next value when asked.

What the output means: No output on its own — these are building blocks. The win is structural: constant memory for the sequence and a much smaller footprint per Point.

Try this: Compare sum(i*i for i in range(10_000_000)) (a generator, tiny memory) against sum([i*i for i in range(10_000_000)]) (a list, builds all 10M in memory first). Same answer, very different memory use.