Memory & concurrency
Two hard problems every system inherits from the OS: how memory is faked so each process thinks it owns the machine (virtual memory + paging), and what goes wrong when threads share it (races, deadlock). Runnable page-replacement and banker's-algorithm simulations turn the theory into fault counts and safe/unsafe verdicts.
Learning objectives
- Explain virtual memory, pages, and why page faults happen.
- Simulate FIFO, LRU and optimal replacement and compare fault counts.
- Diagnose a race condition and fix it with a mutex or a semaphore.
- State the four Coffman conditions and run the banker's algorithm to check safety.
- Reason about producer-consumer and bounded buffers.
threading, so the race demo is genuinely non-deterministic; the paging and banker's labs are deterministic models of the algorithms.1 · Virtual memory & paging
Each process is given a virtual address space: a private, contiguous-looking range of addresses starting at zero. The OS and the CPU's MMU (memory management unit) translate those virtual addresses to real physical RAM addresses. Memory is handled in fixed-size chunks called pages (physical slots are frames). The mapping lives in a page table. When a program touches a page that isn't in RAM, the CPU raises a page fault; the OS fetches the page from disk into a free frame and resumes the program.
paging.py# MODEL: translate virtual pages to physical frames; a miss is a page fault.
class MMU:
def __init__(self, num_frames):
self.table = {} # virtual page -> frame
self.free = list(range(num_frames))
self.faults = 0
def access(self, page):
if page in self.table:
return ("hit", self.table[page])
self.faults += 1 # page fault: not resident
if self.free:
frame = self.free.pop(0)
self.table[page] = frame
return ("fault->loaded", frame)
return ("fault->no-frame", None) # eviction needed (see Step 2)
mmu = MMU(num_frames=2)
for pg in [1, 1, 2, 3]:
print(pg, mmu.access(pg))
print("total faults:", mmu.faults)
1 ('fault->loaded', 0)
1 ('hit', 0)
2 ('fault->loaded', 1)
3 ('fault->no-frame', None)
total faults: 3
2 · Page replacement — FIFO, LRU, optimal
When RAM is full and a new page must load, the OS must evict one. Which? Three reference policies: FIFO (evict the oldest-loaded page), LRU (evict the least-recently-used page — the SD5 cache idea, now for RAM), and optimal (evict the page used furthest in the future). Optimal is unimplementable in reality (it needs the future) but is the theoretical floor every real policy is measured against.
page_replace.py# MODEL: count page faults for a reference string under three policies.
def simulate(refs, frames, policy):
mem, faults = [], 0
for i, page in enumerate(refs):
if page in mem:
if policy == "LRU": # mark as most-recently used
mem.remove(page); mem.append(page)
continue
faults += 1
if len(mem) < frames:
mem.append(page); continue
if policy == "FIFO":
victim = mem[0]
elif policy == "LRU":
victim = mem[0] # front = least recently used
else: # OPTIMAL: page used furthest in the future (or never)
future = refs[i + 1:]
victim = max(mem, key=lambda p: future.index(p)
if p in future else float("inf"))
mem.remove(victim); mem.append(page)
return faults
refs = [7, 0, 1, 2, 0, 3, 0, 4, 2, 3, 0, 3, 2]
for pol in ("FIFO", "LRU", "OPTIMAL"):
print(f"{pol:8s} faults={simulate(refs, 3, pol)}")
FIFO faults=10
LRU faults=9
OPTIMAL faults=7
| Policy | Evicts | Needs | Reality |
|---|---|---|---|
| FIFO | oldest loaded | load order | cheap; can hit Bélády's anomaly |
| LRU | least recently used | access history | great in practice; approximated in HW |
| Optimal | used furthest ahead | the future | unimplementable; a lower-bound yardstick |
3 · The concurrency problem & race conditions
When two threads share memory and at least one writes, the result can depend on the timing of their interleaving — a race condition. The classic example is counter += 1, which is really three steps (read, add, write). If two threads interleave those steps, an update can be lost. The buggy region is the critical section: code that must not run concurrently.
race.py# REAL threads: this genuinely races. counter += 1 is read-modify-write,
# so concurrent increments lose updates. Output varies per run.
import threading
counter = 0
def bump():
global counter
for _ in range(100_000):
counter += 1 # NOT atomic: read, +1, write
threads = [threading.Thread(target=bump) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print("counter:", counter, "(expected 400000)")
counter: 400000 (expected 400000) # or LESS, and it varies per run
4 · Mutexes & semaphores
A mutex (mutual-exclusion lock) lets exactly one thread into a critical section at a time; others wait. A semaphore generalises it to a counter that admits up to N threads — useful for limiting concurrency (e.g. "at most 3 downloads at once"). A mutex is a semaphore with N=1.
mutex.py# REAL threads: the lock serialises the critical section, so no update is lost.
import threading
counter = 0
lock = threading.Lock()
def bump():
global counter
for _ in range(100_000):
with lock: # only one thread in here at a time
counter += 1
threads = [threading.Thread(target=bump) for _ in range(4)]
for t in threads: t.start()
for t in threads: t.join()
print("counter:", counter) # always 400000
counter: 400000
threading.Semaphore(3) to cap concurrency at three. Acquire before the guarded work, release after (a with block does both). The mutex above is just the N=1 case.5 · Deadlock — the four conditions & the banker's algorithm
A deadlock is a cycle of threads each holding a resource the next one needs, so none can proceed. Coffman showed four conditions must all hold for deadlock to be possible: mutual exclusion, hold-and-wait, no preemption, and circular wait. Break any one and deadlock becomes impossible. The banker's algorithm avoids deadlock by only granting a request if the resulting state is still safe — i.e. some ordering exists in which every process can finish.
| Coffman condition | Meaning | One way to break it |
|---|---|---|
| Mutual exclusion | a resource is held exclusively | make resources shareable where possible |
| Hold and wait | hold one, wait for another | request all resources up front |
| No preemption | can't force a release | allow the OS to reclaim resources |
| Circular wait | a cycle of waits | impose a global lock ordering |
bankers.py# MODEL of the banker's algorithm: is the current allocation state SAFE?
# Safe = there exists an order in which all processes can finish.
def is_safe(available, maximum, allocation):
n = len(maximum) # processes
need = [[maximum[i][j] - allocation[i][j] for j in range(len(available))]
for i in range(n)]
work = list(available)
finish = [False] * n
order = []
made_progress = True
while made_progress:
made_progress = False
for i in range(n):
if not finish[i] and all(need[i][j] <= work[j]
for j in range(len(work))):
for j in range(len(work)): # this process can finish; it frees its
work[j] += allocation[i][j] # resources back to the pool
finish[i] = True
order.append(i)
made_progress = True
return all(finish), order
# 3 resource types, 5 processes (classic textbook state)
available = [3, 3, 2]
maximum = [[7,5,3],[3,2,2],[9,0,2],[2,2,2],[4,3,3]]
allocation = [[0,1,0],[2,0,0],[3,0,2],[2,1,1],[0,0,2]]
safe, order = is_safe(available, maximum, allocation)
print("safe?", safe)
print("safe sequence:", order)
safe? True
safe sequence: [1, 3, 4, 0, 2]
6 · Producer–consumer & bounded buffers
A producer thread makes items; a consumer thread uses them; a bounded buffer sits between. The synchronisation problem: the producer must block when the buffer is full, the consumer must block when it is empty, and the buffer itself must not be corrupted by concurrent access. Python's queue.Queue solves all three with an internal lock and condition variables — it is the canonical safe bounded buffer.
producer_consumer.py# REAL threads: Queue(maxsize=...) blocks the producer when full and the
# consumer when empty, and is internally thread-safe.
import threading, queue
buf = queue.Queue(maxsize=5)
DONE = object()
def producer():
for i in range(10):
buf.put(i) # blocks if buffer is full
buf.put(DONE)
def consumer(out):
while True:
item = buf.get() # blocks if buffer is empty
if item is DONE:
break
out.append(item)
out = []
p = threading.Thread(target=producer)
c = threading.Thread(target=consumer, args=(out,))
p.start(); c.start(); p.join(); c.join()
print("consumed:", out)
consumed: [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
✓ Checkpoint — you can move on when you can…
- Explain what a page fault is and trace an address through the page table.
- Run the page-replacement sim and say why optimal beats LRU beats FIFO here.
- Show a race condition and fix it with a mutex.
- State the four Coffman conditions and run the banker's algorithm.
- Implement producer–consumer with a bounded, thread-safe buffer.
Knowledge check
check yourselfOptimal page replacement produces the fewest faults yet no real OS uses it. Why not — and what is it actually useful for?
Show answer
A deadlock needs all four Coffman conditions to hold. A team fixes a deadlock by making every thread acquire locks in a fixed global order (always lock A before B). Which condition does that break, and why does breaking one suffice?
Show answer
🪜 Practice ladder beginner → industry
Six graded exercises, easy to real-world. Try each before opening its solution.
Context: Page-fault counting is the fundamental measurement of a memory system; the reference string is the workload.
Your task: Given a reference string and a frame count, count the page faults under FIFO — the simplest replacement policy.
Requirements:
- A page already resident is a hit (no fault)
- A miss is a fault; if a free frame exists, just load it
- If full, evict the oldest-loaded page (FIFO)
- Return the total fault count for the string
💡 Hint: Model memory as a list; evict index 0, append the new page — that is FIFO order.
Show solution
FIFO is a queue of resident pages — evict the front, append the newcomer:
def fifo_faults(refs, frames):
mem, faults = [], 0
for page in refs:
if page in mem:
continue # hit
faults += 1 # fault
if len(mem) < frames:
mem.append(page)
else:
mem.pop(0) # evict oldest-loaded
mem.append(page)
return faults
print(fifo_faults([7,0,1,2,0,3,0,4,2,3,0,3,2], 3)) # 10Ten faults on this classic string with three frames. FIFO ignores how often a page is used, which is why LRU usually does better.
Context: The three reference policies bracket what is achievable; comparing them on one string shows the value of recency and the unreachable optimum.
Your task: Extend your simulator to also do LRU and optimal, and print all three fault counts for the same reference string and frame count.
Requirements:
- LRU: on a hit, move the page to the most-recently-used position
- LRU eviction: remove the least-recently-used (front) page
- Optimal: evict the page whose next use is furthest in the future (or never)
- Print FIFO, LRU and optimal fault counts side by side
💡 Hint: For optimal, look at refs[i+1:] and pick the resident page with the largest next-use index.
Show solution
One driver, three policies distinguished only by how the victim is chosen:
def simulate(refs, frames, policy):
mem, faults = [], 0
for i, page in enumerate(refs):
if page in mem:
if policy == "LRU":
mem.remove(page); mem.append(page)
continue
faults += 1
if len(mem) < frames:
mem.append(page); continue
if policy in ("FIFO", "LRU"):
victim = mem[0] # oldest / least-recently-used at front
else: # OPTIMAL
future = refs[i+1:]
victim = max(mem, key=lambda p: future.index(p)
if p in future else float("inf"))
mem.remove(victim); mem.append(page)
return faults
refs = [7,0,1,2,0,3,0,4,2,3,0,3,2]
for pol in ("FIFO", "LRU", "OPTIMAL"):
print(pol, simulate(refs, 3, pol))Prints FIFO 10, LRU 9, OPTIMAL 7. Recency (LRU) beats load-order (FIFO); the unreachable optimum (7) is the yardstick for how good either really is.
Context: A lost-update race is the canonical concurrency bug; seeing it and then fixing it with a lock is the core lesson of shared-memory concurrency.
Your task: Write a program where several threads increment a shared counter without a lock (showing it can under-count), then add a mutex and show the count is always correct.
Requirements:
- Run enough iterations that the race can manifest
- Show the unlocked total can be less than the expected total
- Add a mutex around the critical section
- Show the locked total is always exactly correct
- Explain why
counter += 1is not atomic
💡 Hint: The critical section is exactly the read-modify-write of the counter; the lock must span all three sub-steps.
Show solution
The same workload with and without a mutex:
import threading
def run(use_lock):
counter = 0
lock = threading.Lock()
def bump():
nonlocal counter
for _ in range(200_000):
if use_lock:
with lock:
counter += 1
else:
counter += 1
ts = [threading.Thread(target=bump) for _ in range(4)]
for t in ts: t.start()
for t in ts: t.join()
return counter
expected = 4 * 200_000
print("no lock:", run(False), "expected", expected) # may be < expected
print("with lock:", run(True), "expected", expected) # always == expectedcounter += 1 compiles to read, add, write; without the lock two threads can read the same value and one increment is lost, so the unlocked run can fall short (it varies per run, and the GIL can sometimes mask it). The mutex makes the three sub-steps indivisible, so the locked run is always exact.
Context: The banker's algorithm is deadlock avoidance in practice: it only grants a resource request if the resulting state can still be completed by some ordering.
Your task: Implement a request check that tentatively grants a resource request, runs the safety check, and grants only if the resulting state is safe — otherwise rolls back and denies.
Requirements:
- Reject immediately if the request exceeds the process's declared need
- Reject if the request exceeds currently available resources
- Tentatively apply the grant, then run the safety check
- Grant if safe; otherwise roll back and deny
- Demonstrate one granted and one denied request
💡 Hint: Deep-copy (or carefully restore) the state before the tentative grant so a denied request leaves everything unchanged.
Show solution
Request handling wraps the safety check with two fast rejections and a tentative grant:
import copy
def is_safe(available, maximum, allocation):
n, m = len(maximum), len(available)
need = [[maximum[i][j]-allocation[i][j] for j in range(m)] for i in range(n)]
work, finish = list(available), [False]*n
progress = True
while progress:
progress = False
for i in range(n):
if not finish[i] and all(need[i][j] <= work[j] for j in range(m)):
for j in range(m): work[j] += allocation[i][j]
finish[i] = True; progress = True
return all(finish)
def request(pid, req, available, maximum, allocation):
m = len(available)
need = [maximum[pid][j]-allocation[pid][j] for j in range(m)]
if any(req[j] > need[j] for j in range(m)):
return "DENY: exceeds declared need"
if any(req[j] > available[j] for j in range(m)):
return "DENY: not enough available now"
av2 = [available[j]-req[j] for j in range(m)]
al2 = copy.deepcopy(allocation)
for j in range(m): al2[pid][j] += req[j]
return "GRANT" if is_safe(av2, maximum, al2) else "DENY: would be unsafe"
available = [3,3,2]
maximum = [[7,5,3],[3,2,2],[9,0,2],[2,2,2],[4,3,3]]
allocation = [[0,1,0],[2,0,0],[3,0,2],[2,1,1],[0,0,2]]
print("P1 asks [1,0,2]:", request(1, [1,0,2], available, maximum, allocation))
print("P0 asks [0,2,2]:", request(0, [0,2,2], available, maximum, allocation))Prints P1 asks [1,0,2]: GRANT (the post-grant state is still safe) and P0 asks [0,2,2]: DENY: would be unsafe. The tentative state is built on copies, so a denied request leaves the real allocation untouched — that is deadlock avoidance: never enter a state you cannot guarantee to exit.
Context: Bounding concurrency is a daily production need — connection pools, download caps, rate control. A counting semaphore is the primitive.
Your task: Build a limiter that lets at most N worker threads into a guarded region at once, and record the maximum observed concurrency to prove the cap holds.
Requirements:
- Use a semaphore initialised to N
- Track a live in-region counter and the maximum it reaches
- Run many more than N workers so the cap is actually exercised
- Assert the observed maximum never exceeds N
- Explain how a mutex is the N=1 special case
💡 Hint: Guard the in-region counter with its own small lock so the max is recorded correctly under contention.
Show solution
A semaphore caps how many of the 50 workers are inside the region at once:
import threading, time, random
N = 3
sem = threading.Semaphore(N)
live = 0
peak = 0
meter = threading.Lock()
def worker():
global live, peak
with sem: # at most N threads past this point
with meter:
live += 1; peak = max(peak, live)
time.sleep(random.uniform(0.005, 0.02))
with meter:
live -= 1
ts = [threading.Thread(target=worker) for _ in range(50)]
for t in ts: t.start()
for t in ts: t.join()
print("peak concurrency:", peak, "cap:", N)
assert peak <= NThe peak never exceeds N because the semaphore admits at most N holders; a separate small lock protects the meter so the peak is recorded correctly under contention. A mutex is exactly this with N = 1.
Context: Real services run a producer feeding a pool of consumers through a bounded queue, and must shut down cleanly without losing or double-processing work.
Your task: Build a bounded producer/multi-consumer pipeline that processes every item exactly once and shuts every consumer down cleanly after the work is exhausted.
Requirements:
- Use a bounded thread-safe queue so a fast producer applies backpressure
- Fan out to several consumer threads
- Ensure every item is processed exactly once (no loss, no duplication)
- Use a sentinel (one per consumer) or task-done tracking to signal shutdown
- Show the union of consumer outputs equals the produced set
💡 Hint: One sentinel per consumer is the simplest correct shutdown: each consumer that sees a sentinel exits, so N sentinels stop N consumers.
Show solution
A bounded queue, a pool of consumers, and one sentinel per consumer for a clean shutdown:
import threading, queue
NUM_CONSUMERS = 3
work = queue.Queue(maxsize=8) # bounded -> backpressure on the producer
results = []
results_lock = threading.Lock()
SENTINEL = object()
def producer(n):
for i in range(n):
work.put(i) # blocks when full
for _ in range(NUM_CONSUMERS):
work.put(SENTINEL) # one poison pill per consumer
def consumer():
while True:
item = work.get()
if item is SENTINEL:
work.task_done(); break
with results_lock:
results.append(item * item)
work.task_done()
N = 100
p = threading.Thread(target=producer, args=(N,))
cs = [threading.Thread(target=consumer) for _ in range(NUM_CONSUMERS)]
p.start()
for c in cs: c.start()
p.join()
for c in cs: c.join()
print("processed:", len(results), "of", N)
print("exactly once:", sorted(results) == [i*i for i in range(N)])Prints processed: 100 of 100 and exactly once: True. The bounded queue applies backpressure so a fast producer cannot exhaust memory; the per-consumer sentinels guarantee every worker stops exactly once and no real item is mistaken for a shutdown signal. This is the shape of every production work-queue.