Computer organization

1 · The von Neumann architecture essential

The von Neumann architecture (1945) is the blueprint for almost every computer: a single memory holds both instructions and data (the stored-program idea), a CPU reads and executes them, and I/O connects to the outside world. Because code is just data in memory, a program can be loaded, replaced, even generated at runtime — that is why software is soft.

2 · Inside the CPU essential

Registers are to the ALU what your hands are to you: the only place work actually happens. Everything else is about getting data into a register, operating, and putting the result back.

3 · Fetch–decode–execute: simulate a tiny CPU advanced

The CPU runs one loop forever: fetch the instruction at the PC, decode what it means, execute it, advance the PC, repeat. Let's build a working (model) CPU with 5 instructions and run a program that computes (3 + 4) × 2 and stores it.

tiny_cpu.py# Instruction set: (op, arg). Registers: ACC (accumulator) + a small RAM.
#   LOAD  v  -> ACC = v
#   ADD   v  -> ACC = ACC + v
#   MUL   v  -> ACC = ACC * v
#   STORE a  -> RAM[a] = ACC
#   HALT     -> stop
program = [
    ("LOAD", 3),
    ("ADD", 4),
    ("MUL", 2),
    ("STORE", 0),
    ("HALT", None),
]

def run(program):
    PC, ACC, RAM = 0, 0, [None] * 8       # program counter, accumulator, memory
    while True:
        op, arg = program[PC]             # FETCH
        # DECODE + EXECUTE
        if op == "LOAD":  ACC = arg
        elif op == "ADD": ACC += arg
        elif op == "MUL": ACC *= arg
        elif op == "STORE": RAM[arg] = ACC
        elif op == "HALT": break
        print(f"PC={PC} {op:<5} arg={arg}  -> ACC={ACC}")
        PC += 1                            # advance
    return RAM

ram = run(program)
print("RAM[0] =", ram[0])

PC=0 LOAD  arg=3  -> ACC=3
PC=1 ADD   arg=4  -> ACC=7
PC=2 MUL   arg=2  -> ACC=14
PC=3 STORE arg=0  -> ACC=14
RAM[0] = 14

4 · The memory hierarchy intermediate

There is no single memory that is both huge and instant, so computers stack layers: tiny-and-fast at the top, huge-and-slow at the bottom. The CPU checks the fast layers first. The numbers below are approximate orders of magnitude and vary widely by hardware — treat them as ratios, not guarantees.

5 · Caching & locality of reference advanced

A cache keeps recently/nearby-used data in a fast layer, betting on locality: temporal (you'll reuse what you just used) and spatial (you'll use data next to what you just used). Caches load a whole cache line (e.g. 64 bytes) at once, so touching neighbours is nearly free — but jumping around memory pays the full RAM latency each time.

The canonical demonstration: summing a 2-D array row-major (along rows, the way it's laid out in memory) touches consecutive addresses and rides the cache lines; column-major jumps by a whole row each step and thrashes the cache. The code below measures both on your machine:

locality.pyimport time

N = 2000
# a flat list acts as an N x N matrix, row-major: element (r, c) at r*N + c
matrix = [0] * (N * N)

def sum_row_major():
    s = 0
    for r in range(N):
        base = r * N
        for c in range(N):
            s += matrix[base + c]        # consecutive addresses (cache-friendly)
    return s

def sum_col_major():
    s = 0
    for c in range(N):
        for r in range(N):
            s += matrix[r * N + c]       # jumps N elements each step (cache-hostile)
    return s

for name, fn in (("row-major", sum_row_major), ("col-major", sum_col_major)):
    t0 = time.perf_counter()
    fn()
    print(f"{name}: {time.perf_counter() - t0:.3f} s")
# Row-major is typically faster. Exact numbers depend on your CPU,
# cache sizes, and Python build -- run it and compare the two.

6 · A taste of assembly professional

High-level code compiles down to assembly — human-readable names for the CPU's raw instructions. Here is roughly what c = a + b becomes on a register machine (x86-flavoured pseudocode). Notice it is exactly our tiny CPU: move into a register, add, store back.

add.asmmov  eax, [a]     ; load variable a from memory into register eax
add  eax, [b]     ; eax = eax + b   (the ALU does the add)
mov  [c], eax     ; store the register result back into variable c

Compare to our toy CPU: mov eax,[a] is LOAD, add eax,[b] is ADD, mov [c],eax is STORE. Real ISAs (x86, ARM, RISC-V) have hundreds of instructions, but the shape — move to register, compute, store — is the same one you simulated.

Knowledge check check yourself

# PC=0 LOAD 5 -> ACC=5
# PC=1 ADD 10 -> ACC=15
# PC=2 ADD 10 -> ACC=25
# PC=3 HALT    -> stop
print('PC path: 0,1,2,3  final ACC = 25')

program = [('LOAD',20),('SUB',5),('SUB',3),('HALT',None)]
def run(prog):
    PC, ACC = 0, 0
    while True:
        op, arg = prog[PC]
        if   op=='LOAD': ACC = arg
        elif op=='ADD':  ACC += arg
        elif op=='SUB':  ACC -= arg
        elif op=='HALT': break
        PC += 1
    return ACC
print(run(program))   # 12

prog = [
    ('LOAD', 3),   # 0
    ('PRINT', None), # 1
    ('SUB', 1),    # 2
    ('JZ', 6),     # 3  if ACC==0 jump to HALT
    ('JMP', 1),    # 4  else loop back to PRINT
    ('NOP', None), # 5
    ('HALT', None),# 6
]
def run(p):
    PC, ACC = 0, 0
    while True:
        op, arg = p[PC]
        if   op=='LOAD':  ACC = arg
        elif op=='SUB':   ACC -= arg
        elif op=='PRINT': print('ACC =', ACC)
        elif op=='JZ':
            if ACC == 0: PC = arg; continue
        elif op=='JMP':   PC = arg; continue
        elif op=='HALT':  break
        PC += 1
run(prog)   # prints ACC = 3, then 2, then 1

import time
N = 2000
m = [0] * (N*N)
def row():
    s = 0
    for r in range(N):
        b = r*N
        for c in range(N): s += m[b+c]
    return s
def col():
    s = 0
    for c in range(N):
        for r in range(N): s += m[r*N+c]
    return s
row(); col()                       # warm up
t=time.perf_counter(); row(); tr=time.perf_counter()-t
t=time.perf_counter(); col(); tc=time.perf_counter()-t
print(f'row {tr:.3f}s  col {tc:.3f}s  ratio {tc/tr:.2f}x')
# absolute times are machine-dependent; expect ratio > 1 (row faster)

# SLOW: inner loop jumps a full row each step
def copy_slow(src, dst, N):
    for c in range(N):
        for r in range(N):
            dst[r*N + c] = src[r*N + c]

# FAST: inner loop walks consecutive addresses
def copy_fast(src, dst, N):
    for r in range(N):
        base = r*N
        for c in range(N):
            dst[base + c] = src[base + c]

N = 4
src = list(range(N*N)); dst = [0]*(N*N)
copy_fast(src, dst, N)
print(dst == src)   # True -- same result, cache-friendly order

# Approximate orders of magnitude (machine-dependent):
#   L1 hit   ~1 ns        RAM     ~100 ns
#   L3 hit   ~10 ns       SSD     ~10-100 us
#   disk seek ~ms         network ~ms+
#
# Reasoning:
# * A 2-3x CPU-bound slowdown with no I/O change -> likely locality/cache
#   (data grew past L2/L3, or an access pattern regressed). Fix: profile,
#   check working-set size and loop/access order.
# * A jump from ~us to ~ms in p99 -> crossed into disk or network territory
#   (cache miss on a hot dataset, an added remote call, a slow query).
#   Fix: trace the request, look for new I/O, check cache hit-rate.
print('classify by the MAGNITUDE of the regression, then trace that layer')