AI EngineeringZero to ProductionHome·About·What’s new·Contact
Frontier Agent Capabilities · Part 4

Long-term agent memory

Chat models are stateless — the window forgets. Long-term memory is a store you build around the model: extract facts from a conversation, write them down, and retrieve the relevant ones on the next turn. This lesson builds that loop from scratch, then shows what Mem0 and Letta/MemGPT add on top.

⏱️ ~1.5 hours🧪 4 labs🎯 Beginner→Tech-lead

Learning objectives

  • Explain why a context window is not memory (the model is stateless).
  • Distinguish short-term vs long-term, and episodic vs semantic memory.
  • Trace the memory loop: extract facts → write → retrieve → inject.
  • Implement a tiny MemoryStore with write/retrieve, ranking, and forgetting.
  • Describe what Mem0 and Letta/MemGPT add, and self-editing memory.
  • Own the production concerns: privacy, staleness, and cost of stored memory.
📋 Before you startThis page builds on: Agent & tool foundations · Embeddings & vector databases. Jumping in? Skim those first — or open the recap.
30-second recap
  • Agent & tool foundations: The model asks to call a tool, your code runs it and returns the result, and the loop repeats until the model answers — with step caps so it can't run forever.
  • Embeddings & vector databases: An embedding turns text into a vector so that similar meanings sit close together; a vector database finds the nearest vectors to a query fast.

1 · The window is not memory essential

Here's a moment every builder of chat apps eventually lives. You demo your assistant to a colleague. "My name is Dana," they type, and the bot cheerfully greets Dana. Twenty messages later they ask "so what's my name again?" — and the model has no idea. It didn't get tired or confused; the earlier turn simply fell out of the window, and to the model, anything not in front of it right now never happened. The instinct is to call this a bug. It isn't. It's the single most important fact about how these systems work, and misunderstanding it is why so many "smart" agents feel weirdly amnesiac.

The analogy that makes it click: a chat model is like a brilliant consultant with no long-term memory who reads a fresh briefing document before every single meeting and then, the instant the meeting ends, has the document shredded. Each turn, the consultant is dazzlingly capable — but only about what's written on today's page. There is no "remembering yesterday." There is only "what's on the page this time."

That page is the context window, and the precise word for the model's condition is stateless: between two API calls it carries nothing forward — no hidden notebook, no running memory of you. Everything it appears to "know" in a turn is text you physically re-sent in that request. Chat apps hide this by re-pasting the entire transcript on every turn, which fakes continuity beautifully — right up until the conversation grows past the window's token budget, or spans multiple sessions, or you have ten thousand users each with months of history. Then the illusion breaks, because you cannot re-send what no longer fits.

So here's the reframe this whole lesson turns on: memory is not something the model has — it's something you build around it. A chat model is stateless between calls: it remembers nothing on its own. Everything it "knows" in a turn is whatever text you put in the context window right now — the same statelessness you first met in Ch 1 (setup). Chat apps fake memory by re-sending the whole transcript every turn. That works until the conversation outgrows the window, spans sessions, or you have thousands of users — then the transcript can't hold it all. Long-term memory is a separate store you write facts to and read facts from, so the agent can recall things from last week without re-sending last week.

early turn: everything still fits "I'm Dana" turn 2 turn 3 stateless model the window (a fixed token budget) — anything outside it never happened to the model many turns later: the window is full "I'm Dana" turn 18 turn 19 "my name?" first turn pushed out of the window → forgotten The window forgets because it fills. Chat apps re-send the whole transcript each turn (top); once it outgrows the token budget the earliest turn drops out (bottom, faded) — so "what's my name?" now has nothing to answer from.
🗺️ How to read this diagram

Compare the two rows top-to-bottom: the same conversation early (everything fits) versus late (the window is full).

  • The top row is an early turn — "I'm Dana" plus a couple of turns all fit, and the whole strip is re-sent to the stateless model on the right, which carries nothing between calls of its own.
  • The label between the rows is the window: a fixed token budget. Text inside it is all the model can see; anything outside it, to the model, never happened.
  • The bottom row is many turns later. The window is full of recent turns (red), so the faded "I'm Dana" box has been pushed out — when the user asks "my name?", that fact is no longer in front of the model to answer from.

In short: statelessness + a finite window means the earliest turns silently fall out — which is exactly the gap a durable memory store fills.

Recall from Ch 1The model does not "learn" from a conversation. Close the tab and it forgets. Memory is something you build around the stateless model — not a feature of the model.

2 · Short-term vs long-term memory essential

You already own the perfect analogy for this — it's your own desk. Right now, the papers spread in front of you are what you're actively working with: instantly reachable, but there's only so much surface area, and when you clear the desk at the end of the day, they're gone. Behind you sits a filing cabinet: enormous, slow to reach into, but it survives the night and holds years of folders. You don't work out of the cabinet — you fetch the one folder you need onto the desk, use it, and file it back. An agent's memory splits along exactly this line, and confusing the two is where designs go wrong.

Short-term memory is the desk: the context window. It's this turn's transcript — cheap to use because it's already in front of the model, but strictly bounded by the token budget and wiped when the window fills or the session ends. Long-term memory is the filing cabinet: a durable external store — a file, a database, a vector index — that outlives any single session and, crucially, only ever loads the folders relevant to this conversation onto the desk.

The mistake beginners make is treating the window as the memory — just making it bigger. "The model has a 200k-token window, so it can remember everything!" But a bigger desk is still a desk: it clears every session, every token you stuff in costs money and dilutes the model's attention, and it does nothing for user #4,001 who starts a brand-new conversation tomorrow. The two are different kinds of thing, not two sizes of the same thing. Short-term is working attention (fast, tiny, transient); long-term is storage (slow, vast, persistent). Every real agent needs both, and the engineering skill is deciding what lives where — and, on each turn, which folders to pull from the cabinet onto the desk. The table below lines up the two on the properties that actually drive that decision.

Short-term memory is the current context window: this turn's transcript, cheap to use, gone when the window fills or the session ends. Long-term memory is a durable external store (a file, a DB, a vector index) that survives across sessions and only loads what's relevant to this turn. The window is your desk; long-term memory is the filing cabinet you fetch the right folder from.

short-term = the desk context window this turn's transcript fast · tiny · wiped each session long-term = the filing cabinet durable store relevant folder slow · vast · persists fetch just this folder onto the desk Desk vs cabinet — two kinds of thing, not two sizes. The window (left) is small, fast and cleared each session; the store (right) is vast and durable. Each turn you pull only the one relevant folder (green) onto the desk, not the whole cabinet.
🗺️ How to read this diagram

Read it left-to-right: the small box is what the model works out of right now; the tall box is where everything is kept; the arrow is the move between them.

  • The left box (the desk) is short-term memory — the context window. It's fast because it's already in front of the model, but small and wiped when the session ends.
  • The right box (the cabinet) is long-term memory — a durable external store holding many folders that survive across sessions.
  • The blue arrow is the key move: each turn you fetch only the green "relevant folder" onto the desk. You never work out of the cabinet directly, and you never load all of it.

In short: a bigger desk is still a desk — persistence needs the cabinet, and the skill is deciding which one folder to fetch each turn.

Short-term (window)Long-term (store)
Lives inthe context windowexternal DB / index
Survives session?noyes
Size limitthe token budgeteffectively unbounded
Accessalready in-contextmust retrieve + inject

3 · Episodic vs semantic memory essential

Think about how you remember a friend. You know they're a vegetarian, they live in Lisbon, and they hate phone calls — those are settled facts you'd state flatly without reliving anything. But you also remember "last month they told me over dinner they'd just quit their job" — a specific event, anchored in time. These are two genuinely different kinds of remembering, and cognitive scientists gave them names decades ago: semantic memory (timeless facts) and episodic memory (dated events). Agents that store memory well borrow the same split — and agents that store it badly are usually the ones that collapsed everything into one undifferentiated pile.

Episodic memory is events, stamped with when. "On Tuesday the user asked to cancel order #42." It's raw, append-only, and you never edit it — it's a log of what happened, and rewriting history would defeat the purpose. Semantic memory is distilled facts, stripped of the moment they came from. "The user prefers email over phone." It's stable, deduplicated, and it does get updated when the truth changes. Notice the direction of flow: facts are born from episodes. The system watches the user ask for email three separate times (three episodes) and eventually writes down the durable conclusion "prefers email" (one semantic fact) — a process called consolidation, exactly what your own brain does when repeated experiences harden into "things you just know."

Why keep both instead of just the tidy facts? Because they answer different questions and carry different risks. Episodes give you traceable, auditable context — when a customer disputes a charge, "on 3 Sep they requested a refund" is evidence, and a distilled fact isn't. Semantic facts give you compact, reusable signal — you can't inject a thousand raw episodes into every prompt, but you can inject five clean preferences. Drop the episodes and you lose the audit trail and the ability to re-derive facts when your extraction improves; drop the semantic layer and every turn drowns in transcript. The common beginner mistake is storing only episodes ("just log everything") and then wondering why retrieval is noisy and expensive — you skipped the distillation step that makes memory useful rather than merely complete. The table contrasts the two on the properties that decide how you store and update each.

Borrowing from cognitive science, split long-term memory two ways. Episodic memory is events: "on Tuesday the user asked to cancel order #42." Semantic memory is distilled facts: "the user prefers email over phone." Episodes are raw and timestamped; facts are stable and deduplicated. A good agent writes episodes as they happen and, over time, consolidates them into durable semantic facts.

episodic — raw events, stamped with when (append-only) Tue: user asks for the email version Thu: user asks "email me instead" Sat: user replies only by email consolidate semantic fact "prefers email" stable · deduped · updated keep the events (audit trail) AND the fact (compact, injectable) Facts are born from episodes. Three separate timestamped events (amber) about the same behaviour get distilled into one durable, deduplicated semantic fact (green) — the process called consolidation. Keep both: episodes are the audit trail, the fact is what you inject.
🗺️ How to read this diagram

Read it left-to-right: many raw events on the left funnel through "consolidate" into one clean fact on the right.

  • The three amber boxes are episodic memories — specific events, each stamped with when it happened. They are append-only: you log them and never edit them.
  • The arrows converging on the right are consolidation: the system notices the same behaviour recurring across those events.
  • The green box is the distilled semantic fact — "prefers email" — stripped of any single moment, deduplicated, and updated if the truth later changes. It's the compact thing you inject into a prompt; you can't inject a thousand raw episodes.

In short: episodes are the evidence log; the semantic fact is the reusable conclusion drawn from them — keep both, because they answer different questions.

KindStoresExampleChanges
Episodicevents, with time"asked for a refund on 3 Sep"append-only log
Semanticdistilled facts"is a premium-tier customer"updated / deduped

4 · The memory loop intermediate

Here's the good news that de-mystifies this entire field. "Agent memory" sounds like it might need some exotic neural mechanism, some way to actually reach inside the model and change it. It doesn't. Every memory system you'll ever meet — the flashy commercial ones like Mem0 and Letta, and the fifty-line version you'll build in the next lab — is the same four-step loop wrapped around an unchanged, still-stateless model. Once you see the loop, every product in this space stops looking magical and starts looking like a variation on one recipe.

The loop, in one breath: from each turn you extract what's worth keeping, write it to the durable store; then on a later turn you retrieve the memories relevant to the new message and inject them into the context. Extract → write → retrieve → inject. That's the whole thing. The model contributes intelligence in the middle; the loop contributes continuity around the edges.

The cleanest way to hold it in your head is to split those four steps into two halves that run at different times. The write path (extract → write) runs after a turn: it's the librarian filing away what just happened. The read path (retrieve → inject) runs before the next turn: it's the librarian pulling the right folders back out. This timing split matters more than it looks — writing is where you decide what's worth remembering (get it wrong and you store noise or leak secrets), and reading is where you decide what's worth surfacing (get it wrong and the model either forgets or drowns).

And here's the connection that should make you feel at home: the read path is just RAG. Retrieving the memories that match the current message is identical to retrieving the document chunks that match a query in Chapter 3 — same embeddings, same similarity search, same top-k. The only difference is the corpus: instead of a library of documents, you're searching the user's own past. Long-term memory is RAG pointed at yourself. Everything you already learned about retrieval quality transfers directly. The diagram traces one full pass; notice how it loops back to feed the next turn.

Every memory framework — Mem0, Letta, home-grown — is a loop. From each turn you extract what's worth keeping, write it to the long-term store; on the next turn you retrieve the memories relevant to the new message and inject them into the context. The model stays stateless; the loop supplies the state.

Conversation a turn Extract facts what to keep Write to store durable Retrieve relevant by similarity Inject to context next turn
🗺️ How to read this diagram

This is the whole lesson in one picture — the loop that gives a stateless model memory. Read it left to right, then notice it feeds back to the start on the next turn.

  • Conversation — a single turn happens. The model itself will forget it the moment the call ends, so we act before that.
  • Extract facts — we pull out only the durable, worth-keeping bits (a preference, a name), not the whole transcript. This is the write path's "what to remember" decision.
  • Write to store — those facts go into a durable external store (a DB or index) that outlives the session.
  • Retrieve relevant — on a later turn, we search that store for the memories that match the new message, by similarity — exactly retrieval from Ch 3, but over the user's own past.
  • Inject into context — the retrieved memories are pasted into the prompt, so the still-stateless model "remembers." Then the cycle repeats for the next turn.

In short: the model never changes — memory is the loop of extract → write → retrieve → inject that you build around it.

Note the read path is exactly retrieval (Ch 3 · RAG) — but the corpus is the user's own past, not a doc library. Memory is RAG over yourself.

5 · A tiny MemoryStore (write + retrieve) intermediate

Theory has a way of feeling profound until you try to build the thing — and then it either survives contact with code or it doesn't. So let's build the smallest honest memory store that actually works, with nothing but the Python standard library: no vector database, no embedding model, no network. If you've ever assumed "agent memory" requires a fancy infrastructure stack, this section is the antidote — the core is astonishingly small, and seeing it small is what lets you reason about the big versions later.

A store needs exactly two operations, mirroring the two halves of the loop: write(text) to save a memory, and retrieve(query, k) to fetch the k memories most relevant to a new message. Write is trivial — append it. Retrieve is where the thinking lives, and it boils down to one question: given a new message, how do I rank which stored memories matter?

The simplest possible ranker is bag-of-words overlap: treat each memory and the query as a bag of words, and score by how many meaningful words they share. "How do I contact the user?" and "the user prefers email over phone" share contact-ish vocabulary; a memory about Berlin shares nothing, so it's ranked out. One subtlety carries the whole idea, though — you must first drop stopwords ("the", "is", "a", "user"). Skip that and every memory shares the filler words "the user," so everything looks equally relevant and your ranking becomes noise. Filtering to content words is what turns raw overlap into real signal.

Be clear about what this is and isn't. Word overlap is a deliberate stand-in for embedding similarity — it only matches memories that reuse the exact same words, so "contact method" wouldn't match "reach the user by." Production systems swap this scorer for cosine similarity over embeddings (the Chapter 3 machinery), which matches on meaning rather than spelling. But the shape of the code — the write/retrieve seam, the ranking, the top-k slice — is identical. Get the seam right with the toy scorer and upgrading to embeddings is a one-function change, not a rewrite. That is exactly why we start here.

The read path scores each stored memory against the new message and returns the best matches. The simplest scorer is bag-of-words overlap: how many words do they share? (Real systems use embedding similarity — same idea, better matching; see Ch 3.) This runs offline with the standard library only.

new message "contact by email?" score each stored memory by shared content words "prefers email over phone" 2 "is a premium-tier customer" 0 "lives in Berlin" 0 top-k: only the highest-scoring memory is injected Retrieve = rank, then slice. The query is scored against every memory by how many meaningful words they share; the one real match (green, score 2) comes back, the zero-overlap memories stay out. Inject a handful, not the whole store.
🗺️ How to read this diagram

Follow the arrows left-to-right: one query is compared against each stored memory, and each comparison yields a score.

  • The left box is the new message. Arrows fan out to every stored memory — retrieval scores all of them.
  • The numbers on the right are the overlap scores: shared content words after stopwords are dropped. "prefers email over phone" shares email → score 2; the Berlin and tier memories share nothing → score 0.
  • The green memory is the winner. Sorting by score and taking the top-k keeps only relevant memories — the point of the "top-k slice".

In short: score every memory against the query, keep the top few — the same rank-and-slice shape RAG uses, here over the user's own facts.

Example code for learning — review, test, and adapt it before running against real or production systems. Commands can create, change, or delete resources. See the Terms & Disclaimer.
Python · MemoryStore.write / retrieve, overlap scoring (runs)
memory_store.pyimport re

# stopwords carry no signal; drop them so overlap ranks on meaningful words
STOP = {"the", "a", "an", "i", "is", "of", "on", "to", "how", "should",
        "this", "what", "over", "in", "user", "customer"}

def tokenize(text):
    return {w for w in re.findall(r"[a-z0-9]+", text.lower()) if w not in STOP}

class MemoryStore:
    def __init__(self):
        self.memories = []          # list of {"text":..., "toks":...}

    def write(self, text):
        self.memories.append({"text": text, "toks": tokenize(text)})

    def retrieve(self, query, k=2):
        q = tokenize(query)
        scored = []
        for m in self.memories:
            overlap = len(q & m["toks"])          # shared meaningful words
            if overlap:
                scored.append((overlap, m["text"]))
        scored.sort(key=lambda x: (-x[0], x[1]))  # best first, stable
        return [text for _, text in scored[:k]]

store = MemoryStore()
store.write("the user prefers email over phone")
store.write("the user is a premium-tier customer")
store.write("the user lives in Berlin")

print(store.retrieve("how should I contact the user by phone or email?"))
print(store.retrieve("what tier is this customer on, premium?"))
['the user prefers email over phone']
['the user is a premium-tier customer']
▶ How this works

This is the read path in miniature: a store you can write() facts into and retrieve() the relevant ones from. The scoring is deliberately simple so you can see the idea before reaching for embeddings.

  1. tokenize() turns text into a set of words and drops STOP words like "the", "is", "user". Without that, every memory shares "the user" and everything looks equally relevant — the stopword filter is what makes ranking work.
  2. write() just stores the text plus its pre-computed word set, so retrieval is fast.
  3. retrieve() tokenizes the query, counts overlapping meaningful words (q & m["toks"] is set intersection) per memory, sorts best-first, and returns the top k.
  4. Real systems swap word-overlap for embedding similarity (Ch 3) — same shape, better matching on meaning rather than exact words.

What the output means: Only the memory that actually shares content words comes back: the phone/email query returns the contact preference; the tier query returns the premium fact. The "Berlin" memory stays out of both — that selectivity is the point.

Try this: Add store.write("the user speaks German") and query "what language does the user speak?". Then add "language" to STOP and watch that match disappear — proof the stopword list controls what counts as signal.

Only relevant memories come back — the "Berlin" fact stays out of a contact-method query. That selectivity is the whole point: you inject a handful of relevant memories, not the entire history.

6 · The write path — what & when to remember advanced

Now the question that quietly decides whether your memory helps or hurts. The read path is the glamorous half — similarity search, top-k, embeddings. But the write path is where memory systems actually live or die, and it's easy to get lazy here. The lazy version is "remember everything the user says." Follow that instinct and you'll rediscover a painful truth: remembering everything is the same as remembering nothing. You've simply rebuilt the full transcript in your store, so retrieval now sifts through mountains of "Hi!", "thanks", "ok cool" to find the one fact that mattered. A store full of chit-chat retrieves worse than a small store of real facts.

So the write path has to make two judgment calls on every turn: what is worth keeping, and when to keep it. On what: keep durable facts and preferences ("I'm a data engineer," "my name is Dana," "I prefer dark mode") — things that will still be true and useful next week. Throw away the perishable: greetings, acknowledgments, the mechanics of the conversation itself. A good filter is to ask "would this still be worth injecting into a prompt a month from now?" If not, don't store it. On when: extraction usually runs right after each turn, but you might also consolidate periodically (§7) rather than on every message.

How do you actually extract facts? In production, you ask an LLM: "here's a user turn — pull out any durable facts as short statements, or return nothing." The model's language understanding makes it flexible and forgiving of phrasing. But to keep this lesson offline and deterministic, the lab uses a rule-based stub — a handful of regular expressions that catch "I am …", "I prefer …", "my name is …" and reject openers like "hi" and "thanks." It's cruder than an LLM (it only catches the patterns you wrote), but it makes the shape of extraction visible: input a turn, output zero or more normalized fact strings. Swapping in an LLM later changes the extractor's brain, not the plumbing around it. The lab also shows the payoff — the extracted facts formatted into a "System memory about the user" block, which is the inject step: this is the text that will prefix the next prompt so the stateless model appears to remember.

You can't store every sentence — that's just the transcript again. The write path decides what is worth keeping (durable facts and preferences, not chit-chat) and when. Production systems ask an LLM to extract facts; here is a rule-based stub that captures the shape, runs offline, and shows the recall being injected into a prompt.

a turn's text "my name is Dana" "I prefer dark mode" "Hi there!" "Thanks!" write filter keep or drop? store (kept) name · preference discarded: chit-chat "would this still be worth injecting a month from now?" Remember everything = remember nothing. The write filter keeps durable facts and preferences (green) and throws away perishable chit-chat (red) — so retrieval later sifts real facts, not a re-created transcript.
🗺️ How to read this diagram

Follow the turn's text left-to-right through the filter, which routes each line to one of two destinations.

  • The left column is everything the user said this turn — a mix of durable statements ("my name is Dana", "I prefer dark mode") and filler ("Hi there!", "Thanks!").
  • The middle box is the write filter. It asks one question of each line: is this durable? The rule-based stub matches "I am…/I prefer…/my name is…" and rejects openers.
  • The solid arrow to the green box stores the keepers; the dashed arrow to the red box drops the chit-chat. Only durable facts survive to be retrieved and injected later.

In short: the write path's job is selection, not capture — keep what's still useful next week, discard the rest, or retrieval drowns in noise.

Python · rule-based fact extraction + inject (runs)
fact_extract.pyimport re

def extract_facts(utterance):
    """Rule-based stub: keep durable 'I ...' statements, drop chit-chat."""
    u = utterance.strip()
    facts = []
    low = u.lower()
    if low.startswith(("hi", "hello", "thanks", "thank you", "bye")):
        return facts                      # chit-chat: remember nothing
    m = re.match(r"i (?:am|'m) (?:a |an )?(.+)", low)
    if m:
        facts.append("user is " + m.group(1).rstrip("."))
    m = re.search(r"i (?:prefer|like|want) (.+)", low)
    if m:
        facts.append("user prefers " + m.group(1).rstrip("."))
    m = re.search(r"my name is (\w+)", low)
    if m:
        facts.append("user's name is " + m.group(1).capitalize())
    return facts

turns = [
    "Hi there!",
    "My name is Dana",
    "I am a data engineer",
    "I prefer dark mode",
    "Thanks!",
]
memory = []
for t in turns:
    for f in extract_facts(t):
        if f not in memory:
            memory.append(f)

print("stored:", memory)
prompt = "System memory about the user:\n- " + "\n- ".join(memory)
print(prompt)
stored: ["user's name is Dana", 'user is data engineer', 'user prefers dark mode']
System memory about the user:
- user's name is Dana
- user is data engineer
- user prefers dark mode
▶ How this works

This is the write path's other half: deciding what to keep. You can't store every sentence (that's just the transcript again), so extract_facts() keeps durable statements and throws away chit-chat.

  1. The first check returns an empty list for greetings and thanks — chit-chat is not remembered. Storing "Hi there!" forever would be noise.
  2. Each re.match/re.search looks for a durable pattern — "I am …", "I prefer …", "my name is …" — and turns it into a normalized fact string.
  3. The loop over turns collects facts, skipping duplicates (if f not in memory), building the long-term store one fact at a time.
  4. Finally the facts are formatted into a System memory about the user: block — this is the inject step: memories pasted into the prompt the model will see.

What the output means: "Hi there!" and "Thanks!" store nothing; the three real statements become three facts, then get rendered into the system-memory block that would prefix the next prompt.

Try this: Add a turn "I want dark roast coffee" and one more greeting. Only the preference should be stored. This rule-based stub is what an LLM does for real in production — same job, fuzzier matching.

Privacy: you are now storing personal dataThe moment you write memories you hold personal data across sessions. Store the minimum, let users view and delete their memories, honor deletion requests, and never persist secrets or payment details. A memory store is a data-protection surface (GDPR "right to be forgotten"), not just a cache.

7 · Forgetting & consolidation advanced

Picture the store six months into production. It has "user prefers phone" written in January and "user prefers email" written in June. Both are sitting there. On the next turn, retrieval dutifully hauls back both, and now you've injected a flat contradiction into the prompt — the agent confidently offers to call a user who explicitly asked for email. Nobody wrote a bug; the store just accumulated, and accumulation without maintenance rots into slowness, cost, and self-contradiction. This is the failure mode that separates a demo memory system from a production one.

The fix is to treat the store like a garden, not a landfill — two maintenance jobs keep it healthy. Consolidation is deduplication with a rule: when two memories describe the same subject (the same person's same attribute), the newer one wins and supersedes the old. "prefers phone" and "prefers email" both key to user/prefers, so the later write replaces the earlier — the contradiction is resolved at the source. Forgetting is eviction with a rule: drop memories that are both old and rarely used. Note the "and" — age alone isn't enough. A fact written a year ago but retrieved every week is clearly load-bearing; a fact written last month and never touched since is dead weight. Combining recency and usage is what stops you from evicting the things people actually rely on.

Counterintuitively, forgetting is a feature, not a loss. Beginners resist deleting memories — it feels like throwing away information. But every memory you keep is a memory retrieval has to consider, an embedding you pay to store, and a candidate that could crowd out something better in the top-k. A smaller, current store retrieves more sharply than a huge, stale one. This is the store-side twin of the context-budgeting you do on the window: there, you trim what to show the model this turn; here, you trim what to keep at all. And it's not a fringe technique — it's precisely what commercial systems like Mem0 and MemGPT mean when they talk about "consolidating" memory. The lab makes both jobs concrete: watch one preference supersede another, and one stale-and-unused fact get evicted while a stale-but-used one survives.

Memory that only grows becomes slow, expensive, and self-contradictory. Two maintenance jobs keep it healthy: consolidation (dedupe near-identical memories, and let newer facts supersede stale ones) and forgetting (drop memories that are old and rarely used). This is the store-side analogue of the context-budgeting you did earlier.

consolidate — newer fact supersedes older (same subject) Jan: prefers phone Jun: prefers email kept: prefers email forget — evict old AND unused; keep old-but-used "likes jazz" · old · 0 uses → evict "is premium" · old · 5 uses → keep age alone isn't enough Two maintenance jobs keep the store honest. Consolidation (top) lets the later fact win over the stale one on the same subject; forgetting (bottom) drops what is both old and unused (red) but keeps a stale-yet-used fact (green).
🗺️ How to read this diagram

The picture is split by the dashed line into the two jobs; read the top band, then the bottom.

  • Top — consolidation. Two writes about the same subject ("prefers …") collide. The faded January fact is superseded; the newer June one wins, so the store holds one current fact instead of a contradiction.
  • Bottom — forgetting. Each memory is judged on two signals, not one. "likes jazz" is old and never used → evicted (red). "is premium" is just as old but retrieved 5 times → kept (green).
  • The note on the right is the rule that prevents mistakes: age alone isn't enough — a load-bearing old fact stays because usage keeps it alive.

In short: consolidate to kill contradictions, forget on old-and-unused — a smaller, current store retrieves more sharply than a huge stale one.

Python · consolidate (dedup + supersede) and forget (runs)
consolidate.pydef subject_of(m):
    """A memory's subject = who + which attribute (e.g. 'user/prefers', 'user/is')."""
    for verb in (" prefers ", " is ", " likes "):
        if verb in m:
            who = m.split(verb)[0]
            return who + "/" + verb.strip()
    return m

def consolidate(memories):
    """Dedup: a later fact about the same subject supersedes an earlier one."""
    by_subject = {}
    for i, m in enumerate(memories):
        by_subject[subject_of(m)] = (i, m)    # last write wins
    return [text for _, text in sorted(by_subject.values())]

def forget(memories_meta, now, max_age=30, min_uses=1):
    """Drop memories older than max_age days that were used fewer than min_uses times."""
    kept = []
    for m in memories_meta:
        age = now - m["last_used_day"]
        if age > max_age and m["uses"] < min_uses:
            continue                          # forgotten
        kept.append(m["text"])
    return kept

mems = ["user prefers phone", "user is premium", "user prefers email"]
print("consolidated:", consolidate(mems))

meta = [
    {"text": "likes jazz", "last_used_day": 2, "uses": 0},   # old + unused -> forget
    {"text": "is premium", "last_used_day": 40, "uses": 5},  # old but used  -> keep
]
print("kept:", forget(meta, now=45, max_age=30, min_uses=1))
consolidated: ['user is premium', 'user prefers email']
kept: ['is premium']
▶ How this works

A store that only grows gets slow, costly, and self-contradictory. These two routines are the maintenance that keeps it healthy — the store-side version of budgeting the context window.

  1. subject_of() gives each memory a subject key like user/prefers or user/is, so two facts about the same attribute collide on purpose.
  2. consolidate() keeps the last write per subject — so a newer preference supersedes the older one instead of both lingering and contradicting each other.
  3. forget() drops memories that are both old (older than max_age) and rarely used (fewer than min_uses hits). A stale-but-still-used fact survives.
  4. Together these fight staleness (old facts win less) and cost (fewer stored memories = cheaper retrieval every turn).

What the output means: "user prefers phone" is superseded by the later "user prefers email" while "user is premium" (a different subject) is kept; the old, unused "likes jazz" is forgotten but the old-but-used "is premium" survives.

Try this: Bump the jazz memory's uses to 3 and re-run — it now survives despite its age, because it's clearly still relevant. That usage signal is how real systems avoid forgetting things people still rely on.

"user prefers phone" was superseded by the later "user prefers email"; the stale, unused "likes jazz" was forgotten while the frequently-used "is premium" survived. That is how a store stays small and current instead of accumulating contradictions.

8 · Mem0, Letta & MemGPT — self-editing memory professional

By now you've hand-built every piece of the loop, which means you're in the best possible position to evaluate the tools that sell it as a product. The names — Mem0, Letta, MemGPT — can feel intimidating from the outside, like they must contain some secret you're missing. They don't. Having built the write path, the read path, and the consolidation logic yourself, you can look at each of these and ask the only question that matters: which part of my loop is this automating, and which part am I still on the hook for? That framing turns a scary landscape into a short checklist.

Mem0 is a drop-in memory layer: it runs the whole extract → embed → store → retrieve loop for you, with dedup and updates baked in, behind a tidy add() / search() API. Conceptually it's the lab you just wrote, but with real embeddings instead of word overlap and production hardening you didn't have to build. You call add() after a turn and search() before the next one — the same two seams, someone else's plumbing.

MemGPT is the research idea that reframed the whole problem: treat the LLM like an operating system managing its own memory. An OS keeps a small, fast working set in RAM and a large, slow store on disk, and pages data between them on demand. MemGPT casts the context window as "RAM" and the external store as "disk," and lets the model itself decide what to page in and out. Letta is the framework that grew from that paper, and its headline feature is self-editing memory: the model calls tools to write, update, and search its own memory, rather than you doing it externally from your app code.

That self-editing twist is powerful and worth pausing on, because it moves a responsibility. In your hand-built version, you decided what to store; with self-editing memory, the write path from §6 becomes a tool the model invokes — memory management becomes part of the agent's own reasoning. The upside is flexibility; the catch is that you haven't escaped the hard questions, you've relocated them. You still must bound what the model is allowed to remember and for how long, except now you enforce it through a tool policy and guardrails instead of a function you control directly. Under all the branding, every one of these is the §4 loop with production polish — which is exactly why building it yourself first was worth it. The table lines up who does what.

You don't have to build the loop from scratch. Mem0 is a memory layer: it runs the extract → embed → store → retrieve loop for you, with dedup and updates, behind a add()/search() API. MemGPT (the research paper) framed the LLM as an OS managing its own memory: a small in-context working set plus a larger external store, with the model deciding what to page in and out. Letta is the framework that grew from MemGPT — agents with self-editing memory: the model calls tools to write, update, and search its own memory rather than you doing it externally.

"RAM" — context window (small, fast working set) in-context working set what the model sees now "disk" — external store (large, slow) external memory store everything else page out page in the model itself calls tools to page in/out Treat the LLM like an OS managing its own memory. The window is "RAM" (fast, tiny), the store is "disk" (vast, slow), and self-editing memory means the model itself calls tools to page facts in and out — the write path from §6 becomes the agent's own decision.
🗺️ How to read this diagram

Read the two top boxes as the classic RAM/disk split, then note who is driving the paging from below.

  • The left box is the context window cast as "RAM" — small and fast, holding only what the model needs this turn.
  • The right box is the external store cast as "disk" — vast and slow, holding everything else. The blue arrows between them are paging: moving facts in when needed, out when not.
  • The green box below is the twist Letta/MemGPT add: the model itself calls tools (the amber arrows) to decide what to page. Memory management becomes part of the agent's reasoning instead of your app's code.

In short: same window-plus-store loop as §4, reframed as an OS — and with self-editing memory the model, not you, decides what to keep in context.

AddsMem0Letta / MemGPT
Core ideadrop-in memory layerLLM manages its own memory (OS metaphor)
Who edits memoryyour app, via add/searchthe model itself, via tools
You still ownwhat to store, privacythe tool policy + guardrails

Both are the same loop from §4 with production polish (embeddings, dedup, updates). "Self-editing" means the write path in §6 becomes a tool the model calls — powerful, but you must still bound what it may store and for how long.

SDK · Mem0 add + search — needs the SDK (illustrative, not run here)
mem0_sdk.py# needs the SDK:  pip install mem0ai   (network + API key; not runnable offline)
from mem0 import Memory

mem = Memory()                       # backed by a vector store under the hood
mem.add("I prefer window seats", user_id="dana")   # extract + embed + store
hits = mem.search("seating preference?", user_id="dana")
# hits -> the relevant memories to inject into your prompt
# Same loop as memory_store.py, with real embeddings + dedup + updates.
Staleness: memories rotPeople move, change jobs, and reverse preferences. A memory written six months ago may now be wrong. Timestamp memories, prefer recent facts, let newer writes supersede older ones (§7), and re-confirm high-stakes facts instead of trusting them forever.

9 · Tech-lead — a memory policy for the system tech-lead

Here's the uncomfortable realization that arrives the day your memory feature ships. The moment your agent starts remembering, it stops being a stateless toy and becomes a system that holds personal data about real people across time. That's a genuinely different kind of thing to own. An engineer asks "does retrieval work?" A tech lead has to answer a harder set of questions — because the very property that makes memory valuable (it accumulates knowledge about users) is the same property that makes it a liability (it accumulates data about users). Value and risk scale together, and they scale with exactly one dial: how much you choose to store.

So a lead owns the whole memory contract, not just the code that implements it. That contract is a small set of decisions you make deliberately and write down, rather than letting them emerge by accident from whatever the extractor happened to grab:

The through-line: a memory store is user data under active management, not a cache you can forget about. Treat the write policy, the retention rules, and the deletion path as first-class parts of the design — specified and measured — and the feature is an asset. Leave them to emerge on their own and you've built a liability that grows every day it runs.

A lead owns the whole memory contract, not just the code. Decide: what the system is allowed to remember (and explicitly what it must not — secrets, payment data); how long memories live and the forgetting policy; the retrieval budget (how many memories inject per turn — this is context engineering again); a privacy interface so users can view and delete their memories; and the cost (every stored memory is storage + an embedding + a retrieval on every turn). Write it down; a memory store is user data under management, not a cache you can forget about.

the memory contract what may / must not retention + forget retrieval budget privacy interface secrets · payment hard exclusion value AND risk scale with one dial: how much you store + cost: every memory = storage + embedding + a retrieval every turn A lead owns the contract, not just the code. Five deliberate decisions hang off "how much you store" — because the property that makes memory valuable (it accumulates data about users) is the same one that makes it a liability.
🗺️ How to read this diagram

Read top-down: the one contract at the top fans out into the decisions a lead must make on purpose rather than let emerge.

  • The top box is the memory contract — the small set of decisions written down and owned deliberately.
  • The row of boxes is those decisions: what may be remembered, how long memories live (retention + forgetting), the per-turn retrieval budget, and a privacy interface so users can view and delete. The green box is the loudest one — and its red child is the hard exclusion list: secrets and payment data, never.
  • The lines at the bottom are the through-line: value and risk both scale with how much you store, and every stored memory carries an ongoing cost (storage + embedding + a retrieval every turn).

In short: specify the write policy, retention, retrieval budget, deletion path and cost up front — leave them to emerge and you've built a liability that grows daily.

Memory is a product decision, not just a featureThe value (an agent that remembers you) and the liabilities (privacy, staleness, cost, wrong-memory bugs) both scale with how much you store. A lead treats the write policy, retention, and deletion path as first-class — designed and measured, not emergent.

Exercise FA4.1 — Wire the loop end to end

Context: The payoff of the labs is one working loop: extract facts from turns, write them, and retrieve the relevant ones for a new message — the retrieval-augmented memory cycle.

Your task: Combine the labs: feed turns through fact_extract.py's extract_facts, write() the results into memory_store.py's MemoryStore, then retrieve() for a new message and print the injected prompt.

Requirements:

  • Extract facts from a few turns
  • Write the extracted facts into the store
  • Retrieve for a new message
  • Print the prompt with the injected memories
  • Confirm only relevant memories come back

💡 Hint: The end-to-end test is that an unrelated new message pulls back nothing, while a related one surfaces the earlier fact.

Exercise FA4.2 — Keep the store healthy

Context: A healthy store needs pruning as well as writing. Consolidation and forgetting are what keep retrieval sharp as memories pile up and go stale.

Your task: Add several overlapping and stale memories, then run consolidate and forget from consolidate.py, showing a superseded preference and a forgotten unused fact, and write a two-line memory policy.

Requirements:

  • Seed overlapping and stale memories
  • Run consolidation to merge/supersede overlapping ones
  • Run forgetting to evict an unused, stale fact
  • Demonstrate a superseded preference and a forgotten fact
  • Write a two-line policy: what you store and how long you keep it

💡 Hint: Consolidation collapses duplicates and updates superseded preferences; forgetting removes what hasn't earned its keep by recency or use.

🪜 Practice ladder beginner → industry

Six graded exercises, easy to real-world. Try each before opening its solution.

Exercise 1 · The window is not memoryBeginner

Context: The context window is not memory: it's finite, and once it overflows the oldest turns silently fall out. Seeing that failure is the motivation for a real memory store.

Your task: Model a fixed-size context window and show that an early fact is forgotten after enough turns push past its capacity.

Requirements:

  • Represent the window as a bounded buffer of recent turns
  • Append turns until the buffer overflows
  • Demonstrate that an early fact is no longer present
  • Make the window size a parameter

💡 Hint: A deque(maxlen=n) drops the oldest entry automatically — perfect for showing the early fact vanish.

Show solution
from collections import deque

class Window:
    def __init__(self, size=3):
        self.turns = deque(maxlen=size)   # oldest turns evicted automatically
    def add(self, turn): self.turns.append(turn)
    def recall(self): return list(self.turns)

w = Window(size=3)
for t in ["my name is Ada", "the sky is blue", "2+2=4", "what is my name?"]:
    w.add(t)
print(w.recall())
# ['the sky is blue', '2+2=4', 'what is my name?']  -- the name is GONE

The window is working attention, not storage. Anything the agent must remember beyond the window has to be written to an external memory.

Exercise 2 · Episodic vs semanticIntermediate

Context: Agents store two kinds of memory: episodic (a specific timestamped event) and semantic (a durable fact). Routing each to the right store is the foundation of an agent-memory design.

Your task: Write a classifier that labels stored items as episodic (a specific, timestamped event) vs semantic (a durable fact) and routes each to the right store.

Requirements:

  • Distinguish event-like items from durable facts
  • Label each item episodic or semantic
  • Route to the corresponding store
  • Handle ambiguous items with a defined default

💡 Hint: Time-anchored, one-off statements are episodic; general, timeless statements ("the user prefers X") are semantic.

Show solution
def classify(item):
    # semantic = stable preference/fact; episodic = a dated event
    episodic_cues = ["yesterday", "today", "at ", "on monday", "just now"]
    if any(c in item.lower() for c in episodic_cues):
        return "episodic"
    if item.lower().startswith(("i prefer", "my", "i am", "i like")):
        return "semantic"
    return "episodic"

for i in ["I prefer dark mode", "Booked a flight today", "My name is Ada"]:
    print(f"{i!r} -> {classify(i)}")
# 'I prefer dark mode' -> semantic
# 'Booked a flight today' -> episodic
# 'My name is Ada' -> semantic

Semantic memory holds timeless facts you always want; episodic memory holds events you retrieve by recency/relevance. Storing them separately keeps retrieval clean.

Exercise 3 · A tiny MemoryStore (write + retrieve)Advanced

Context: Every memory system needs a store you can write to and query. Building a tiny one with bag-of-words overlap keeps it offline and deterministic before you add embeddings.

Your task: Build a MemoryStore with write(text) and retrieve(query, k) that returns the k most relevant memories, using a bag-of-words overlap score so it runs offline.

Requirements:

  • write appends a memory
  • retrieve scores memories by word overlap with the query
  • Return the top-k most relevant, best first
  • Runs offline — no embeddings or network
  • Handle k larger than the store

💡 Hint: Overlap can be as simple as the size of the shared word set between query and memory; sort by that score and slice the top k.

Show solution
class MemoryStore:
    def __init__(self): self.mem = []
    def write(self, text):
        self.mem.append(text)
    def _score(self, query, text):
        q, t = set(query.lower().split()), set(text.lower().split())
        return len(q & t) / (len(q) or 1)      # word-overlap stand-in for cosine
    def retrieve(self, query, k=2):
        ranked = sorted(self.mem, key=lambda m: self._score(query, m),
                        reverse=True)
        return [m for m in ranked if self._score(query, m) > 0][:k]

ms = MemoryStore()
for f in ["user prefers dark mode", "user lives in Berlin",
          "user dislikes email notifications"]:
    ms.write(f)
print(ms.retrieve("what mode does the user prefer?"))
# ['user prefers dark mode']

The real store swaps the overlap score for embedding cosine similarity, but the write/retrieve seam is identical — this is the offline version of RAG-over-memory.

Exercise 4 · The write path — dedupe before storingExpert

Context: Not every turn deserves to be remembered, and duplicates bloat the store and degrade retrieval. A disciplined write path filters and merges instead of blindly appending.

Your task: Add a write path that skips low-value text and merges near-duplicates instead of appending them.

Requirements:

  • Skip low-value/empty text before storing
  • Detect near-duplicates against existing memories
  • Merge a near-duplicate rather than adding a second copy
  • Only genuinely new information grows the store

💡 Hint: Reuse the overlap score from the previous rung as a similarity check: above a threshold, merge; below it and non-trivial, store.

Show solution
class MemoryWriter:
    def __init__(self): self.mem = []
    def _similar(self, a, b):
        A, B = set(a.lower().split()), set(b.lower().split())
        return len(A & B) / len(A | B) if (A | B) else 0   # Jaccard
    def write(self, text):
        if len(text.split()) < 3:
            return "skipped: too trivial"
        for i, m in enumerate(self.mem):
            if self._similar(text, m) > 0.6:
                self.mem[i] = max(m, text, key=len)         # keep the fuller one
                return "merged with existing"
        self.mem.append(text); return "stored"

w = MemoryWriter()
print(w.write("ok"))                                   # skipped: too trivial
print(w.write("user prefers dark mode"))               # stored
print(w.write("the user prefers dark mode in the app"))# merged with existing
print(w.mem)

Deciding what and when to remember is the hard part of memory. Filtering trivia and merging duplicates keeps retrieval precise and the store small.

Exercise 5 · Forgetting & consolidationProfessional

Context: Unbounded memory eventually hurts retrieval. Real systems consolidate: score memories by recency and use, then evict the weakest when over capacity.

Your task: Implement decay — score memories by recency plus access count — and evict the lowest-scoring memories when the store exceeds capacity (consolidation).

Requirements:

  • Score each memory by a mix of recency and access count
  • When over capacity, evict the lowest-scoring memories
  • Frequently-used and recent memories survive
  • Capacity and the scoring weights are parameters

💡 Hint: A memory touched often and recently should outscore a stale, never-retrieved one; evict from the bottom of that ranking.

Show solution
import time

class DecayingMemory:
    def __init__(self, capacity=3):
        self.capacity = capacity; self.mem = []   # list of dicts
    def write(self, text):
        self.mem.append({"text": text, "ts": time.time(), "hits": 0})
        self._consolidate()
    def retrieve(self, text):
        for m in self.mem:
            if text in m["text"]:
                m["hits"] += 1; return m["text"]
    def _score(self, m):
        age = time.time() - m["ts"]
        return m["hits"] * 10 - age            # frequent + recent scores high
    def _consolidate(self):
        while len(self.mem) > self.capacity:
            worst = min(self.mem, key=self._score)  # evict least valuable
            self.mem.remove(worst)

dm = DecayingMemory(capacity=2)
dm.write("A"); dm.write("B"); dm.retrieve("B")  # B accessed -> higher score
dm.write("C")                                    # over capacity -> evict A
print([m["text"] for m in dm.mem])               # ['B', 'C']

Forgetting is a feature: recency + usage decay keeps the store bounded and biases retrieval toward what actually matters, mirroring how Mem0/MemGPT consolidate.

Exercise 6 · A memory policy for the system (SDK-ready)Industry scenario

Context: As tech lead you define the whole memory architecture and show how it plugs into a single agent turn — retrieve before the prompt, decide what to write after. The real embedding/SDK calls slot into named seams.

Your task: Define the memory architecture — semantic store, episodic log, retrieve-before-prompt, and a write-after decision — and show how it plugs into an agent turn, noting where the real embedding/SDK calls go.

Requirements:

  • Combine a semantic store and an episodic log
  • Retrieve relevant memories and inject them before the model prompt
  • Decide what (if anything) to write back after the turn
  • Mark the seams where real embeddings / SDK calls would be wired in
  • Show it as one coherent turn, not disconnected pieces

💡 Hint: The turn is: retrieve → build prompt with memories → generate → write-after decision; the offline pieces stand in for the embedding and model calls.

Show solution

Offline-runnable skeleton; the two labeled seams take the real embedding model and Anthropic client in production.

class AgentMemory:
    def __init__(self):
        self.semantic = []    # durable facts/preferences
        self.episodic = []    # timestamped events

    # SEAM 1 (needs embeddings/SDK): swap overlap for vector similarity
    def retrieve(self, query, k=3):
        pool = self.semantic + self.episodic
        score = lambda m: len(set(query.split()) & set(m.split()))
        return sorted(pool, key=score, reverse=True)[:k]

    def write(self, text, kind):
        (self.semantic if kind == "semantic" else self.episodic).append(text)

def agent_turn(memory, user_msg, classify, llm):
    context = memory.retrieve(user_msg)          # 1. retrieve before prompting
    prompt = f"Memories: {context}\nUser: {user_msg}"
    reply = llm(prompt)                          # 2. SEAM 2: real Anthropic call
    if "remember" in user_msg.lower():           # 3. write-after decision
        memory.write(user_msg, classify(user_msg))
    return reply

mem = AgentMemory()
mem.write("user prefers metric units", "semantic")
print(agent_turn(mem, "what units do I prefer?", lambda t: "semantic",
                 llm=lambda p: f"[echo] {p[:40]}"))

Policy: retrieve relevant memories into every prompt, classify+write only high-value items, and consolidate/expire on a schedule. The pattern is provider-agnostic; only the embedding and generation seams call out to the SDK.

✓ Checkpoint — you can move on when you can…

  • Explain why the context window is not memory (stateless model).
  • Distinguish short-term vs long-term and episodic vs semantic memory.
  • Trace and implement the extract → write → retrieve → inject loop.
  • Consolidate/dedup and forget so the store stays current and cheap.
  • Say what Mem0 and Letta/MemGPT add, including self-editing memory.
  • Set a memory policy covering privacy, staleness, retention, and cost.

Knowledge check check yourself

✓ Knowledge check

Since a chat model is stateless, what four-step loop gives it long-term memory, and how does it relate to RAG?

Show answer
The loop is extract facts from a turn → write them to a durable store → retrieve the relevant ones for the next message → inject them into context. The read path is exactly retrieval, so long-term memory is RAG over the user's own past rather than a document library.
✓ Knowledge check

What is the difference between episodic and semantic memory in an agent, and why keep both?

Show answer
Episodic memory records events (“on Tuesday the user asked to cancel order #42”); semantic memory stores distilled durable facts (“the user prefers email”). You keep both because events give traceable context while distilled facts are the compact, reusable preferences you inject each turn.
© 2026 studybydoing.in · AI Engineering: Zero to Production · All rights reserved. · About · Privacy Policy · Terms · Contact
Educational content, provided as-is and without warranty. Code samples are examples — review, test, and adapt them before using in production. See the Terms of Use & Disclaimer. Use at your own risk.
© studybydoing.in