An agent without memory is a brilliant colleague with amnesia — capable in the moment, useless across time. Memory is what makes it cumulative.
TL;DR
- Memory lets an agent carry context across steps, sessions, and projects.
- Distinguish short-term (this task), session (this conversation), and long-term (persistent) memory.
- Store facts as small, retrievable units — and prune aggressively to avoid stale context.
Why statelessness hurts
Models are stateless by default: each call knows only what you put in the context window. For a one-shot question that is fine. For an agent doing real work it is a problem — it re-derives the same project facts every session, forgets decisions it made an hour ago, and cannot learn your preferences. Memory is the system you build around the model to fix that.
Three layers of memory
It helps to separate memory by lifespan:
- Short-term (working) memory — the current task's state: what it has tried, what failed, what is left. Lives in the active context.
- Session memory — the thread of the current conversation or run, so the agent stays coherent across many turns.
- Long-term memory — facts that persist across sessions: project conventions, decisions, user preferences, prior outcomes.
Short-term ─ this task ─ ephemeral, in-context
Session ─ this run ─ summarized as it grows
Long-term ─ across runs ─ persisted, retrieved on demand
Patterns that work
- Write facts as small units. One fact per record, with a short description, so retrieval is precise. "We use Vitest, not Jest" is more useful than a 2,000-word architecture dump.
- Summarize, don't accumulate. As a session grows, compress old turns into a summary instead of carrying the raw history forever. Long context degrades attention.
- Retrieve, don't preload. Keep long-term memory in a store and pull in only what is relevant to the current task — the dynamic-context discipline from context engineering.
- Make memory inspectable. You want to see, and correct, what the agent "remembers." Opaque memory becomes a source of confident, persistent errors.
The failure mode of memory is not forgetting — it is remembering the wrong thing forever. Prune like it matters, because it does.
The staleness trap
The danger with long-term memory is that the world changes and the memory does not. A remembered "the auth service lives at X" becomes a landmine after a refactor. Defenses:
- Timestamp memories and weight recent ones more heavily.
- Verify before trusting — if a memory names a file or endpoint, confirm it still exists before acting on it.
- Let memories be deleted. Wrong memory is worse than no memory.
The payoff
Done well, memory turns an agent from a clever one-shot tool into something that compounds — it learns your stack, remembers your decisions, and stops starting from zero. That continuity is much of what separates a demo from a teammate.