Tiered Memory
Working, episodic and semantic memory stored and recalled separately.
Problem
An agent that forgets everything between sessions makes users repeat themselves and can’t learn from past work. But dumping every past conversation into the prompt is slow, expensive and noisy. Different kinds of memory need different storage, lifetimes and retrieval.
- Users expect the agent to remember facts and preferences across sessions.
- Raw history grows without limit and is mostly noise.
- Wrong or stale memories are worse than none.
- Personal data in memory raises privacy and deletion requirements.
Solution
Split memory into tiers. Working memory is the current context window: the task, recent turns and live tool results. Episodic memory stores summaries of past sessions and outcomes, retrieved when a similar situation comes up. Semantic memory holds durable facts and preferences, such as “prefers metric units” or “account owner is Priya”, each with a source and a date. A write gate decides what is worth keeping and in which tier. Retrieval pulls only the relevant items into working memory at the start of each task. Users can view, correct and delete what’s stored.
Diagram
The write gate is as important as the stores. Most of what happens in a session should never become long-term memory.
When to use
Use it when
- Users return over days or weeks and expect continuity
- Past outcomes can improve future runs
- Personal preferences change how the agent should respond
- Sessions are too long or too many to keep in raw form
Avoid it when
- Single-shot tasks with no returning users
- You can't offer users a way to view and delete stored memory
- Stored facts change so quickly that memory is mostly stale
Trade-offs
Gains
- Continuity across sessions without huge prompts
- Agent improves from its own past outcomes
- Personalization grounded in stated facts and preferences
Costs
- Stale or wrong memories mislead future answers
- Privacy duties for storing personal data
- Deciding what to keep is a hard, ongoing design problem
Worked example
A personal work assistant used daily by an account manager.
- Monday sessionThe user says “I always want customer briefs as one page, risks first.” The write gate saves this to semantic memory:
preference: brief format = 1 page, risks first. - Monday sessionThe user preps for an Acme renewal. At session end, an episodic summary is saved: “Acme renewal prep; legal blocker on DPA; follow up Nov 14.”
- Write gateSkips 40 turns of small talk and lookups. Only the 2 items above are stored.
- Thursday sessionThe user asks for an Acme update. Retrieval pulls the Acme episode and the brief preference into working memory. The agent writes a 1-page brief with risks first, opening with the DPA blocker.
- CorrectionThe user says the DPA was signed yesterday. The agent updates the episodic note, marks the old fact as superseded and dates the change.
Failure modes
| Failure | Symptom | Mitigation |
|---|---|---|
| Memory hoarding | Everything is stored; retrieval returns noise | Strict write gate; store facts and outcomes, not transcripts |
| Stale facts | Agent acts on information that changed months ago | Date every fact; prefer recent ones; ask when facts look old |
| Inferred facts treated as stated | Agent "remembers" things the user never said | Store only stated facts or verified outcomes, with a source |
| No user control | Users can't see or remove what's stored | Memory view, edit and delete; honor deletion everywhere |