How AI Agent Memory Works

AI agent memory is the machinery that lets an agent carry information across turns, sessions, and tasks instead of starting cold every time you talk to it. It is suddenly one of the most active areas in applied AI: in early June 2026 OpenAI introduced Dreaming, a memory upgrade for ChatGPT, and on the same day two separate research papers landed on arXiv — one exploring the cross-scenario generality of agentic memory systems and another, SaliMory, on orchestrating cognitive memory for conversational agents. When a flagship product move and independent research converge in the same week, it is a strong signal that the field is consolidating around shared patterns. If you build, evaluate, or simply rely on AI agents, understanding how agent memory works is quickly becoming table stakes.

This is a vendor-neutral explainer: what memory is in an agent, the distinct types, the write-and-recall loop that makes it useful, and the trade-offs you have to make on purpose rather than by accident.

What is AI agent memory?

A large language model is, by default, stateless. Each request is answered from the prompt in front of it plus whatever was baked into its weights during training. AI agent memory is the layer wrapped around that model so it can persist and reuse information the base model would otherwise forget the instant a response finishes.

Concretely, memory is anything the agent writes down now so it can read it back later: your stated preferences, facts learned mid-task, the outcome of a previous tool call, a summary of last week's conversation. The model still does the reasoning; memory decides what context that reasoning gets to see.

It helps to separate two things people often blur together:

• The context window is short-term and bounded — the tokens the model can attend to in a single call.
• Memory is the external store the agent reads from and writes to, deliberately selecting a small, relevant slice to load into that limited window.

Memory exists precisely because the context window is finite and resets. The art is choosing what to keep and what to surface.

What are the types of agent memory?

Most agent memory designs borrow a rough taxonomy from cognitive science. The names vary by framework, but the categories are consistent:

• Working memory — the here-and-now scratchpad for the current task: the active conversation, intermediate reasoning, the last tool result. It usually lives directly in the context window and is discarded when the task ends.
• Episodic memory — a record of specific past events: "on Tuesday the user asked for a refund and we issued it," or "this build failed with this error last run." Episodic memory is what lets an agent say "last time we tried X."
• Semantic memory — distilled, durable facts decoupled from any single event: "the user prefers metric units," "this customer is on the enterprise plan." This is the layer most consumer "memory" features expose, and it is what ChatGPT's memory broadly draws on.
• Procedural memory — learned how-to: reusable workflows, tool-use patterns, or instructions the agent has found effective. In practice this often shows up as saved playbooks or refined system prompts.

A capable agent blends several of these. Working memory handles the immediate turn; episodic and semantic memory give continuity across sessions; procedural memory makes the agent better at doing things over time, not just recalling them.

How does an agent decide what to remember?

Writing to memory is a decision, not a default. Storing every token an agent ever sees is expensive, slow to search, and quickly fills with noise. So agents use a write policy. Common strategies:

• Salience-based writes — keep only information judged important enough to matter later. The SaliMory paper frames this explicitly as orchestrating cognitive memory for conversational agents, prioritizing what is salient rather than logging everything.
• Consolidation — periodically compress raw episodes into compact summaries or semantic facts, the way human memory turns a day of events into a few durable takeaways. OpenAI describes its Dreaming update along these lines: an offline process that reflects on past interactions to produce better, more helpful memory rather than storing transcripts verbatim.
• Explicit user control — letting the user pin, edit, or delete what is remembered, which is both a usability and a trust feature.

The recurring lesson: memory quality depends more on what you choose not to store than on raw capacity.

How does an agent recall the right memory?

Recall is the other half of the loop, and it is where most real-world failures hide. The agent has potentially thousands of stored items and a tiny context window; it must fetch the few that matter for the current step.

The dominant approach is retrieval, usually semantic search over embeddings: the current query is embedded, compared against stored memories, and the closest matches are loaded into the prompt. This is the same retrieval-augmented pattern used for documents, applied to the agent's own history. Variations layer on recency weighting, metadata filters (this user, this project), or a reranking pass to push the most relevant items to the top.

Retrieval is powerful but fallible. If the embedding of a stored fact does not sit near the embedding of the question, the right memory simply never gets loaded — and the agent confidently answers as if it never knew. Most "the agent forgot what I told it" complaints are retrieval misses, not storage failures.

Retrieval vs. consolidation: what is the difference?

These are the two levers of a memory system, and they operate at different moments:

• Retrieval happens at read time, on every step: which stored items do we pull into the context window right now?
• Consolidation happens at write time, often in the background: how do we compress, merge, and refine what is stored so future retrieval is cheaper and cleaner?

Systems that lean entirely on retrieval over raw logs tend to get slower and noisier as history grows. Systems that consolidate aggressively risk discarding a detail that later turns out to matter. Good designs do both: consolidate to keep the store compact and meaningful, then retrieve precisely from that cleaner store.

What are the main design trade-offs?

There is no free lunch in agent memory. The decisions you make are mostly about which failure mode you can tolerate:

• Completeness vs. cost — more stored, longer retained memory means higher storage and search cost and more tokens loaded per call.
• Recall vs. precision — pull in more candidate memories and you rarely miss the relevant one, but you crowd the window with noise that degrades reasoning. Pull in fewer and you stay sharp but risk forgetting.
• Freshness vs. stability — should a new fact overwrite an old one, or coexist? "The user moved to Berlin" should update the old city; "the user likes concise answers" probably should not be erased by one verbose request.
• Personalization vs. privacy — durable memory is exactly what makes an agent feel personal, and exactly what raises questions about what is retained, where, and who can see or delete it. Treat user-facing controls as part of the architecture, not an afterthought.

How do you keep agent memory accurate over time?

Memory rots. Facts go stale, get duplicated, or contradict each other across sessions. The arXiv work on the cross-scenario generality of agentic memory systems gets at a related, harder problem: a memory design that works in one scenario may not transfer to another, which is a reminder that there is no single configuration that is correct everywhere.

Practical hygiene that holds up across designs:

• Deduplicate and merge near-identical memories so retrieval does not surface five copies of one fact.
• Update in place when new information supersedes old, rather than appending contradictions.
• Timestamp and decay — let stale, unused memories age out so the store reflects what is currently true.
• Make memory inspectable — you cannot debug what you cannot see, which is exactly where memory and observability meet. Once your agents remember and act on their own history, you need to watch what they actually do with it; see our companion explainer, AI agent observability explained.

Practical takeaways for builders

• Memory is a system around the model, not a feature of the model. Design the write policy and the retrieval policy explicitly.
• Match the memory type to the job: working memory for the current task, episodic for "last time," semantic for durable facts, procedural for reusable how-to.
• Treat retrieval quality as the thing most likely to break. Most "it forgot" bugs are recall failures, so test retrieval directly.
• Consolidate to keep the store clean; retrieve precisely from it. Doing both beats doing either alone.
• Build user controls and memory hygiene in from the start — they are core to trust, not polish.

The convergence of a major product launch and same-week research on agent memory tells you this is no longer a niche concern. The teams that treat memory as deliberate architecture — and pair it with the ability to observe what their agents do — will ship agents that feel reliable instead of forgetful.

Want to put these ideas into practice? Explore how Clawvard helps you build and run agents, and read AI agent observability explained for the other half of the picture. Follow the Clawvard blog for ongoing, vendor-neutral explainers on agent infrastructure.