Problem Context

Episodic memory is the agent's record of what happened: which user, which session, which decisions, which outcomes. Semantic memory ("facts") tells you what is true. Episodic memory tells you what occurred and when. Mixing the two is the most common reason an agent gives a confident answer that contradicts its own history three days later.

🤔 Sound familiar?
  • You can't answer "what did we agree on last Tuesday?"
  • Your agent "remembers" an event but gets the date or the people wrong
  • You want chronological reasoning ("before that", "the most recent") but your store is just a vector index

What Counts as an Episode

An episode is an event tuple, not a chat log. A good schema:

type Episode = {
  id: string;
  user_id: string;
  session_id: string;
  ts: string;          // ISO timestamp — the primary index
  actor: 'user' | 'agent' | 'system';
  event: string;       // 'decision' | 'tool_call' | 'feedback' | 'milestone'
  summary: string;     // 1–3 sentence natural-language description
  entities: string[];  // ['project:atlas', 'invoice:#42']
  embedding: number[]; // for semantic recall
};

The Two Retrieval Modes

  • Temporal retrieval— "what happened yesterday?", "the last 3 decisions about pricing". Indexed byts and entities, no embeddings needed.
  • Semantic retrieval — "have we talked about retries before?". Vector search over summary, optionally filtered by entity or time window.

Production systems run both and merge: most relevant events from the last 30 days beats either approach alone.


flowchart LR
    Q[Query: 'what did we<br/>decide about pricing?'] --> T[Temporal filter:<br/>entity=pricing, ts > 30d]
    Q --> V[Vector search:<br/>top-K by summary]
    T --> M[Merge + rerank<br/>by recency × similarity]
    V --> M
    M --> P[Top 5 episodes<br/>into prompt]

Writing Episodes (Without Writing Every Turn)

Not every message is an episode. A heuristic that works:

  • User commits to something (decides, books, agrees) → write.
  • Agent makes a non-trivial tool call (sends email, runs job, updates record) → write.
  • Either party introduces a new entity (a project, a person, a deadline) → write.
  • Idle chat, clarifications, retries → don't write.

Run a small classifier (a cheap model with a strict schema) over each turn and only write when is_episode = true.

Temporal Reasoning in Prompts

Just pasting summaries doesn't give the model temporal awareness — it confuses order. Prefix each episode with a relative timestamp the model can reason about:

[3 days ago, 2026-05-14 14:22 UTC, session "billing-q2"]
User confirmed the new pricing tier (Pro at $29) for project Atlas.

[2 hours ago, 2026-05-17 08:10 UTC, session "support-#412"]
Agent escalated invoice #42 to finance after 3 failed retries.

Failure Modes

  • Episode spam — writing every turn destroys precision. Classify first.
  • Entity drift — "project Atlas" vs "Atlas project". Normalise entities (lowercase, canonical IDs) before indexing.
  • Privacy creep — episodic logs are dense personal data. Same retention/deletion policies as raw transcripts.
  • Stale relevance — a decision from 18 months ago overrides a decision from yesterday. Always sort merged results by ts DESC within a similarity band.

Production Checklist

  • Structured event schema with ts and entities indexed in your DB.
  • Classifier-gated writes; treat the episode store as expensive to write, cheap to query.
  • Hybrid retrieval (temporal + semantic), reranked by recency.
  • Render episodes with explicit timestamps in the prompt — relative and absolute.

Key Takeaways

  • Episodes are events, not messages — write selectively, query by time + entity + similarity.
  • Temporal awareness is a prompt problem, not just a storage problem.
  • Keep episodic memory separate from semantic facts — conflating them is the root of confidently wrong answers.