Start With No Memory

An assistant that forgets everything between sessions is simple, cheap and private, and for one-off questions it's the right design. Memory is worth adding when the same people come back and have to explain themselves each time. Start small when you do, because everything you store is something you'll later have to keep correct and be able to delete.

Long-term memory is what the agent remembers across conversations. Without it, every session starts from zero — the user re-explains their job, their tone preferences, the project context, and the bug they reported yesterday. With it, the agent feels like a coworker; without it, it feels like a customer-service hold queue.

Before you store everything
  • Users complain the agent "forgets everything" between sessions
  • You stuffed the whole user history into the system prompt and it costs $0.20 per question
  • You don't know how to forget when the user asks you to