In this article
Short-term (in-context) memory consists of everything in the LLM's current context window: conversation history, retrieved documents, tool outputs, and intermediate reasoning. It's fast and immediately accessible but ephemeral and size-limited. Key management strategies include: context pruning (dropping irrelevant messages), conversation summarisation (replacing old turns with a summary), and sliding window (keep only the last N turns). When short-term memory fills the context window, agents must rely on external retrieval systems for additional context.
What it means in practice
Short-term Memory is not just vocabulary; it is a design handle. In agentic systems, this term usually appears when engineers are designing, reviewing, or troubleshooting real production flows rather than only naming the concept. It shows up when an AI system must choose actions, call tools, remember state, and recover from partial failures.
Why engineers care
- It gives teams a shared name for the behaviour, risk, or architecture choice being discussed.
- It helps separate the goal from the implementation detail, so you can compare alternatives instead of copying a tool pattern blindly.
- It creates a useful checklist for reviews: inputs, outputs, failure modes, ownership, cost, latency, and measurement.
Production watch-outs
Do not add autonomy without observability and limits. Tool permissions, retries, budgets, timeouts, and human review paths should be explicit.
Related context
Useful neighbouring concepts: Context Window, Long Term Memory, Episodic Memory, Chunking.