Message trimming, summarisation, sliding window, token budgets.

Short-term Memory — Context Window Management

Short-term memory is the agent's context window — everything the LLM can "see" in one generation. Context is finite and expensive. Strategies to manage it: sliding window (keep only last N messages), summary memory (compress older messages into a summary), token-based trimming (remove messages to fit a budget), and selective retention (keep tool results, drop intermediate thoughts). Context management directly impacts agent coherence and cost.

Interactive Pipeline Flow

UnderstandImplementOptimise

📥 Receive — new message

Every user message, tool result, and agent response adds tokens to the context. Estimate token counts with tiktoken before adding to context.

Architecture Diagrams

Optimal ordering of context components for agent coherence.

Loading diagram...

Key Design Decisions

📌Pin the user goal, never let it scroll off

The original user goal is the most important context element. Mark it as pinned so it's never trimmed. Agents that lose sight of the goal wander into tangential tasks.

🔢Measure tokens, not character counts

Use tiktoken (for OpenAI models) or the model's tokenizer to count tokens precisely. A 1000-character message may be 250 or 500 tokens depending on content.

✂️Summarise before you trim

Trimming discards potentially critical information. Summarisation compresses it. Use tiktoken to monitor context; when it exceeds 70% of the model's limit, summarise older exchanges.

Discussion