Context Engineering: What Goes in the Window

The Agent Got Worse When You Gave It More

You ship an on-call assistant. For the first ten turns of an incident it is sharp. By turn thirty it has pasted four thousand log lines into the conversation, re-read the same runbook twice, and started quoting a stale deploy timestamp from turn three as if it were current. Nothing in your prompt changed. The window just filled up with things that were true once, or never mattered.

A 1M-token window did not fix this, and that surprised a lot of teams. The window got bigger, but the model still has to work out what is useful, what is current and what is noise from whatever you put in front of it. Context engineering is the job of deciding what goes in the window on every turn. Prompt writing is one slice of it. Tool results, retrieved documents, memory and history are the rest, and on a working agent they are most of the tokens.

What Is Actually in the Window

Dump one real request and sort it by source before you change anything. For most agents the shape is something like this. Treat the shares as an illustration and measure your own, because the point is the exercise.

Anthropic's guidance puts the goal well: find the smallest set of high-signal tokens that makes the outcome you want most likely. Note that this asks for the smallest set, which is a harder target than a complete one. Arthur Casals said it plainly in InfoQ's 2026 AI, ML and Data Engineering Trends Report: “Context pollution is actually the biggest problem,” and more context can make an agent slower, more expensive and less reliable.

The Four Moves

Nearly everything that works is one of four moves. I would add them in this order, because each one is more work than the last.

• Clear tool results. Once the model has acted on a result, the raw payload rarely needs to stay. Replace it with a one-line stub that says what it was and how to get it again.
• Retrieve just in time. Keep identifiers in context (file paths, ticket IDs, saved queries) and load the content when it is needed. Claude Code works this way: CLAUDE.md loads up front, and everything else is found with search tools during the task.
• Compact. When the window nears a limit, summarise it and restart from the summary. Keep decisions, open bugs and current state. Drop redundant tool output. Tune for recall first, because a summary that loses a decision is worse than one that runs long.
• Isolate.Send noisy work to a subagent with a clean window and take back a condensed result, often 1,000 to 2,000 tokens. Anthropic reports this working best for research, where exploration parallelises. Coding tasks usually parallelise less, so don't reach for it first.

flowchart LR
    I["New turn"] --> B{"Over budget?"}
    B -- no --> M["Model call"]
    B -- yes --> C["Clear old tool results"]
    C --> B2{"Still over?"}
    B2 -- no --> M
    B2 -- yes --> S["Compact history<br/>into a summary"]
    S --> M
    M --> T["Tool result"]
    T --> N["Write notes<br/>outside the window"]
    N --> I
      

Clearing Tool Results, in Code

This is the first thing I would add to any agent, because it is cheap and the payoff is immediate. It keeps the last few results intact and replaces older ones with a stub the model can act on.

KEEP_RECENT = 3

def clear_old_tool_results(messages: list[dict]) -> list[dict]:
    tool_turns = [i for i, m in enumerate(messages) if m["role"] == "tool"]
    stale = set(tool_turns[:-KEEP_RECENT])

    trimmed = []
    for i, message in enumerate(messages):
        if i in stale:
            ref = message.get("ref", "unknown")
            trimmed.append({**message, "content": f"[cleared: re-run the tool with {ref}]"})
        else:
            trimmed.append(message)
    return trimmed

The stub is the part people skip. “[removed]” leaves the model guessing. A pointer to the query that produced the data lets it fetch the data again if it turns out to matter.

Keep the Stable Part First

Context layout decides your cache hit rate. Prompt caching matches on a prefix, so anything that changes early in the request invalidates everything after it. The classic mistake is a timestamp at the top of the system prompt, which gives you a 0% hit rate that no dashboard explains.

The maths is worth doing once. Take a 50,000-token stable prefix over 20 turns on Claude Sonnet 5.5, which charges $2 per million for input, $2.50 to write the cache and $0.10 to read it. Without caching that prefix costs $2.00. With caching it is $0.125 for the first write plus 19 reads at $0.005 each, about $0.22 in total. Put the things that change at the end: stable instructions and tools first, then memory, then the current turn.

What Goes Wrong

What I Would Do First

Log one long session and break it down by source. Add tool-result clearing with useful stubs. Move anything volatile to the end of the request and check your cache hit rate. Most agents improve noticeably from those three changes alone, before you touch compaction or subagents. When sessions run for hours rather than minutes, this stops being a prompt problem and becomes a harness problem.