Cache hits vanish after a teammate reorders your tool definitions. Why?
Why A is the answer
Tools sit at the start of the prefix, so the change invalidates the whole cache
The prefix runs tools, system, then messages, and hits need identical segments. Changing tools invalidates everything after them.
In detail
- The cache prefix runs in order: tools, then system, then messages.
- A hit needs identical content up to the breakpoint.
- Tools come first, so reordering them invalidates everything after.
- Generate tool definitions in a stable order, so a deployment does not change them by accident.
Use case
An illustrative scenario, not a real incident.
A teammate sorts the tool list alphabetically. Cache hits fall to zero, because tools sit at the start of the prefix. Keeping tools in a fixed order brings the hits back.
See it in different ways
flowchart LR A["Tools"] --> B["System prompt"] B --> C["Long reference docs"] C --> D["Cache breakpoint"] D --> E["New user question"]
flowchart LR A["Tools"] --> B["System"] B --> C["Messages"] C --> D["Newest turn, not cached"]
stateDiagram-v2 state "Cache hit" as s0 state "Tools reordered" as s1 state "Cache miss" as s2 state "New entry written" as s3 [*] --> s0 s0 --> s1: change s1 --> s2 s2 --> s3 s3 --> [*]
Key point: Any change to the prefix invalidates what comes after it.
Why the other options are wrong
- B. Cache entries are tied to one person and expire when a teammate edits them
- Cache entries are not tied to a person in that way. The content changed.
- C. Reordered tools turn caching off until a new cache breakpoint has been added
- Caching stays on. The existing entries simply no longer match.
- D. Tool definitions are never cached, so the loss must have another cause
- Tool definitions are part of the cached prefix.