You put a large system prompt behind prompt caching, but the cache hit rate is zero. The prompt begins with the current timestamp. What is wrong?
Why B is the answer
The changing timestamp alters the prefix, so no two requests match
Caching matches on an exact prefix. Anything that changes at the start invalidates everything after it, so stable content goes first and variable content goes last.
In detail
- Caching matches on an exact prefix.
- A timestamp at the start makes each request different from the first character, so nothing can match.
- Move variable content to the end of the prompt.
- Keep timestamps, user names and other changing values after the cached content.
Use case
An illustrative scenario, not a real incident.
A travel app puts "Today is 12 May" at the top of its system prompt. Every day starts a new prefix, and within the day any change misses. Moving the date to the end of the request restores cache hits.
See it in different ways
flowchart LR A["Tools"] --> B["System"] B --> C["Messages"] C --> D["New user turn"]
flowchart LR A["Tools"] --> B["System"] B --> C["Messages"] C --> D["Variable content last"]
flowchart TD
Q{"Where is the changing value?"}
Q -->|"Before stable content"| A0["No cache hits"]
Q -->|"After stable content"| A1["Prefix can match"]Key point: Anything that changes goes after the stable content.
Why the other options are wrong
- A. Caching is only available when you use the Message Batches API
- Caching works on standard requests, not only batches.
- C. Caching is supported on Opus models only, so other tiers never get a hit
- Caching is not limited to Opus models.
- D. The system prompt is too long for the cache, so it gets skipped entirely
- Length is not the failure here, because the prefix changes on each request.