Your cached prompt is reused every 20 minutes, yet you see almost no cache reads. What is the likely cause?
Why B is the answer
The default cache lifetime is 5 minutes, so it expires between uses
The default lifetime is 5 minutes. A 1-hour TTL costs more to write, and it pays off when the same prefix is reused at intervals like this.
In detail
- The default lifetime is 5 minutes, so an entry expires between uses.
- A 1-hour TTL costs more to write, and it pays off at intervals like this.
- Compare the extra write cost with how often the prefix is read.
Use case
An illustrative scenario, not a real incident.
A report job reuses a long prompt every 20 minutes and sees no reads, because the default lifetime is 5 minutes. A 1-hour TTL costs more to write and saves on each read.
See it in different ways
stateDiagram-v2 [*] --> Written: first request Written --> Read: same prefix within the lifetime Read --> Read: reused again Read --> Expired: idle past the lifetime Written --> Expired: idle past the lifetime Expired --> Written: next request
flowchart TD
Q{"How often is the prefix reused?"}
Q -->|"Within 5 minutes"| A0["Default lifetime"]
Q -->|"Every 20 minutes"| A1["1-hour TTL"]Key point: Match the cache lifetime to how often you reuse it.
Why the other options are wrong
- A. Cache entries are removed as soon as a different user sends a request
- A different user does not remove the entry. It simply expired.
- C. Caching does not work with system prompts
- System prompts can be cached.
- D. You are limited to one cache breakpoint
- A single breakpoint is enough for this case, and more are allowed.