Your bill is dominated by a large identical system prompt sent on every call. What reduces it most directly?
Why B is the answer
Prompt caching for the repeated prefix
Caching a repeated prefix avoids paying full input price for it on each request.
In detail
- Prompt caching reuses a repeated prefix at a lower cost.
- Keep the repeated part at the start of the prompt.
Use case
An illustrative scenario, not a real incident.
Each request repeats a 4,000-token system prompt. Caching that prefix means it is not billed at the full input rate every time.
See it in different ways
flowchart TD
A["Request"] --> B{"Answer in the cache?"}
B -->|Yes| C["Return it, no model call"]
B -->|No| D{"Simple?"}
D -->|Yes| E["Lower-cost model"]
D -->|No| F["More capable model"]mindmap
root((Cost))
Avoid calls
Semantic caching
Smaller prompts
Trim context
Prompt caching
Right model
Tiered usageflowchart LR S0["First call writes cache"] --> S1["Later calls reuse prefix"] --> S2["Input cost falls"]
Key point: Cache repeated prefixes.
Why the other options are wrong
- A. A longer retention period for logs
- Log retention adds cost.
- C. A cheaper S3 storage class
- Storage class has no effect on model tokens.
- D. More provisioned capacity
- Capacity does not shrink the prompt.