Most requests are simple, but a few need deep reasoning. How do you cut cost?
Why B is the answer
Tiered model usage based on complexity
Tiered usage sends simple requests to cheaper models, and only complex ones to stronger models.
In detail
- Send simple requests to cheaper models.
- Watch quality on the cheap route.
Use case
An illustrative scenario, not a real incident.
Most tickets are password resets, and a small model handles them. A strong model handles the few complex ones.
See it in different ways
flowchart TD
A["Request"] --> B{"Answer in the cache?"}
B -->|Yes| C["Return it, no model call"]
B -->|No| D{"Simple?"}
D -->|Yes| E["Lower-cost model"]
D -->|No| F["More capable model"]mindmap
root((Cost))
Avoid calls
Semantic caching
Smaller prompts
Trim context
Prompt caching
Right model
Tiered usageflowchart TD
Q{"How hard is it?"}
Q -->|"Simple"| A0["Smaller model"]
Q -->|"Complex"| A1["Stronger model"]Key point: Tier models by complexity.
Why the other options are wrong
- A. A single fixed model, chosen once a year
- A fixed model ignores differences.
- C. The most capable model for all traffic
- That overpays on simple work.
- D. A longer system prompt for each request
- That adds cost.