Simple requests should go to a cheaper model, and hard ones to a stronger model. What do you implement?
Why B is the answer
A model router
A model router sends each request to the most suitable model, which controls cost while keeping quality for hard cases.
In detail
- A model router sends each request to a suitable model.
- Check quality on each route, not only the savings.
Use case
An illustrative scenario, not a real incident.
A help desk sends password questions to a small model and contract analysis to a strong one, so most requests cost less.
See it in different ways
flowchart LR
A["Request"] --> B{"Model router"}
B -->|"Simple"| C["Lower-cost model"]
B -->|"Complex"| D["More capable model"]sequenceDiagram
participant U as Request
participant R as Model router
participant S as Smaller model
participant L as Stronger model
U->>R: Prompt
alt Simple
R->>S: Handle it
else Hard
R->>L: Handle it
endflowchart TD
Q{"How hard is the request?"}
Q -->|"Simple"| A0["Cheaper model"]
Q -->|"Hard"| A1["Stronger model"]Key point: A router matches requests to models.
Why the other options are wrong
- A. A longer system prompt
- A longer prompt does not route anything.
- C. A second tenant
- A tenant is an organizational boundary.
- D. A larger quota on the strong model
- More quota sends everything to the costly model.