What should decide the choice of foundation model?
Why A is the answer
Fit for the use case, measured on your own data
Selection should rest on your use case, cost and evaluation results.
In detail
- Measure fit on your own data.
- Compare cost and latency too.
Use case
An illustrative scenario, not a real incident.
A team tests three models on its own data. The one that ranks lower publicly fits best, and it costs less.
See it in different ways
mindmap
root((GenAIOps))
Prompts
In Git
Variants
Models
Fit on your data
Gradual rollout
Capacity
Standard
Provisioned
Evaluation
In CI/CD
On production tracesflowchart TD
Q{"Where does the score come from?"}
Q -->|"Your own data"| A0["Relevant to your use case"]
Q -->|"A public leaderboard"| A1["General ability only"]Key point: Measure fit on your own data.
Why the other options are wrong
- B. Whichever model has the most recent release date
- Release date is not fit.
- C. The model with the longest context window available
- A long window may add cost, with no benefit.
- D. The model with the highest public benchmark score
- Public scores may not reflect your task.