What tells you whether an answer addresses what the user asked?
Why A is the answer
Relevance
Relevance measures fit to the query. An answer can be grounded yet off-topic.
In detail
- Relevance measures whether an answer addresses the question.
- Groundedness alone would pass it.
Use case
An illustrative scenario, not a real incident.
A bot gives accurate text that misses the point of the question. Relevance catches this.
See it in different ways
flowchart LR A["Test dataset"] --> B["Map columns to evaluator inputs"] B --> C["Quality and safety evaluators"] C --> D["Compare with the baseline"] D --> E["Release decision"]
sequenceDiagram participant C as Change participant P as CI/CD pipeline participant R as Production participant T as Traces C->>P: Runs evaluators on a test dataset P->>R: Releases if it passes R->>T: Records real traffic T->>P: New failing cases
flowchart LR S0["Question"] --> S1["Answer"] --> S2["Relevance score"]
Key point: Relevance checks the question was answered.
Why the other options are wrong
- B. Token count
- Token count measures size.
- C. Groundedness alone
- A grounded answer can still miss the question.
- D. Requests per minute
- That measures traffic.