You must test how well answers address the question that was asked. Which evaluator fits?
Why D is the answer
Relevance
Relevance measures how well a response addresses the query.
In detail
- Relevance measures how well an answer addresses the question.
- Combine it with groundedness for RAG.
Use case
An illustrative scenario, not a real incident.
A bot gives true but unrelated answers. Relevance measures how well each answer addresses what was asked.
See it in different ways
flowchart TD A["Foundry evaluators"] --> B["General: coherence, fluency"] A --> C["RAG: groundedness, relevance"] A --> D["Safety: hate, violence, protected material"] A --> E["Agents: tool call accuracy, task completion"]
mindmap
root((Evaluators))
Quality
Relevance
Coherence
Fluency
RAG
Groundedness
Agents
Tool call accuracy
Safety
Violence
Hate and unfairness
Protected materialsflowchart LR S0["Question"] --> S1["Answer"] --> S2["Relevance score"]
Key point: Relevance checks the answer fits the question.
Why the other options are wrong
- A. Violence
- That is a safety evaluator.
- B. Hate and unfairness
- That is a safety evaluator.
- C. Protected materials
- That detects protected content.