How can you evaluate a gen AI answer when there is no exact expected output?
Why B is the answer
LLM-as-a-judge
LLM-as-a-judge uses a language model to score answers, which helps when there is no exact expected output.
In detail
- An LLM can judge an answer against criteria.
- Check the judge against human ratings.
Use case
An illustrative scenario, not a real incident.
There is no single right summary, so a strong model scores answers against a rubric.
See it in different ways
stateDiagram-v2 [*] --> Prompting Prompting --> Rag: lacks current facts Prompting --> FineTuning: quality or format is not reached Prompting --> Done: meets the need Rag --> Done FineTuning --> Done
sequenceDiagram participant P0 as Test set participant P1 as Model under test participant P2 as Judge model P0->>P1: Question P1->>P2: Answer P2-->>P0: Score
Key point: LLM-as-a-judge scores open-ended output.
Why the other options are wrong
- A. A shorter training split
- A split has no bearing on this.
- C. Root mean squared error
- RMSE needs a numeric target.
- D. A confusion matrix on the prompt
- A confusion matrix needs labels.