You must review responses by hand. What are you looking for?
Why D is the answer
Common issues such as quality and safety
Reading responses by hand finds problems in quality and safety that automatic metrics can miss.
In detail
- Reading responses is a qualitative assessment.
- Look for problems in quality and safety.
- Use a short checklist, so reviews stay consistent.
Use case
An illustrative scenario, not a real incident.
A reviewer reads 50 replies each week and notes wrong facts, a poor tone and any unsafe content. Patterns become fixes and new test cases.
See it in different ways
stateDiagram-v2 state "Read replies" as s0 state "Note problems" as s1 state "Fix and add test cases" as s2 [*] --> s0 s0 --> s1 s1 --> s2 s2 --> s0
mindmap
root(("Manual review"))
n1["Quality"]
n2["Correct"]
n3["Complete"]
n4["On topic"]
n5["Safety"]
n6["Harmful content"]
n7["Private data"]Key point: Look for quality and safety problems.
Why the other options are wrong
- A. The token count of each response
- Token count measures size, not quality.
- B. The number of workers on the cluster
- Worker count is about compute.
- C. The scale-out limit of the serving endpoint
- Scale limits are about capacity.