An agent's answers are often rated unhelpful for one topic. What is a sensible next step?
Why A is the answer
Inspect those conversations and fix the knowledge or topic
Use the evidence to find the cause, then fix it and measure again.
In detail
- Read those conversations.
- Fix the knowledge or the topic.
- Measure again after the fix.
Use case
An illustrative scenario, not a real incident.
Answers about returns get poor ratings. Reading those conversations shows an outdated policy, and updating it lifts the ratings.
See it in different ways
flowchart LR A["Agent metrics"] --> B["User feedback and backlog"] B --> C["Tune"] C --> A
stateDiagram-v2 [*] --> Pilot Pilot --> Measure Measure --> Improve: low ratings on a topic Improve --> Measure Measure --> Expand: targets met Measure --> Rollback: serious problem
flowchart LR S0["Find poor ratings"] --> S1["Read conversations"] --> S2["Fix the source"] --> S3["Measure again"]
Key point: Follow the evidence to the cause.
Why the other options are wrong
- B. Remove the rating feature from the agent, so that the score can no longer fall
- Removing ratings hides the problem.
- C. Move the agent to a new tenant
- A new tenant changes nothing.
- D. Add the same topic several more times
- Duplicate topics add confusion.