Before release, you want to see how a deployed agent handles real conversations over time. What do you integrate?
Why A is the answer
Monitoring and evaluation of agent behavior
Monitoring and evaluation show how a deployed agent behaves across real conversations, not just in one test.
In detail
- Monitoring and evaluation show how an agent behaves over time.
- Add those failures to the evaluation set.
Use case
An illustrative scenario, not a real incident.
A team watches how a deployed agent handles real conversations, and it finds a topic with repeated failures.
See it in different ways
flowchart LR A["App or agent"] --> B["OpenTelemetry traces"] B --> C["Application Insights"] A --> D["Evaluators"] D --> E["Quality and safety scores"]
sequenceDiagram participant A as Agent or app participant O as OpenTelemetry participant M as Application Insights A->>O: Spans for each step O->>M: Exports the trace M-->>A: Debug and monitor
stateDiagram-v2 state "Deployed" as s0 state "Monitor" as s1 state "Evaluate" as s2 state "Improve" as s3 [*] --> s0 s0 --> s1 s1 --> s2 s2 --> s3 s3 --> s0
Key point: Monitor and evaluate behavior over time.
Why the other options are wrong
- B. A static document describing the agent
- A static document does not show behavior.
- C. A larger max tokens setting for the agent
- A token setting does not measure anything.
- D. A longer name for the agent resource, so that it is easier to find
- A name is not monitoring.