Your error tracking shows no failures, but some users get no useful answer. The responses are HTTP 200 with stop_reason refusal. What should change?
Why B is the answer
Handle refusal as an outcome and branch on stop_reason
A refusal is a normal HTTP 200 response, not an error, so status-based monitoring misses it. Read stop_reason on every response.
In detail
- A refusal is a normal HTTP 200 response, not an error.
- Monitoring based on status codes cannot see it.
- Read stop_reason on every response, and branch on it.
- Count refusals in monitoring, as a separate outcome.
Use case
An illustrative scenario, not a real incident.
Error tracking is clean, but some users receive no answer. Refusals are 200s, so nothing alerts. A branch on stop_reason shows a helpful message and records the event.
See it in different ways
flowchart LR
A{"stop_reason"} -->|"end_turn"| B["Use the answer"]
A -->|"tool_use"| C["Run tools, send results"]
A -->|"max_tokens"| D["Reply was cut off"]
A -->|"pause_turn"| E["Send content back to continue"]
A -->|"refusal"| F["Handle the refusal"]stateDiagram-v2 [*] --> Response Response --> Complete: end_turn Response --> RunTool: tool_use Response --> CutOff: max_tokens Response --> Blocked: refusal Response --> Continue: pause_turn Continue --> Response: send content back
flowchart TD
Q{"Response is 200. What is stop_reason?"}
Q -->|"refusal"| A0["Show a helpful message and record it"]
Q -->|"end_turn"| A1["Show the answer"]Key point: A refusal is a 200, so branch on stop_reason.
Why the other options are wrong
- A. Retry the same request in a tight loop
- Retrying the same request in a tight loop adds load and changes nothing.
- C. Treat every HTTP 200 as complete, and alert only on 4xx and 5xx responses
- This is the current gap that hides these failures.
- D. Raise max_tokens until the refusals stop
- A refusal is not a length problem.