Answers from your app became malformed this morning. Nothing changed in your prompts. What is the most useful first step?
Why A is the answer
Compare the raw request and response in your traces to find which side changed
Isolate the layer before changing anything. Traces show whether the integration sent something different or the model returned something different, and each has a different fix.
In detail
- Before changing anything, find out which side changed.
- Traces show exactly what was sent and what came back.
- Each cause has a different fix, so guessing wastes time.
- Keep raw requests and responses in traces, with sensitive data removed.
Use case
An illustrative scenario, not a real incident.
Answers look broken one morning. The trace shows the request is the same as yesterday, but a new library changed how the reply is parsed. The fix goes in the integration, not the prompt.
See it in different ways
flowchart TD
A["Malformed answers"] --> B["Compare raw request and response"]
B --> C{"What changed?"}
C -->|"The request"| D["Fix the integration"]
C -->|"The response"| E["Look at the model or prompt"]stateDiagram-v2 [*] --> Response Response --> Complete: end_turn Response --> RunTool: tool_use Response --> CutOff: max_tokens Response --> Blocked: refusal Response --> Continue: pause_turn Continue --> Response: send content back
flowchart TD
Q{"What differs from a good run?"}
Q -->|"The request"| A0["Fix your integration"]
Q -->|"The response"| A1["Look at the model or prompt"]
Q -->|"Neither"| A2["Check your parsing"]Key point: Find which side changed before you fix anything.
Why the other options are wrong
- B. Rewrite the system prompt with stricter rules about the output format you expect
- Changing the prompt before you know the cause can hide the real problem.
- C. Switch to a different model tier and check whether the problem goes away
- Switching tier is a guess, and it changes cost and behavior.
- D. Add a retry around the whole request and log how often the retry succeeds
- Retries can hide a real fault without explaining it.