An agent must work with a slow vendor API that often times out. How should the design handle this?
Why A is the answer
Retry in code with backoff, and pass a clear error to the agent if it still fails
Routine transient failures belong in code. The model should see a clear result, not a stream of retry noise.
In detail
- Routine transient failures belong in code.
- Retry with backoff there, and give the agent a clear error if it still fails.
- The model should not have to read retry noise.
- Add a circuit breaker if the vendor stays down.
Use case
An illustrative scenario, not a real incident.
A shipping API times out often. Code retries with backoff, and the agent sees only the final result or a clear error, so it stays focused on the customer.
See it in different ways
sequenceDiagram
participant U as Request
participant F as Fast model
participant V as Validation
participant S as Stronger model
U->>F: Handle first
F->>V: Check the output
alt Fails validation
V->>S: Escalate
else Passes
V-->>U: Return the answer
endflowchart LR S0["Call vendor"] --> S1["Timeout"] --> S2["Retry with backoff"] --> S3["Result or clear error"]
Key point: Handle routine failures in code.
Why the other options are wrong
- B. Remove the timeouts entirely, so that each call waits for as long as it takes
- A call with no timeout can hang the whole agent.
- C. Run ten parallel copies of the same call and keep whichever one returns first
- Ten copies add load to a vendor that is already slow.
- D. Let the model decide how to deal with each timeout, in free-form prose, on every single call
- This spends model calls on a problem that code handles reliably.