A client asks for an agent to answer questions from a fixed policy handbook. What do you propose first?
Why D is the answer
A single model call with retrieval, as the simplest thing that could work
Add complexity only when it improves outcomes. Retrieval plus one call answers this, and it is cheaper to run and easier to test.
In detail
- Start with the simplest thing that could work.
- Retrieval plus one model call answers from a fixed handbook.
- It is cheaper to run and easier to test.
- Add complexity only when an eval shows the need.
Use case
An illustrative scenario, not a real incident.
A client wants an agent for its policy handbook. A single call with retrieval answers well, costs little and is easy to test, so no agent is needed.
See it in different ways
flowchart LR A["Single call with retrieval"] --> B["Workflow"] B --> C["Single agent"] C --> D["Multi-agent system"]
stateDiagram-v2 [*] --> SingleCall SingleCall --> Workflow: steps are known Workflow --> Agent: path depends on findings Agent --> MultiAgent: evals show overflow or specialist failures MultiAgent --> [*]
stateDiagram-v2 state "Single call with retrieval" as s0 state "Measure on evals" as s1 state "Keep it simple" as s2 state "Add a part" as s3 [*] --> s0 s0 --> s1 s1 --> s2: targets met s1 --> s3: measured gap s2 --> [*] s3 --> [*]
Key point: Start with the simplest thing that could work.
Why the other options are wrong
- A. A fine-tuned model trained on the handbook and retrained after every policy change
- Fine-tuning fixes content into the weights, so each policy change needs retraining.
- B. A multi-agent system with a planner, a researcher and a separate reviewing agent
- Three agents add cost for no measured benefit.
- C. An autonomous agent that decides for itself which handbook sections it should read
- An autonomous agent is not needed to find sections in a handbook.