A slow RAG app does retrieval, then generation. How do you find the bottleneck?
Why D is the answer
Measure each step separately
Measure before you tune, because the fix depends on the slow step.
In detail
- Measure each step, so you fix the real bottleneck.
- Measure each step in production.
Use case
An illustrative scenario, not a real incident.
A RAG app takes eight seconds. Timing shows seven in retrieval, so tuning the model would not have helped.
See it in different ways
flowchart TD A["GenAI evaluation"] --> B["Retrieval: did it find the right passages?"] A --> C["Model: accuracy and relevance"] A --> D["Agent: task completion"] A --> E["Users: feedback"]
stateDiagram-v2 [*] --> InspectLogs InspectLogs --> Retrieval: wrong context InspectLogs --> Prompt: right context, wrong answer Retrieval --> Evaluate Prompt --> Evaluate Evaluate --> [*]
sequenceDiagram participant P0 as App participant P1 as Retrieval participant P2 as Model P0->>P1: Search, 7 seconds P0->>P2: Generate, 1 second
Key point: Measure each step.
Why the other options are wrong
- A. Upgrade the model first
- That is a guess.
- B. Double the Lambda memory first
- That is also a guess.
- C. Ask users to send shorter questions
- That moves the problem to users.