A company wants a model to answer from its own frequently changing policies. Which approach is the most practical?
Why B is the answer
Use RAG to retrieve current policies at request time
RAG reads current content at question time, so an update takes effect without retraining.
In detail
- RAG retrieves current policies at request time.
- Keep the retrieval index in step with the documents.
Use case
An illustrative scenario, not a real incident.
HR policies change monthly. With RAG, the assistant reads the current policy at question time, so nobody retrains anything.
See it in different ways
sequenceDiagram participant U as User participant K as Knowledge base participant M as Foundation model U->>K: Question K-->>M: Relevant chunks with the question M-->>U: Grounded answer
sequenceDiagram participant U as User participant K as Knowledge Bases participant S as Vector store participant M as Foundation model U->>K: Question K->>S: Finds similar chunks S-->>K: Passages K->>M: Prompt with passages M-->>U: Answer
sequenceDiagram participant P0 as User participant P1 as App participant P2 as Policy store participant P3 as Model P0->>P1: Question P1->>P2: Find current policy P1->>P3: Question and policy P3-->>P0: Answer
Key point: RAG suits frequently changing knowledge.
Why the other options are wrong
- A. Fine-tune the model every time a policy changes
- Fine-tuning each change is slow and costly.
- C. Write every policy into the system prompt
- A prompt holding every policy gets huge.
- D. Train a new model from scratch
- A new model is far too much.