A chatbot forgets what the user said earlier. What must the application do?
Why B is the answer
Send the relevant history with each request
LLM APIs are usually stateless, so the app supplies the context.
In detail
- LLM APIs are usually stateless, so the app sends the history.
- Long history uses tokens.
Use case
An illustrative scenario, not a real incident.
A chatbot forgets a user's name, because the app never sent earlier turns.
See it in different ways
mindmap
root((LLM service))
Hardware
GPU memory
Throughput
Batching
Experience
Streaming
Conversation history
Packaging
NIMsequenceDiagram participant P0 as User participant P1 as App participant P2 as LLM API P0->>P1: New message P1->>P2: History and new message P2-->>P0: Reply
Key point: Apps supply conversation context.
Why the other options are wrong
- A. Store the conversation in the model's weights
- Conversations are not stored in weights.
- C. Retrain the model after each user message
- Retraining is far too costly.
- D. Increase the sampling temperature of the model
- Temperature has no link to memory.