Quality slowly falls, although the model has not changed. What should you suspect?
Why C is the answer
Drift in the incoming data
User inputs change over time, while the model stays fixed.
In detail
- Inputs change over time, while the model stays fixed.
- Keep monitoring inputs.
Use case
An illustrative scenario, not a real incident.
The model is unchanged, but users ask about a new product line. That is drift.
See it in different ways
flowchart LR
A["New version"] --> B["Canary traffic"]
B --> C{"Metrics good?"}
C -->|Yes| D["Full rollout"]
C -->|No| E["Roll back"]stateDiagram-v2 [*] --> Production Production --> Drift: inputs change Drift --> Retrain Retrain --> Compare: same test as the old version Compare --> Canary Canary --> Production: metrics hold Canary --> Rollback: metrics fall
flowchart LR S0["Quality falls"] --> S1["Compare inputs with past data"] --> S2["Refresh the model or data"]
Key point: Inputs change while the model stays fixed.
Why the other options are wrong
- A. A shortage of disk space on the build server
- A build server is unrelated.
- B. A problem with the tokenizer's file permissions
- Permissions would cause errors.
- D. A change in the model's weights on disk
- The weights have not changed.