Training loss keeps falling while validation loss rises. What is happening?
Why B is the answer
Overfitting
The model is memorizing training data rather than generalizing.
In detail
- Rising validation loss with falling training loss is overfitting.
- Stop early, or add regularization.
Use case
An illustrative scenario, not a real incident.
Training loss falls while validation loss rises, so the model is memorizing.
See it in different ways
flowchart TD
A{"Training loss falling?"} -->|No| B["Underfitting or a learning-rate problem"]
A -->|Yes| C{"Validation loss falling?"}
C -->|Yes| D["Keep training"]
C -->|"No, it rises"| E["Overfitting: regularize or stop early"]stateDiagram-v2 [*] --> Watch Watch --> Healthy: both losses fall Watch --> Overfitting: validation loss rises Watch --> Unstable: loss swings wildly Overfitting --> Regularize Unstable --> LowerLearningRate Regularize --> Watch LowerLearningRate --> Watch
stateDiagram-v2 state "Both falling" as s0 state "Learning well" as s1 state "Overfitting" as s2 [*] --> s0 s0 --> s1 s1 --> s2: validation loss rises s2 --> [*]
Key point: Diverging loss means overfitting.
Why the other options are wrong
- A. The learning rate has reached exactly zero
- At zero, learning would stop.
- C. The tokenizer vocabulary is too small for the model
- Vocabulary would not cause this pattern.
- D. The GPU is throttling because of its temperature
- Throttling slows training, and does not cause this.