A model scores 99% on training data and 70% on new data. What is happening?
Why B is the answer
Overfitting
Doing far better on training data than on new data is the classic sign of overfitting.
In detail
- High training scores and low scores on new data mean overfitting.
- Regularization and more varied data both help.
Use case
An illustrative scenario, not a real incident.
A churn model scores 99% on past customers but 70% on new ones. It memorized the training data instead of learning general patterns.
See it in different ways
flowchart TD
A{"Performance"} -->|"Poor on both"| B["Underfitting: high bias"]
A -->|"Good on training, poor on test"| C["Overfitting: high variance"]
A -->|"Good on both"| D["Good fit"]stateDiagram-v2 [*] --> Compare Compare --> Underfit: poor on both Compare --> Overfit: good on training, poor on new data Compare --> GoodFit: good on both
flowchart TD
Q{"How does it do on new data?"}
Q -->|"About the same"| A0["Generalizing well"]
Q -->|"Far worse"| A1["Overfitting"]Key point: A large gap between training and new data signals overfitting.
Why the other options are wrong
- A. Underfitting
- Underfitting scores poorly on both.
- C. Data drift
- Drift is a change over time after deployment.
- D. Tokenization error
- Tokenization is not the issue here.