A column has a few missing values and the rows are valuable. What is a common approach?
Why B is the answer
Impute the missing values
Imputation keeps the rows. Fit it on training data only, to avoid leakage.
In detail
- Imputing keeps the valuable rows.
- Fitting on all data causes leakage.
Use case
An illustrative scenario, not a real incident.
A few ages are missing in valuable records. Imputing the median keeps them, and the value is computed on training data only.
See it in different ways
flowchart LR A["Raw data in S3"] --> B["Clean and transform"] B --> C["Feature engineering"] C --> D["Check quality and bias"] D --> E["Training set"]
mindmap
root((Data preparation))
Missing values
Impute
Categories
One-hot encoding
Ranges
Scaling
Imbalance
Resampling
Class weights
Leakage
Remove future dataflowchart TD
Q{"How much is missing?"}
Q -->|"A little, rows are valuable"| A0["Impute"]
Q -->|"Most of a column"| A1["Consider dropping it"]Key point: Impute to keep valuable rows.
Why the other options are wrong
- A. Stop the project until the source data is complete
- Perfect data is rarely available.
- C. Delete the entire column, whatever it contains
- Deleting the column loses information.
- D. Replace the missing values with a random string
- A random string adds noise.