Training is too slow on one GPU, and the model fits in memory. What helps?
Why B is the answer
Data parallelism
Data parallelism splits the batches across devices.
In detail
- If the model fits, data parallelism speeds up training.
- Scaling is rarely perfectly linear.
Use case
An illustrative scenario, not a real incident.
A model fits one GPU but takes days. Splitting batches across four GPUs shortens the job.
See it in different ways
flowchart TD
A{"What is the bottleneck?"} -->|"Training is too slow"| B["Data parallelism"]
A -->|"Model exceeds one device"| C["Model parallelism"]stateDiagram-v2 [*] --> FitsInMemory FitsInMemory --> DataParallelism: yes, but too slow FitsInMemory --> ModelParallelism: no DataParallelism --> Train ModelParallelism --> Train Train --> Tune: hyperparameter tuning Tune --> [*]
mindmap
root(("Data parallelism"))
n1["GPU 1"]
n2["Batch part 1"]
n3["GPU 2"]
n4["Batch part 2"]Key point: Data parallelism splits batches.
Why the other options are wrong
- A. A public endpoint
- An endpoint serves predictions.
- C. Model parallelism
- That is for models that do not fit.
- D. A smaller test set
- A test set has no effect on training.