Which hardware choice matters most for training a large model on a single server?
Why D is the answer
GPU count and memory
Model size and training speed depend on GPU capacity.
In detail
- Model size and speed depend on GPU capacity.
- Capacity limits both size and speed.
Use case
An illustrative scenario, not a real incident.
A large model will not fit in a small GPU's memory, and fewer GPUs mean slower training.
See it in different ways
flowchart LR A["CPU: few fast cores"] --> B["Serial control work"] C["GPU: many parallel cores"] --> D["Matrix math"]
mindmap
root((AI cluster))
Compute
GPU nodes
NVLink
Fabric
InfiniBand
Ethernet
Storage
High throughput
Management
Out-of-band networkmindmap
root(("Single server"))
n1["GPU memory"]
n2["Model size"]
n3["GPU count"]
n4["Training speed"]Key point: GPU capacity limits training.
Why the other options are wrong
- A. The server's boot disk speed and its capacity
- A boot disk does not drive training.
- B. The number of drive bays in the chassis
- Drive bays do not limit training.
- C. The size of the management network switch
- The management switch is not in the data path.