You must see whether CPU, GPU or data loading is the bottleneck across a training run. What fits?
Why A is the answer
NVIDIA Nsight Systems
Nsight Systems shows a system-wide timeline, so you can see if CPU, GPU or data loading is holding back a run.
In detail
- Nsight Systems shows a system-wide timeline.
- Look at the whole system first.
Use case
An illustrative scenario, not a real incident.
GPUs sit idle during training. A system-wide timeline shows that data loading is the bottleneck.
See it in different ways
flowchart LR
A["Slow training"] --> B["Nsight Systems timeline"]
B --> C{"Where is the time?"}
C -->|"Data loading"| D["Fix the input pipeline"]
C -->|"GPU kernels"| E["Inspect with Nsight Compute"]stateDiagram-v2 [*] --> FitsOnOneGpu FitsOnOneGpu --> DataParallel: yes FitsOnOneGpu --> ModelParallel: no DataParallel --> Profile ModelParallel --> Profile Profile --> [*]: Nsight Systems
flowchart LR S0["Record a run"] --> S1["Read the timeline"] --> S2["Find idle GPUs"] --> S3["Fix the cause"]
Key point: Nsight Systems shows the whole timeline.
Why the other options are wrong
- B. NeMo Guardrails
- Guardrails control behavior.
- C. NVIDIA NGC CLI
- That is for the catalog.
- D. An RLHF labeling tool
- That is for labeling.