Several small inference jobs waste a whole GPU each. What do you configure?
Why C is the answer
MIG partitions on supported GPUs
MIG splits one GPU into isolated instances, so small jobs stop wasting whole GPUs.
In detail
- MIG splits a GPU, so small jobs share it.
- Only supported GPUs offer MIG.
Use case
An illustrative scenario, not a real incident.
Small inference jobs each hold a whole GPU. MIG splits it, and seven jobs share one GPU.
See it in different ways
flowchart LR A["Enable MIG mode"] --> B["Create GPU instances"] B --> C["Create compute instances"] C --> D["Assign to workloads"]
mindmap
root((Sharing))
Run:ai
Project quotas
Over quota, then preempted
Slurm
Fairshare
MIG
GPU instances
Hardware isolationmindmap
root(("GPU with MIG"))
n1["Instances"]
n2["Job A"]
n3["Job B"]
n4["Job C"]Key point: MIG stops small jobs wasting GPUs.
Why the other options are wrong
- A. A lower clock speed
- A lower clock does not share a GPU.
- B. A larger container
- A larger container does not share a GPU.
- D. More Slurm partitions with the same GPUs
- Partitions do not split one GPU.