What is a common first step for adapting a large model on limited GPUs?
Why C is the answer
Use a parameter-efficient method, and keep most layers frozen
Freezing the base saves memory and compute.
In detail
- Parameter-efficient methods keep most layers frozen, saving memory.
- Freezing saves memory and compute.
Use case
An illustrative scenario, not a real incident.
A team has one GPU, so it freezes most layers and trains an efficient adapter.
See it in different ways
flowchart TD
A{"Is the need new knowledge?"} -->|Yes| B["RAG"]
A -->|"No, it is behavior or style"| C{"Is the budget limited?"}
C -->|Yes| D["LoRA or other PEFT"]
C -->|No| E["Full fine-tuning"]stateDiagram-v2 [*] --> Need Need --> Retrieval: facts that change Need --> FineTuning: stable style or format FineTuning --> Lora: limited GPU budget FineTuning --> FullTuning: budget allows it Lora --> EarlyStopping: small dataset
flowchart LR S0["Freeze the base"] --> S1["Add adapter"] --> S2["Train"]
Key point: Freeze most layers on limited GPUs.
Why the other options are wrong
- A. Pretrain a new model from random initial weights
- That is the most costly.
- B. Double the base model's number of layers
- That adds cost.
- D. Unfreeze all layers and train with full precision
- That uses the most memory.