You need domain behavior from a large base model, with limited GPU budget. Which approach fits?
Why D is the answer
Train a LoRA adapter on the base model
LoRA trains a small set of parameters on top of frozen weights, which needs much less memory and compute.
In detail
- A LoRA adapter trains few weights, so it fits a limited budget.
- The base weights stay frozen.
Use case
An illustrative scenario, not a real incident.
A team adapts a large model for legal text on one GPU, by training a LoRA adapter.
See it in different ways
flowchart TD
A{"Is the need new knowledge?"} -->|Yes| B["RAG"]
A -->|"No, it is behavior or style"| C{"Is the budget limited?"}
C -->|Yes| D["LoRA or other PEFT"]
C -->|No| E["Full fine-tuning"]stateDiagram-v2 [*] --> Need Need --> Retrieval: facts that change Need --> FineTuning: stable style or format FineTuning --> Lora: limited GPU budget FineTuning --> FullTuning: budget allows it Lora --> EarlyStopping: small dataset
mindmap
root(("LoRA"))
n1["Frozen"]
n2["Base model weights"]
n3["Trained"]
n4["Small adapter"]Key point: LoRA adapts with few parameters.
Why the other options are wrong
- A. Retrain the full model from scratch
- A full retrain is the most costly.
- B. Delete the base model's layers
- That destroys the model.
- C. Ensemble ten base models
- Ten models cost more.