What does LoRA train?
Why C is the answer
Small low-rank adapters on top of frozen weights
LoRA is a parameter-efficient method. It trains few weights.
In detail
- LoRA trains small adapters over frozen weights.
- Adapters can be swapped.
Use case
An illustrative scenario, not a real incident.
LoRA adds small adapters to frozen weights, so training touches few parameters.
See it in different ways
flowchart TD
A{"Is the need new knowledge?"} -->|Yes| B["RAG"]
A -->|"No, it is behavior or style"| C{"Is the budget limited?"}
C -->|Yes| D["LoRA or other PEFT"]
C -->|No| E["Full fine-tuning"]stateDiagram-v2 [*] --> Need Need --> Retrieval: facts that change Need --> FineTuning: stable style or format FineTuning --> Lora: limited GPU budget FineTuning --> FullTuning: budget allows it Lora --> EarlyStopping: small dataset
mindmap
root(("Model with LoRA"))
n1["Most weights"]
n2["Frozen"]
n3["Adapters"]
n4["Trained"]Key point: LoRA trains small adapters.
Why the other options are wrong
- A. All base model weights from a random starting point
- That is full training.
- B. A separate retrieval index for each adapter
- That is about retrieval.
- D. A new tokenizer that replaces the base vocabulary
- It does not replace the tokenizer.