What does pruning remove from a model?
Why B is the answer
Weights that contribute little
Pruning removes low-value parameters, and may need retraining to recover quality.
In detail
- Pruning removes low-value weights.
- Measure after pruning.
Use case
An illustrative scenario, not a real incident.
A team removes weights that add little, then retrains briefly to recover quality.
See it in different ways
flowchart TD
A{"Which goal?"} -->|"Fewer bits per weight"| B["Quantization"]
A -->|"Remove low-value weights"| C["Pruning"]
A -->|"Teach a smaller model"| D["Knowledge distillation"]stateDiagram-v2 [*] --> Goal Goal --> Quantization: smaller weights Goal --> Pruning: fewer weights Goal --> Distillation: a smaller student model Goal --> Speculative: fewer main model steps
stateDiagram-v2 state "Full model" as s0 state "Pruned model" as s1 state "Fine-tuned model" as s2 [*] --> s0 s0 --> s1: remove weights s1 --> s2: retrain s2 --> [*]
Key point: Pruning removes low-value weights.
Why the other options are wrong
- A. Any prompts that exceed the context window
- That is input validation.
- C. The training examples that had the highest loss
- That is data cleaning.
- D. Rare tokens from the model's input vocabulary
- That changes vocabulary.