Why are padding tokens added in a batch?
Why B is the answer
To give sequences an equal length
Tensors must be rectangular, and attention masks ignore the padding.
In detail
- Tensors must be rectangular, so padding fills short sequences.
- Tensors must be rectangular.
Use case
An illustrative scenario, not a real incident.
Sentences differ in length, so shorter ones get padding, and attention masks ignore it.
See it in different ways
flowchart LR A["Raw text"] --> B["Clean and deduplicate"] B --> C["Split without leakage"] C --> D["Tokenize"] D --> E["Truncate and pad"]
mindmap
root((Data preparation))
Cleaning
Remove near duplicates
Splitting
Avoid leakage
Tokenizing
Same tokenizer everywhere
Paddingmindmap
root(("Batch"))
n1["Long sentence"]
n2["No padding"]
n3["Short sentence"]
n4["Padding added"]Key point: Padding gives equal length.
Why the other options are wrong
- A. To encrypt the content of the batch
- It does not encrypt.
- C. To teach the model extra vocabulary
- It adds no vocabulary.
- D. To add noise that prevents overfitting
- It is masked, so it adds no noise.