What is the point of multiple attention heads?
Why B is the answer
To capture different relationships in parallel
Each head learns its own projections.
In detail
- Each head learns its own projections, and captures different relationships.
- Each head learns its own projections.
Use case
An illustrative scenario, not a real incident.
One head might follow subject and verb, while another follows nearby words.
See it in different ways
flowchart LR A["Input embeddings"] --> B["Query, key, value"] B --> C["Several attention heads"] C --> D["Concatenate"] D --> E["Linear layer"]
mindmap
root((Transformer block))
Attention
Query, key, value
Several heads
Stability
Residual connections
Normalization
Feed-forward
Per tokenmindmap
root(("Multi-head attention"))
n1["Head 1"]
n2["One relationship"]
n3["Head 2"]
n4["Another relationship"]Key point: Heads capture different relationships.
Why the other options are wrong
- A. To reduce the vocabulary used by the tokenizer
- Vocabulary is separate.
- C. To store separate copies of the training data
- Heads do not store data.
- D. To run a different model for each language
- It is one model.