What does RLHF add when training an LLM?
Why C is the answer
Human preferences that steer behavior
RLHF stands for reinforcement learning from human feedback.
In detail
- RLHF uses human preferences to steer behavior.
- RLHF means reinforcement learning from human feedback.
Use case
An illustrative scenario, not a real incident.
People rank model replies, and those preferences steer training toward helpful behavior.
See it in different ways
flowchart LR A["Collect and tokenize"] --> B["Pretraining"] B --> C["Fine-tuning"] C --> D["RLHF"] D --> E["Evaluation"]
stateDiagram-v2 [*] --> Tokenize Tokenize --> Pretraining Pretraining --> FineTuning FineTuning --> RLHF RLHF --> Evaluation Evaluation --> [*]
flowchart LR S0["Model replies"] --> S1["People rank them"] --> S2["Training uses rankings"]
Key point: RLHF uses human preferences.
Why the other options are wrong
- A. A larger vocabulary for the tokenizer to work with
- It is not about vocabulary.
- B. Scraped documents that increase the training set
- It is not a scraping step.
- D. Extra GPUs that are assigned during fine-tuning
- It is not about GPUs.