Reward model calibration is audited to prevent alignment drift during reinforcement learning from human feedback (RLHF).
Outline Introduction: Defining the challenge of RLHF and why the reward model is a “moving target.” Key Concepts: Reward model…
preferences