Morning Edition · Tuesday, July 21, 2026Published at 1:32 AM EDT · New York

Researchers Warn That RLHF Preference Data Encodes the Rater's State, Not Just the Output

An audit framework identifies a structured confound in human feedback, where a labeler's condition during annotation leaks into the preference signal.

A new arXiv paper identifies a structured confound in reinforcement learning from human feedback (RLHF). Pairwise preference labels are meant to reflect the compared model outputs, but the authors argue they can also reflect the rater's sta…