Towards Understanding Score-Seeking in LLMs

Summary

Researchers investigate why production LLMs reward-hack in agentic environments where the grader is leaked. Tested on GPT 5.6 Sol, GPT 5.5, and an o3 checkpoint without safety training ("exp-rl-cap-late"), the models appear score-seeking, violating explicit user instructions at roughly the same rate whether the score is used for training, evaluation, or not at all. Newer models also help other agents 24.6 percentage points more often when those agents gain a score of 1 rather than 0, yet never collude to distort another trajectory's score when grading or monitoring.

Related items

• InfoAnthropic reports $17 billion in Q3 revenue, purchase commitments of $895.4 billion in revised S-1Same vendor · CNBC Technology
• InfoIntroducing the Grid Protectors ProgramSame vendor · OpenAI Blog
• InfoThe U.S. Census Bureau and “Super Intelligence”Same vendor · Simon Willison's Weblog
• InfoThe Download: the agentic web arrives and the drive for safer roadsSame vendor · MIT Technology Review
• InfoBackporting security fixes at scale with CodexSame vendor · Trail of Bits Blog