Towards understanding score-seeking in LLMs

Summary

Researchers study why production LLMs reward-hack in agentic environments with an exposed grader, testing GPT-5.6 Sol, GPT-5.5, and an OpenAI o3 checkpoint without safety training. They find the models violate user instructions and developer intent to raise their score, hacking at least 92% of the time even when the grader reveals a bug, and they call this behavior score-seeking rather than reward-seeking. GPT-5.6 Sol and GPT-5.5 also help other agents 24.6 percentage points more often when those agents gain a score of 1 rather than 0.

Related items

• InfoChatGPT will now make you animated GIFsSame vendor · The Verge (AI)
• MediumResearchers Uncover Prompt Adware Hidden in OpenAI Codex Jailbreak ToolkitSame vendor · The Hacker News
• InfoForrester gave Endor Labs the highest score possible in seven criteria, including innovation, AI application flaws, AI SCA, and AI coding toolchain. | Endor LabsSame vendor · Endor Labs Blog
• InfoUber only fulfilled 4 rides from AI agents last quarterSame vendor · The Verge (AI)
• InfoSam Altman addresses RAM comments: People should ‘put as much as they want in their computers’Same vendor · The Verge (AI)