Predicting model behavior before release by simulating deployment

Summary

OpenAI describes Deployment Simulation, a method that replays privacy-preserving samples of past conversations with a new candidate model before release to preview its behavior. The company says the method improved estimates of undesired behavior rates across multiple GPT-5-series Thinking deployments, surfaced novel misalignment, and reduced models' ability to recognize they are being tested. It can measure behaviors occurring at no less than 1 in 200,000 messages.

Related items

• InfoChatGPT will now make you animated GIFsSame vendor · The Verge (AI)
• MediumResearchers Uncover Prompt Adware Hidden in OpenAI Codex Jailbreak ToolkitSame vendor · The Hacker News
• InfoForrester gave Endor Labs the highest score possible in seven criteria, including innovation, AI application flaws, AI SCA, and AI coding toolchain. | Endor LabsSame vendor · Endor Labs Blog
• InfoUber only fulfilled 4 rides from AI agents last quarterSame vendor · The Verge (AI)
• InfoSam Altman addresses RAM comments: People should ‘put as much as they want in their computers’Same vendor · The Verge (AI)