Evaluating and monitoring for AI scheming

Summary

Google DeepMind researchers evaluated whether current frontier models have the prerequisite capabilities for AI scheming, namely stealth and situational awareness. They built and open-sourced an evaluation suite and tested Gemini 2.5 Pro, GPT-4o and Claude 3.7 Sonnet as of May 2025. The most capable models passed 2 of 5 stealth challenges and 2 of 11 situational awareness challenges, which the authors read as no concerning levels of either capability.

Topics

• Frontier model safety

Related items

• MediumHackers abuse Google Ads, Bing redirects to push Claude ClickFix attacksSame vendor · BleepingComputer
• InfoGoogle is launching a one-stop Gemini agent for your work tasksSame vendor · The Verge (AI)
• MediumUAT-11985: AI-assisted event lures delivering real-time Google AitM phishingSame vendor · Cisco Talos Blog
• MediumTop MCP security resources — October 2026Same vendor · Adversa AI Blog
• InfoThe Pentagon Hopes to Speed Up ‘Kill Chain’ AI Buys With 5-Minute VideosSame vendor · Wired (Security)