ProtoArena: Can AI Agents Discover and Exploit Real-World Cross-Component Protocol Vulnerabilities?

Summary

ProtoArena is a black-box benchmark of 80 cross-component protocol vulnerability-exploitation environments spanning 11 domains and 7 attack categories, primarily informed by published protocol-attack mechanisms. Five LLMs were evaluated, and Claude Opus 4.8 reached 48.8% success under L1 guidance and 12.5% under effect-directed black-box L3. Results show agents can achieve verifiable exploitation effects in some environments, but black-box success remains low and varies across domains.

Topics

• AI agents

Related items

• InfoResearchers Used a Public OpenAI Model to Build a $250,000 Chrome Full-Chain Exploit in Six DaysSimilar attack · The Hacker News
• MediumAn AI Agent Found a Critical OTP Bypass in Production. The Attack Was the Easy Part.Similar attack · The Hacker News
• LowFBI arrests Saudi OpenAI employee who allegedly planted recording device in conference roomSimilar attack · CNBC Technology
• MediumVisual Studio Copilot and SSMS Remote Code Execution (CVE-2026-103572)Similar attack · Embrace The Red
• LowAnthropic warns self-replicating AI could spread to your smart devicesSimilar attack · BleepingComputer