Replication of Agentic Security Events: Audit and Investigation (ROGUE-AI)
- Published
- Record updated
Summary
ROGUE-AI is a multi-agent framework that rebuilds reported LLM agent incidents offline, paired with a control lacking only the blamed cause. It reproduced the reported harm in 89% of 108 attempted replications, and agents built on five GPT, Claude, Gemini and DeepSeek models repeated the mistake in 18% to 33% of runs, versus at most 1% without the blamed cause. Of 72 incidents with real-world harm, 62 had no attacker, and agents missed ordinary hazards such as a database shared with the live site.
Mitigation
A warning after each tool output that names the hazard and its consequence lowered harm for all five models, whereas a same-length note that does not mention the hazard left most harm in place. The authors also suggest vendors rerun the ROGUE-AI replications as an audit before each new release.
Topics
Related items
- InfoIn a year of AI solving maths problems, Anthropic may have just solved the biggestSame vendor · BBC Technology
- InfoAnthropic shares fall after blockbuster opening as wider tech sell-off continuesSame vendor · BBC Technology
- MediumBazuka Android Malware Uses Fake Screen Overlays to Hide Credential Theft and Device ControlSame vendor · The Hacker News
- InfoAnthropic mathematician claims to have solved prestigious Millennium Prize problem with ClaudeSame vendor · The Verge (AI)
- InfoQuoting Levent AlpögeSame vendor · Simon Willison's Weblog