The Target Agent Is Its Own Best Critic but a Poor Proposer: Self-Improved Red Teaming

Summary

This paper studies how automatic red teaming of LLM agents should split its roles. It finds that self-criticism by the target session is more effective than external criticism, because it draws on live execution context, and that self-proposal is weaker because awareness of the adversarial goal triggers refusal and sandbagging. A role-separated setup, with the target diagnosing failures and a separate attacker proposing the next attack, reaches 63.2% overall attack success rate on DTAP-BENCH.

Related items

• MediumIn-the-Wild Attack Abuses AI Agents' Trust in MarkdownSimilar attack · Dark Reading
• InfoI-HITL: Inalienable Human-in-the-LoopSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoThen there is prompt injection, a malicious email, ...: The Wild Wide West of OpenClaw Agentic System SecuritySimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumMalicious downloads can trick AI coding agents into compromising themselvesSimilar attack · CSO Online
• InfoCoSA: The Conflict of Skills Attack in Real-World LLM Agent EcosystemsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)