PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents
- Published
- Record updated
Summary
Researchers show that RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse: they repeatedly find similar indirect prompt injection strategies, so high attack success rates during red-teaming do not produce robustness gains in later adversarial training. They introduce PIPETTE, a purple-teaming framework that uses a dictionary learning-based reward for broader attack coverage and an on-policy adversarial training pipeline. On AgentDojo and ASB, PIPETTE yields defenders with near-zero attack success rates on static attacks that withstand strong adaptive attacks, with benign task utility close to the undefended baseline.
Mitigation
PIPETTE, a unified prompt injection purple-teaming framework that converts red-teaming discoveries into defender robustness improvements, using a dictionary learning-based reward and an on-policy adversarial training pipeline tailored to IPI defense.
Topics
Related items
- HighGHSA-3mgh-93r5-r7jf: CodeWhale: exec_shell parallel-readonly classifier auto-approves fd/rg/git grep exec flags enabling silent RCESimilar attack · GitHub Advisory Database
- HighGHSA-8hp3-xxrg-v9c8: CodeWhale: Automation auto_approve defaults to true, bypassing the task-path hardening fix for silent recurring agent executionSimilar attack · GitHub Advisory Database
- MediumCVE-2026-18883: ServiceNow Email Response AI Agent indirect prompt injection flawSimilar attack · NVD/CVE Database
- InfoAI-Defined Vehicles: Securing the Decisions That Drive Vehicle BehaviorSimilar attack · VicOne Blog
- HighCVE-2026-75612: yt-mcp arbitrary file read through add_attachment file_pathSimilar attack · NVD/CVE Database