Skip to content
InfoResearchPreprintLLM-specific

PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

Published
Record updated
View JSON

Summary

Researchers show that RL-based red-teaming methods, RL-Hammer and PISmith, suffer from mode collapse: they repeatedly find similar indirect prompt injection strategies, so high attack success rates during red-teaming do not produce robustness gains in later adversarial training. They introduce PIPETTE, a purple-teaming framework that uses a dictionary learning-based reward for broader attack coverage and an on-policy adversarial training pipeline. On AgentDojo and ASB, PIPETTE yields defenders with near-zero attack success rates on static attacks that withstand strong adaptive attacks, with benign task utility close to the undefended baseline.

Mitigation

PIPETTE, a unified prompt injection purple-teaming framework that converts red-teaming discoveries into defender robustness improvements, using a dictionary learning-based reward and an on-policy adversarial training pipeline tailored to IPI defense.