Attention-based Classification for Indirect Prompt Injection

Summary

This paper evaluates Attention Tracker, a white-box detector for indirect prompt injection that monitors how a model's attention splits between user task tokens and data tokens, on the AgentDojo agentic benchmark. The authors find the dataset-agnostic claim fails there, with long structured tool outputs producing near-chance detection across most model families. They propose AttnDA, which replaces the scalar focus-score threshold with a Fisher Linear Discriminant Analysis classifier over the per-head focus score vector, and report better detection across four benchmarks, though transfer to AgentDojo remains hard.

Topics

• Prompt injection and jailbreaks

Related items

• InfoIs Claude Code safe for the enterprise? What the official settings cover, and what they don'tSimilar attack · Adversa AI Blog
• Info‘Context Bombs’ Cut AI Attack Success by Misleading Agents With Fake TargetsSimilar attack · SecurityWeek
• InfoPrompt Injection Through Forensic Evidence: A Taxonomy and Proof of Concept of Anti-Forensic Attacks on LLM-Assisted Digital InvestigationsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoThe Harness Is the Hard Part of Agentic PentestingSimilar attack · The Hacker News
• InfoWhere AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden VulnerabilitiesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)