On Task, Off Track: Benchmarking LLM and Agent Robustness to Near-Target Prompt Injection

Summary

NTBench, a new benchmark, tests whether LLMs and agents resist near-target prompt injection, where injected instructions produce answers that look legitimate but are incorrect or harmful. It builds attack targets from legitimate reference outcomes across ten language datasets and two agent benchmarks, evaluating seven LLMs. Under three attack executions per sample, both incorrect-target and harmful-target tracks raised attack success over the fixed-target baseline for every model, by 9.3 to 35.8 points and 1.7 to 17.3 points on language tasks respectively.

Topics

• Prompt injection and jailbreaks

Related items

• MediumMalicious downloads can trick AI coding agents into compromising themselvesSimilar attack · CSO Online
• LowYou gave your AI agent access, but who is giving it orders?Similar attack · BleepingComputer
• LowThe AI notetaker risk is now an everywhere problemSimilar attack · CSO Online
• InfoAgentic AI Defense: Decode-Time Provenance Binding for MCPSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoAgentPhishing: A Benchmark of LLM Email Agents Under Adaptive AttackSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)