Treating the Model as an Untrusted Component: Deterministic Policy Enforcement, Human Approval and Taint Tracking Against Prompt Injection in Tool-Using Agents, with Validation on Open-Weights Models

Summary

This research compares in-prompt defences (hardened prompts, spotlighting, LLM detectors) against deterministic enforcement outside the model, human approval and taint tracking for prompt injection in tool-using agents. In evaluation with Qwen2.5-1.5B and Qwen2.5-7B across 156 attack episodes per defence and model, the models proposed the attacker's call in 64% and 84% of episodes behind the task-bound monitor, which admitted 1.3% and 7.7%.

Topics

• Prompt injection and jailbreaks

Related items

• LowHelp defend against memory poisoning in Amazon Bedrock AgentCore with multi-layer write validationSimilar attack · AWS Security Blog
• InfoCan the Confidence and Decisions of System 1 Models Be Trusted? An Explainability and Perturbation Audit of Jev-Style Decision ModelsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoHarness Matters: Measuring and Dissecting Security Resilience in Web AgentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumTop MCP security resources — November 2026Similar attack · Adversa AI Blog
• InfoScaling Adaptive Prompt Injection Attacks with Agent SwarmsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)