Do Benign-Trained Hidden-State Probes Detect Tool Errors Under Adversarial Conditions?

Summary

This research asks whether hidden-state probes trained only on benign tool use still detect incorrect tool actions under adversarial conditions. The authors test three models (Qwen2.5-7B-Instruct, Qwen2.5-14B-Instruct, GLM-4.6V-Flash) on four AgentDojo suites across benign, prompt-injection, system-override and tool-poisoning conditions, comparing the probes with three scalar uncertainty metrics. Under the system-override condition with Qwen2.5-14B-Instruct, all three uncertainty metrics beat the probe on AUROC and average precision in six significant paired tests after Holm correction, so the authors conclude benign-trained probes should not be assumed reliable for this purpose.

Related items

• LowHelp defend against memory poisoning in Amazon Bedrock AgentCore with multi-layer write validationSimilar attack · AWS Security Blog
• InfoCan the Confidence and Decisions of System 1 Models Be Trusted? An Explainability and Perturbation Audit of Jev-Style Decision ModelsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoHarness Matters: Measuring and Dissecting Security Resilience in Web AgentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumTop MCP security resources — November 2026Similar attack · Adversa AI Blog
• InfoScaling Adaptive Prompt Injection Attacks with Agent SwarmsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)