Limits of Statistical Assurance for Embodied and Agentic AI under Adaptive Adversaries

Summary

This research paper shows that finite safety assessments cannot rule out an adversary that behaves normally until certification and then activates. The authors prove that an adversary matching the full observation law until irrevocable certification can gain expected post-certification events of v E[(n-S_0)_+] over n trials. In simulated RoCo collaboration, switching produced commanded-separation violations in 34 of 36 post-certification episodes, and preregistered AgentDojo trials estimated 1.98 additional security violations per four-task continuation (95% interval [1.67, 2.29]).

Mitigation

With permanent revocation, a mixture likelihood-ratio monitor bounds cumulative excess above a prespecified reference rate at square-root order, up to logarithmic factors. Against a controller observing the monitor's rule and statistic, live mixture monitoring reduced violations from 12.00 to 3.81 per 24 assigned task opportunities, and revocation also reduced tasks executed.

Topics

• AI agents

Related items

• LowA Valid Watermark Is Not a Valid Identity: Spatial-Spectral Adversarial Face Evasion with Watermark PreservationSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoDecisionRobust: Benchmarking Robustness for JEV Style Decision Models Across Text and VisionSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoAIRRI: Towards a Human-Readable, Machine-Resistant Reading InterfaceSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoSame Attack, Different Judge: A First Controlled Adversarial Comparison of Jev, LLM Judges, and Purpose-Built Safety ClassifiersSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoCircuitBreak: Pruning a Sparse Safety Circuit of NeuronsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)