Don't Judge an Agent by Its Messages: Private-State Leakage under Passive Monitoring of Agent Communication

Summary

A research paper shows that an AI agent can match the distribution of its messages and the receiver's task outcomes while leaking private information the task never required. Selecting among 16 responses from Qwen2.5-1.5B-Instruct on a synthetic routing task let a public decoder recover a private bit in all 8,192 held-out rounds at 98.60% routing accuracy. The authors argue that passive auditing cannot certify this kind of leakage, and that a canonical task interface bounds the remaining information about a balanced attribute by twice the routing-error rate.

Mitigation

A canonical task interface preserves every receiver decision and bounds the remaining information about a balanced attribute by twice the routing-error rate, even for a receiver that shares a key with the sender.

Related items

• MediumOpenCode Console Breach May Have Exposed OpenAI, Anthropic, and Google API KeysSimilar attack · The Hacker News
• InfoMapSteal: Map-based Autonomous Knowledge Stealing Attack Against Large Language ModelsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoFrom knowledge retrieval to generation: a comprehensive study of RAG's architectures, evolution, and applicationsSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoThe LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM SystemsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoModel-as-a-Trojan: Self-Activating Covert DNS Exfiltration Baked Into the Weights of an LLM AgentSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)