Disclosure Begets Disclosure: A Self-Reinforcing Privacy Loop in LLM Agents with Persistent Memory

Summary

Researchers simulated multi-session dialogues in which a user with 24 personal attributes talked to LLM support agents that either consulted or ignored persistent memory. Consulting memory raised response quality but also increased empirical leakage, the share of attribute values correctly inferred from the accumulated record, and the gap widened over sessions. The authors attribute this to a self-reinforcing loop in which memory-conditioned responses carry earlier values forward as presuppositions that users affirm or leave uncorrected.

Mitigation

Instructing the agent not to state or presuppose attributes brings leakage to about the no-memory level while keeping most of the quality gains. The source reports that filtering or anonymizing what users say or what is stored gives up much of the quality gains.

Related items

• LowMeta pauses Muse sharing after researchers find AI chats on the webSimilar attack · The Verge (AI)
• InfoRAGLeak: Supply-Chain Privacy Backdoors in Retrieval-Augmented GenerationSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoThe LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM SystemsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• LowEncoded but Exposed: Language Embeddings can Reveal More Than They Should in Recommender SystemsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumCVE-2026-110248 - Insertion of sensitive information into log files in Amazon SageMaker Python SDKSimilar attack · AWS Security Bulletins