AI Sec Watch: A Security Intelligence Platform for AI Systems

Luu, T.J.

LLMs and Contextual Integrity

infonewsLLM-Specific

researchprivacy

Source: Schneier on SecurityAugust 18, 2026

Summary

Researchers have identified a significant problem with how large language models (LLMs) handle sensitive information when they remember details from past conversations. When LLMs use persistent memory (stored information from previous interactions), they often leak private details in situations where those details shouldn't be shared, even in frontier models (the most advanced current systems) which showed up to 69% attribute-level violations (inappropriate leaking of specific user details). One research team found that simply asking an LLM to be more careful about privacy doesn't work well because models tend to either share everything or nothing instead of making thoughtful, context-specific decisions.

Solution / Mitigation

One paper explicitly describes a solution: develop a reinforcement learning (RL, a machine learning technique where a system learns by receiving rewards for good behavior) framework that teaches models to reason explicitly about contextual integrity when deciding what information to disclose. The authors demonstrate that this approach "substantially reduces inappropriate information disclosure while maintaining task performance" using a synthetic dataset of only 700 examples with diverse contexts. The improvements from this method transfer to established privacy benchmarks with human annotations, showing the approach works across multiple model sizes and families.