The LLM-Attacker: A Unified Red-Team Framework for Privacy Evaluation of Split and Distributed LLM Systems

Summary

LLM-Attacker is a red-team framework for evaluating privacy in split and distributed LLM systems, where input text stays on the client and intermediate representations go to a server. It records attacker access, datasets and scoring rules so that evaluations can be checked and compared, and it requires a linked undefended reference before a defence can pass. In a controlled GPT-2 experiment, returning gradients during training raised token recovery from 94.20% to 97.38% and exact-document recovery from 13.71% to 37.77%.

Related items

• InfoFrom knowledge retrieval to generation: a comprehensive study of RAG's architectures, evolution, and applicationsSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoModel-as-a-Trojan: Self-Activating Covert DNS Exfiltration Baked Into the Weights of an LLM AgentSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoPrivate Retrieval Augmented Generation via Random ProjectionSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• LowEncoded but Exposed: Language Embeddings can Reveal More Than They Should in Recommender SystemsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumCVE-2026-110248 - Insertion of sensitive information into log files in Amazon SageMaker Python SDKSimilar attack · AWS Security Bulletins