AGFPS: An Automated Gradient-Free Framework for Prompt Stealing

Summary

Researchers present AGFPS, an automated gradient-free framework that uses evolutionary optimization to steal system prompts from LLM applications. The method treats prompt stealing as a discrete optimization problem, with adversarial queries evolved through elite retention, selection, adaptive crossover and mutation. It reports a 95.2% exact system prompt stealing success rate and outperforms gradient-based methods in 80.6% of scenarios, with queries that transfer across models and remain robust against various defenses.

Related items

• MediumCVE-2026-125386: langchain4j-aideepin insecure direct object reference in message endpointsSimilar attack · NVD/CVE Database
• InfoSNAPRAG: Cost-Efficient Knowledge Extraction Attack on RAG Systems via Adversarial PerturbationsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• LowMGE: A Membership Guided Data Extraction Attack against Diffusion ModelsSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoSecuring Locally Deployed LLM-RAG Assistants: A Cross-Model and Cross-Lingual Evaluation of Indirect Prompt Injection DefensesSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoGem5-Shark: Smaug-based Heterogeneous Deep Learning Accelerator Security and Robustness Evaluation and Characterization FrameworkSimilar attack · OpenAlex (peer-reviewed AI security)