Extracting System Prompt of LLM Services in Complex (Agentic) Settings: Hierarchical Optimization and Reliable Evaluation Metrics

Summary

Existing prompt extraction attacks degrade on recent LLM services, whose system prompts are longer and more structured and whose raw outputs are often hidden behind post-processing. The authors propose ExtraHerO, which decomposes extraction into a hierarchical reconstruction guided by a structural prompt template and iteratively updates adversarial queries, with early stopping based on information gain. Across chatbots and agents, it reports average improvements of 12.5% (fidelity), 2.3% (functionality), and 6.0% (adversarial utility) on chatbots, and 12.2%, 34.3%, and 30.5% on agents.

Topics

• AI agents

Related items

• MediumGoogle: LLMjacking attacks rack up $1 million in cloud AI chargesSimilar attack · BleepingComputer
• InfoMCP Tool Poisoning in Agentic AI Frameworks: A Cross-Framework Measurement StudySimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• MediumHow We Took Over ChatGPT AccountsSimilar attack · Simon Willison's Weblog
• Medium“Black October” for AI Developer PlatformsSimilar attack · Wiz Research Blog
• InfoHow Much Can Tool Calls Reveal About You? Privacy Leakage from Tool-Call TracesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)