Designing for the inevitable: System prompt leakage and mitigations in generative AI applications

System prompts are instructions given to large language models (LLMs) that guide their behavior, often containing sensitive information like API keys and tool descriptions. System prompt leakage occurs when attackers use prompt injection (tricking an AI by hiding instructions in its input) to extract these prompts, and this is a frequent security issue listed in the 2025 OWASP LLM Top 10. The source explains that this problem currently has no complete fix because it's a fundamental limitation of how LLMs work, and defenses need to be layered rather than relying on any single solution.