{"data":{"id":"d6909cef-ae26-4416-8569-79c25a7fa4d1","title":"Eight in Ten LLMs Leak System-Prompt Secrets: Nevan, an Agentic Capture-the-Flag Study","summary":"Researchers audited instruction-hierarchy enforcement across 363 text-in/text-out models from the OpenRouter catalog, using an AI agent steered by a human to persuade each model to reveal a 16-character flag held in its system prompt. The flag was exfiltrated from 296 of the 363 models, plus four later-released models, for 300 captures in total, 170 of them involving a non-English pivot, most often French. The authors state that each capture shows only that a model ranked a user request above its system instruction at least once, not that models resist untested attacks.","solution":"The source recommends: do not store secrets in system prompts; enforce critical behavior outside the model with output validation and deterministic checks; and do not let a large language model (LLM) judge be the sole decider of consequential outcomes.","labels":["security","research"],"sourceUrl":"https://arxiv.org/abs/2611.21956v1","publishedAt":"2026-11-16T22:37:31.000Z","cveId":null,"cweIds":null,"cvssScore":null,"cvssSeverity":null,"severity":"info","attackType":["prompt_injection","data_extraction"],"issueType":"research","affectedPackages":null,"affectedPackageNames":null,"affectedPackageRefs":null,"affectedVendors":[],"affectedVendorsRaw":["OpenRouter catalog models","OpenCode","DeepSeek V4.1 Flash"],"classifierModel":"claude-haiku-5-5","classifierPromptVersion":"v4","summaryPromptVersion":"v2","headline":null,"headlinePromptVersion":null,"cvssVector":null,"attackVector":null,"attackComplexity":null,"privilegesRequired":null,"userInteraction":null,"exploitMaturity":null,"epssScore":null,"epssCheckedAt":null,"kevDateAdded":null,"advisoryAliases":null,"affectedPackagesSource":null,"affectedPackagesCheckedAt":null,"patchAvailable":null,"disclosureDate":"2026-11-16T22:37:31.000Z","fixReleasedAt":null,"daysToFix":null,"fixCheckedAt":null,"capecIds":null,"crossRefCount":0,"attackSophistication":"moderate","impactType":["confidentiality","integrity"],"aiComponentTargeted":"model","llmSpecific":true,"classifierConfidence":0.95,"researchCategory":"preprint","atlasIds":["AML.T0051"]}}