InfoResearchPeer-reviewedLLM-specific
Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations
- Published
- Record updated
Summary
This paper shows that a few harmful in-context demonstrations can override the safety alignment of LLMs, a technique it calls the In-Context Attack (ICA). It also proposes the In-Context Defense (ICD), which uses examples of refusing harmful requests to strengthen resilience. The authors report theoretical analysis and empirical validation across multiple models, datasets and attack baselines.
Mitigation
In-Context Defense (ICD): bolstering model resilience with in-context examples that demonstrate refusal to produce harmful responses.
Topics
Related items
- InfoThreatsDay: GPT-6.2 Jailbreak Flaws, BubbleLoader PNG Malware, $500M Ransom Demands + 17 More StoriesSimilar attack · The Hacker News
- Info30 Years of Gatekeeper Asymmetry Doom: The Paradox that Makes Powerful LLMs More VulnerableSimilar attack · Check Point Research
- InfoJailbreak Detection on Large Language Models via Contrastive Prototype StructuresSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- InfoIDEA: Intent-Dominance Evidence Audit for Inference-Time Backdoor Detection in GUI AgentsSimilar attack · Arxiv (cs.RO + cs.CV security)
- InfoDressed in Authority: Practical Black-Box Jailbreaking of MLLMs via Visual Trust BiasSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)