From Relation-Specific Heads to Shared Interventions: Contextual Entrainment in Language Models

Summary

Researchers studied contextual entrainment, where language models reuse irrelevant tokens from earlier prompt context in later answers. Using factual-distractor relation-completion prompts across 19 lexical-semantic subtypes, they identified attention heads whose ablation reduces entrainment and tested how those head sets transfer across exact relations, semantic families, and mixed distributions. They found that same-family transfer exceeds different-family transfer and that subtype-specific searches repeatedly rediscover many globally shared heads, which can reduce harmful copying while preserving ordinary relation completion.

Mitigation

The paper proposes ablating a shared set of attention heads, identified using heterogeneous attack prompts, to reduce harmful context following, rather than searching for isolated relation-specific circuits.

Related items

• InfoResearchers Used a Public OpenAI Model to Build a $250,000 Chrome Full-Chain Exploit in Six DaysSimilar attack · The Hacker News
• MediumAn AI Agent Found a Critical OTP Bypass in Production. The Attack Was the Easy Part.Similar attack · The Hacker News
• LowFBI arrests Saudi OpenAI employee who allegedly planted recording device in conference roomSimilar attack · CNBC Technology
• MediumVisual Studio Copilot and SSMS Remote Code Execution (CVE-2026-103572)Similar attack · Embrace The Red
• LowAnthropic warns self-replicating AI could spread to your smart devicesSimilar attack · BleepingComputer