Skip to content
InfoResearchPreprintLLM-specific

Centered Ablation: Why Last-Layer Directional Ablation Under-Fires or Collapses, and a Closed-Form Fix

Published
Record updated
View JSON

Summary

Directional ablation, a method for switching off behaviours such as refusal during model inference, is erratic: it under-fires on some models and collapses text into repetition on others. The authors attribute this to a shared common mode in last-layer hidden states, which makes the direction fire on benign text and under-fire on targets. Their Centered Ablation (CA) projects the common mode out of the direction in closed form, raising attack success by 29 to 46 points on under-firing models across 8 tested models and cutting Gemma-2-9B collapse from 50% to 1%.

Mitigation

CA projects the common mode out of the behaviour direction in closed form; the source names this as the fix but gives no version, patch or configuration change.