Perturbation Probing: A New Diagnostic for the Fragility of LLM Safety
Summary
Researchers developed perturbation probing, a method that identifies which neurons (individual computational units) in an AI model are responsible for safety behaviors like refusing harmful requests. The study found that safety in some models depends on extremely few neurons (as few as 20-50 out of hundreds of thousands), meaning the safety defense is concentrated in a thin layer rather than distributed throughout the model, similar to relying on a single firewall. The research also introduced the FFN/Skip ratio, a diagnostic score that can quickly predict whether a model's safety is vulnerable to being bypassed.
Solution / Mitigation
The source text explicitly recommends a defense-in-depth strategy: implementing external content filters and runtime guardrails layered on top of the base model's training. Additionally, the source suggests using perturbation probing as a pre-deployment diagnostic so security teams can measure how fragile a model's safety is before putting it into production. The text also notes that amplifying identified neurons improved factual self-correction performance on at least one tested model.
Classification
Affected Vendors
Related Issues
Original source: https://unit42.paloaltonetworks.com/perturbation-probing-llm-safety/
First tracked: August 28, 2026 at 08:01 PM
Classified by LLM (prompt v3) · confidence: 92%