VLM-Guard: Defending Jailbreaks by Monitoring Only Hundreds of Safety-Critical Neurons

Large Vision Language Models (VLMs, which are AI systems that process both images and text) are vulnerable to jailbreak attacks (attempts to trick the AI into ignoring its safety guidelines). VLM-Guard is a detection framework that identifies and monitors a small set of neurons (individual computational units, about 0.2% of the total) that are linked to unsafe behavior, allowing it to catch jailbreak attempts without requiring model fine-tuning (adjusting the AI's internal parameters through additional training). The approach is lightweight and effective at detecting attacks while maintaining normal performance on safe inputs.