NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models

Vision-Language Models (VLMs, AI systems that understand both images and text together) like CLIP are powerful but vulnerable to adversarial attacks (malicious inputs designed to fool AI systems, especially in images). This research presents NAP-Tuning, a method that uses learnable text prompts and lightweight neural modules called TokenRefiners to clean up distorted features inside the model's layers, making these systems more resistant to such attacks while keeping normal performance intact.