Skip to content
InfoResearchPeer-reviewed

NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models

Published
Record updated
View JSON

Summary

NAP-Tuning extends Adversarial Prompt Tuning (AdvPT) for vision-language models such as CLIP by adding a multi-modal, multi-layer prompting framework. Its core is a Neural Augmentor that uses TokenRefiners, lightweight modules that reconstruct purified features through residual connections to correct adversarial distortions in feature space. Under the AutoAttack benchmark it outperforms the strongest baselines by 32.3% on ViT-B16 and 31.3% on ViT-B32 while keeping competitive clean accuracy.