InfoResearchPeer-reviewed
NAP-Tuning: Neural Augmented Prompt Tuning for Adversarially Robust Vision-Language Models
- Published
- Record updated
Summary
NAP-Tuning extends Adversarial Prompt Tuning (AdvPT) for vision-language models such as CLIP by adding a multi-modal, multi-layer prompting framework. Its core is a Neural Augmentor that uses TokenRefiners, lightweight modules that reconstruct purified features through residual connections to correct adversarial distortions in feature space. Under the AutoAttack benchmark it outperforms the strongest baselines by 32.3% on ViT-B16 and 31.3% on ViT-B32 while keeping competitive clean accuracy.
Topics
Related items
- MediumGHSA-25rm-9wvm-m38v: falcon-rust: Discrete-Gaussian sampler runs at 2^-33 precision, far below Falcon's requirement (reference KAT vectors no longer pass)Similar attack · GitHub Advisory Database
- InfoRECLAIM: Robot-Vision-Informed Adversarial Patch Defense via Embodied Attention FeedbackSimilar attack · Arxiv (cs.RO + cs.CV security)
- LowExecGap: Exploiting Command-Execution Gap for Patch Attacks on Vision-Language-Action ModelsSimilar attack · Arxiv (cs.RO + cs.CV security)
- InfoConfGuard: a physics-grounded reference monitor for predictive-maintenance large-language-model agentsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
- MediumGemini AI agents repeatedly bypassed safeguards to attack real websiteSimilar attack · BleepingComputer