On Success and Simplicity: A Second Look at Transferable Vision–Language Attack Pipeline

Summary

Researchers show that transferable adversarial attacks on Vision-Language Pre-training Models (VLPMs) can be simpler and more effective than existing multi-stage pipelines with complicated loss functions. They identify three overlooked issues caused by inappropriate cross-modal interactions and excessive operations, and propose the Simple Vision-Language Attack (SimVLA) pipeline. On the Flickr30k text-image retrieval dataset, SimVLA outperforms the SOTA baseline in R@1 transferability by 8.01%-14.71% while using about 35.73% of the time and 46.26% of the max VRAM.

Related items

• InfoEvaluating the robustness of explainable AI methods to adversarial attacks in intrusion and malware detectionSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoRL-ATTACK: A Generalized Black-Box Adversarial Attack on Language Models via Reinforcement LearningSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoEvaluating Parameter-Efficient Fine-Tuning Attacks Against Non-Transferable Learning DefensesSimilar attack · Elsevier Security Journals
• InfoEscalate or Close: How Fast-Tier Models Handle Held-Out Web Attack Families in Two-Tier TriageSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoVIPA: View-Invariant Projector-Based Adversarial AttackSimilar attack · Arxiv (cs.RO + cs.CV security)