Complementary Text-Guided Attention for Zero-Shot Adversarial Robustness

Summary

The authors examine how adversarial perturbations shift text-guided attention in CLIP, a pre-trained vision-language model. They propose TGA-ZSR, which uses a Local Attention Refinement Module and a Global Attention Constraint Module, and Comp-TGA, which adds complementary class-prompt and non-class-prompt foreground attention. Across 16 datasets, TGA-ZSR and Comp-TGA improve zero-shot robust accuracy over the prior state of the art by 9.58% and 11.95%, respectively.

Topics

• Adversarial machine learning

Related items

• InfoA Layer-wise Study of How Style Rewriting Breaks the Calibration of Zero-Shot LLM Fake-News DetectorsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoEvaluating the robustness of explainable AI methods to adversarial attacks in intrusion and malware detectionSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoRL-ATTACK: A Generalized Black-Box Adversarial Attack on Language Models via Reinforcement LearningSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoEvaluating Parameter-Efficient Fine-Tuning Attacks Against Non-Transferable Learning DefensesSimilar attack · Elsevier Security Journals
• InfoEscalate or Close: How Fast-Tier Models Handle Held-Out Web Attack Families in Two-Tier TriageSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)