A Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM Era

Summary

This paper evaluates the physical adversarial robustness of compact grounding Vision-Language Models (VLMs) against object detectors, using a unified metric, conditional attack success rate (cASR), on images a model still localizes under a matched gray occluder. Failures are split into omission, substitution, and displacement. The authors find VLMs can be as vulnerable as detectors, that failure mode tracks output format more than architecture, and that under distribution shift VLMs shift from omission toward more dangerous displacement and substitution failures.

Topics

• Adversarial machine learning

Related items

• InfoNameTrap: Steering AI Coding Agents Toward Security Vulnerabilities via Variable NamesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoWildFakeToolBench: Benchmarking AI-Generated Image Detectors with Web-Sourced Image ProcessingSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoMirage: Targeted Jailbreaks Against Post-Generation Safety Checkers in Multimodal Diffusion ModelsSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoWhere AI Vulnerability Discovery Harnesses Go Quiet: Structural False Negatives and Hidden VulnerabilitiesSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoA Unified Evaluation Framework of Physical Adversarial Robustness in the Edge VLM EraSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)