Skip to content
InfoResearchPreprintLLM-specific

Where Can We Fool the Model's Mind? Response-Guided Semantic Probing and Adaptive Routing for Transferable Attacks on Black-Box MLLMs

Published
Record updated
View JSON

Summary

VISTA is a black-box attack framework that finds semantically vulnerable image regions in multimodal LLMs using only their text responses, without gradients, logits, parameters or internal representations. It builds an Uncertainty-Based Multimodal Behavior (UBMB) map from response drift, conflict, misleading and semantic shifts, then routes perturbations to those regions. Across GPT-5, Gemini-3.5-Flash and Claude Sonnet 4.6 it reports up to 97 % attack success rate at PSNR above 30 dB, and VISTA-Proxy reaches up to 93 % without querying the final victim model.