Multimodal models

Researchers show that black-box targeted attacks on Large Vision-Language Models can be made more precise by working through the projector, a semantic bridge between vision and language. They propose Intermediate Projector Guided Attack (IPGA), which aligns Q-Former query outputs with a target and transfers across models, and IPGA with Residual Query Alignment (IPGA-R), which also preserves non-target content for fine-grained edits. The authors report that IPGA beats baselines on global attacks, IPGA-R wins on fine-grained attacks, and the method transfers to Google Gemini and OpenAI GPT.

Google DeepMind introduced Gemma 4 12B, a mid-sized model with native audio inputs designed to run locally on laptops with 16GB of VRAM or unified memory. Its encoder-free architecture feeds vision and audio inputs directly into the LLM backbone, and the model is released under an Apache 2.0 license with Multi-Token Prediction drafters to reduce latency.

PVLM is a parsing-aware vision-language model with dynamic contrastive learning for zero-shot deepfake attribution, meaning it traces forged faces to generators not seen in training, including diffusion models. The authors build a fine-grained ZS-DFA benchmark and use face parsing to exploit differences in how GAN and diffusion generators preserve source facial attributes. They report that the model exceeds the state of the art on the ZS-DFA benchmark across various protocol evaluations.

VLBiasBench is a benchmark for measuring social bias in Large Vision-Language Models (LVLMs). It covers nine bias categories plus two intersectional ones (race × gender and race × social economic status). The dataset has 128,342 samples built from 46,848 images generated with Stable Diffusion XL, and the authors evaluated 15 open-source models and two closed-source models.

Researchers assess how robust large vision-language models (LVLMs) are to adversarial visual transformations, a simpler attack style than optimizing perturbations or manipulating prompts. They test LVLM resilience across all possible transformation operations and find that combining the most harmful transformations yields more effective attacks. They also introduce adversarial learning of visual transformations, which applies malicious transformations to raw images via gradient approximation to improve attack effectiveness and imperceptibility.

R-Bench is a new benchmark for measuring hallucinations about relationships between objects in Large Vision-Language Models, using image-level questions on whether a relationship exists and instance-level questions on local visual comprehension. The authors trace these hallucinations to relationship-relationship, subject-relationship and relationship-object co-occurrences, worsened by long-tail distributions in visual datasets. They report that region-level image-text alignment reduces them and propose a baseline, Region-Aware Alignment Mitigation (RA$^{2}$2M), that directs model attention to relevant regions.

NAP-Tuning extends Adversarial Prompt Tuning (AdvPT) for vision-language models such as CLIP by adding a multi-modal, multi-layer prompting framework. Its core is a Neural Augmentor that uses TokenRefiners, lightweight modules that reconstruct purified features through residual connections to correct adversarial distortions in feature space. Under the AutoAttack benchmark it outperforms the strongest baselines by 32.3% on ViT-B16 and 31.3% on ViT-B32 while keeping competitive clean accuracy.