DPOison-VL: Clean Preference Poisoning in Vision-Language Direct Preference Optimization

Summary

DPOison-VL is a clean preference poisoning attack against vision-language models trained with Direct Preference Optimization (VLM-DPO). It applies imperceptible perturbations to a small share of training images, leaving prompts, responses and annotations intact, and reaches up to 98% ASR with 2% poisoned data. The authors report cross-model transferability while clean accuracy and general utility are largely preserved.

Related items

• InfoFedGraph: Defending Federated Large Language Model Fine-Tuning Against Backdoor Attacks via Graph-Based AggregationSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• InfoRFLDefense: robust federated learning defense against targeted attacksSimilar attack · OpenAlex (peer-reviewed AI security)
• InfoSpace Over Statistic, Retraining Over Ranking: Auditing Training Labels Under Label-Only BackdoorsSimilar attack · Arxiv (cs.RO + cs.CV security)
• InfoCancerous Backdoor Attack: Gradual Activation via Invisible Tokens and A Stage ControllerSimilar attack · Arxiv (cs.CR + cs.CL + cs.LG)
• LowGNN-Guided Selection of Benign-like Anomalies for Backdoor Attacks Against Network Intrusion Detection SystemsSimilar attack · OpenAlex (peer-reviewed AI security)