VLBiasBench: A Comprehensive Benchmark for Evaluating Bias in Large Vision-Language Model

Summary

VLBiasBench is a benchmark for measuring social bias in Large Vision-Language Models (LVLMs). It covers nine bias categories plus two intersectional ones (race × gender and race × social economic status). The dataset has 128,342 samples built from 46,848 images generated with Stable Diffusion XL, and the authors evaluated 15 open-source models and two closed-source models.

Topics

• Multimodal models