Can Quantum Layers offer Performance and Security Advantages in Vision-Language Models?

Summary

This study tests classically simulated quantum layers in the LLaVA-OneVision vision-language model, using 4- and 8-qubit circuits inserted at or replacing classical components. On the Visual Spatial Reasoning dataset, a quantum layer inserted after the visual projector reached 80.7% development accuracy, 2.28 percentage points above classical full fine-tuning. Replacing the final decoder feed-forward sublayer kept 79.22% accuracy with 13.07 million fewer trainable parameters, though benefits depend strongly on placement.

Topics

• Multimodal models