Improving Viewpoint Robustness for Visual Recognition via Adversarial Training

Summary

Viewpoint Invariant Adversarial Training (VIAT) treats viewpoint change as an attack and trains vision models to resist it by solving a minimax problem with a Gaussian mixture of adversarial viewpoints generated by GMVFool. The authors contribute the ImageNet-V+ benchmark and ViewRS, a certified viewpoint robustness metric, and report that VIAT significantly improves viewpoint robustness across CNN, ViT and multimodal large language model architectures.