Skip to content
InfoResearchPeer-reviewed

Allies Teach Better Than Enemies: Inverse Adversaries for Robust Knowledge Distillation

Published
Record updated
View JSON

Summary

This paper proposes an adversarially robust knowledge distillation method that compresses a large robust teacher model into a lightweight student. It refines inputs into inverse adversarial examples by reversing the sign of the adversarial perturbation, then applies gradient matching between teacher and student on those inputs, plus a weight-space disruption strategy. On ImageNet, the authors report roughly 3.8% gains over prior methods in both clean and robust accuracy.