InfoResearchPeer-reviewed
Allies Teach Better Than Enemies: Inverse Adversaries for Robust Knowledge Distillation
- Published
- Record updated
Summary
This paper proposes an adversarially robust knowledge distillation method that compresses a large robust teacher model into a lightweight student. It refines inputs into inverse adversarial examples by reversing the sign of the adversarial perturbation, then applies gradient matching between teacher and student on those inputs, plus a weight-space disruption strategy. On ImageNet, the authors report roughly 3.8% gains over prior methods in both clean and robust accuracy.