Allies Teach Better Than Enemies: Inverse Adversaries for Robust Knowledge Distillation

This research proposes a new method for knowledge distillation (training a smaller AI model to mimic a larger one) that preserves adversarial robustness (the ability to resist attacks designed to fool AI systems). Instead of having the student model copy all predictions from the teacher model, the method uses "inverse adversarial examples" (inputs created by reversing the direction of adversarial attacks) to guide learning toward more reliable predictions, resulting in better robustness transfer between models.