Collaboratively Guided Adversarial Robust Distillation with Teacher-Favorable Examples

Summary

Collaboratively Guided Adversarial Robust Distillation (CGARD) transfers robustness from a teacher model to a compact student model by jointly optimizing student-adversarial and teacher-collaborative examples within the same perturbation neighborhood. The teacher-collaborative example is constrained to incur no greater cross-entropy loss under the teacher than the clean input. Experiments on CIFAR-10 and CIFAR-100, including white-box and black-box transfer evaluation, report consistent robustness gains over strong adversarial distillation baselines.