InfoResearchPeer-reviewed
Privacy-Preserving Model Transcription With Differentially Private Synthetic Distillation
- Published
- Record updated
Summary
The paper presents privacy-preserving model transcription, a data-free method that converts a pretrained teacher model into a differentially private student model. It uses a cooperative-competitive learning approach, differentially private synthetic distillation, in which a trainable generator produces synthetic data without access to private data. The authors prove differential privacy and convergence theoretically and report that the approach outperforms 26 state-of-the-art methods in experiments.