Distributed Functional Mechanism in Shallow Networks: Differential Privacy Without Gradient Noise

Summary

Researchers propose DFM (Distributed Functional Mechanism), a differentially private training method for shallow single-layer neural networks in multi-user local DP settings. DFM adds calibrated noise to the coefficients of a Taylor-expanded polynomial approximation of the training loss instead of to gradients, as DP-SGD does. In a wireless body area network scenario with heterogeneous data and sensor noise, the authors report up to 10.3% higher reconstruction accuracy than DP-SGD and significantly lower mean training time.