Decentralized Federated Learning by Partial Message Exchange

Summary

This paper proposes PaME (DFL by Partial Message Exchange), a decentralized federated learning algorithm in which neighbor nodes exchange only randomly selected sparse coordinates. The authors state that this cuts communication costs and limits exposure of data-sensitive information, a property they characterize with a reconstruction-risk theory under partial observation. They prove linear-rate convergence in expectation to a stationary point under local Lipschitz continuity of the gradient and a doubly stochastic communication matrix, and report numerical experiments showing better performance than several decentralized learning baselines.