Asynchronous Federated Learning With Nonconvex Client Objective Functions and Heterogeneous Dataset

Summary

This article extends asynchronous federated learning (AFL) to clients with nonconvex objective functions and heterogeneous datasets. The authors provide a convergence analysis that bounds the expected gradient norm and examines the effects of staleness, variance and heterogeneity. They introduce a staleness-aware aggregation mechanism and a dynamic learning rate schedule, and implement the methods in PyTorch with Python's asyncio library.