Momentum-Based Zeroth-Order Gradient Method for Distributed Black-Box Optimization

This paper presents DZO-SNGM, an algorithm for distributed black-box optimization, where multiple computers work together to solve problems where they can only test solutions rather than calculate exact gradients (zeroth-order methods). The algorithm addresses two main challenges: handling data that differs across computers and reducing the noise in gradient estimates by using momentum (a technique that builds on previous updates to smooth progress). Experiments show this method converges faster and performs better than existing approaches.