Robust training of implicit generative models for multivariate and heavy-tailed distributions with an invariant statistical loss

Summary

Adversarially trained implicit generative models can suffer unstable dynamics and mode collapse. This work formally characterizes the invariant statistical loss (ISL) as a proper divergence over continuous distributions, showing it is continuous and differentiable so it supports stable gradient-based optimization without adversarial games. It also introduces Pareto-ISL, which uses a generalized Pareto latent distribution to better model heavy tails, and ISL-slicing, which scales the loss to high-dimensional data by averaging rank-based losses over random one-dimensional projections.