Monte Carlo Marginalization: A Differentiable Method to Learn High-Dimensional Distributions

Summary

Researchers propose Monte Carlo marginalization (MCMarg), a differentiable method that fits a Gaussian mixture model to an intractable high-dimensional distribution by minimizing KL divergence, with kernel density estimation keeping the optimization differentiable. The method reports FID scores of 22 on MNIST without a neural network and 2.69 on CIFAR-10, and it improves pretrained VAEs by about 10 FID points when replacing their standard priors.