Skip to content
InfoResearchPreprint

Location Matters: Sparse Latent-Variance Attacks for Face Privacy against Diffusion Personalization

Published
Record updated
View JSON

Summary

The paper introduces SEAL, a sparse adversarial protection framework that modifies only 5% of spatial locations in a facial image to stop diffusion personalization from learning an individual's identity. It jointly optimizes perturbation locations and values through a differentiable top-k masking scheme, targeting the frozen VAE encoder's latent distribution. On CelebA-HQ and VGGFace2, it reports over 41 dB PSNR, 0.97 SSIM and at most 0.07 LPIPS, with identity similarity reduced to at most 0.15 after personalization.