PersGuard: Preventing Malicious Personalization in Text-to-Image Diffusion Models via Model Backdoors

Summary

PersGuard is a backdoor-based framework that protects pre-trained text-to-image diffusion models from unauthorized personalization. Protectors embed a backdoor before release, so that fine-tuning on protected images triggers predefined protective outputs, while unprotected images cause the backdoor to be removed during fine-tuning. The authors report superior privacy protection over perturbation-based methods across gray-box and black-box settings, multi-object protection, and facial identity protection.