PromptGuard: Soft Prompt-Guided Unsafe Content Moderation for Text-to-Image Models

Summary

PromptGuard is a content moderation technique for text-to-image models that optimizes a universal safety soft prompt in the model's textual embedding space to act as an implicit system prompt. A divide-and-conquer strategy builds category-specific soft prompts into holistic safety guidance. Across five datasets it reduces NSFW generation, reaches an average unsafe ratio of 5.84% and 6.18% under multi-head classifiers and VLM-based guardrails, and runs 3.8 times faster than prior methods.

Related items

• MediumInside the loop: Five AI attack frameworks that never ask permissionSimilar attack · Elastic Security Labs
• MediumARTEX: Attackers Adopted This AI Pentesting Tool in Three Months. Leaked Secrets Are Its Fastest Way In.Similar attack · StepSecurity Blog
• MediumFBI Says North Korean Hackers Stole Over $400 Million in Crypto Using Deepfake Video CallsSimilar attack · The Hacker News
• MediumCVE-2026-126395: vLLM information disclosure via uppercase PIL image addresses in error messagesSimilar attack · NVD/CVE Database
• MediumCVE-2026-127202: MLflow path traversal through model_id in the log outputs APISimilar attack · NVD/CVE Database