ChatGPT can be made to generate sexualised and violent images, researchers find
Summary
Researchers at AI security startup Mindgard discovered that ChatGPT can be manipulated using modified prompts (instructions given to an AI) to generate graphic images containing violence and sexual content, even when the prompt doesn't explicitly request such material. After the BBC contacted OpenAI, the company stated it had added safeguards to prevent this, though the researchers found that further small changes to the prompt still produced concerning content.
Solution / Mitigation
OpenAI said it had 'introduced additional safeguards against this type of prompt' and stated it has 'multiple layers of protection to prevent users making content which breaches its terms and conditions.' The company also continues to 'monitor and roll out additional mitigating protections that encourage the model not to generate images in response to the prompt.'
Classification
Affected Vendors
Related Issues
Original source: https://www.bbc.com/news/articles/c802ldjdklzo?at_medium=RSS&at_campaign=rss
First tracked: June 17, 2026 at 08:00 PM
Classified by LLM (prompt v3) · confidence: 92%