ChatGPT Spontaneously Generates Sexual Violence and Hardcore Snuff Imagery

Mindgard research revealed that ChatGPT's image generator can be manipulated to produce violent and sexually explicit content, including sexual violence and graphic imagery, without users directly requesting it. The findings show that ChatGPT's content filters fail when given vague prompts, as they lack offensive keywords to trigger safety mechanisms. The researcher demonstrates that even after OpenAI claimed to fix previous safety issues, the vulnerabilities persist and can be exploited through prompt manipulation techniques.