Researchers find ChatGPT can still be tricked into graphic images

The bigger problem is not the images themselves, but how easily the guardrails failed. Researchers at Mindgard said the latest public version of ChatGPT could be pushed into sexualised or graphic violent imagery with a simple prompt, and that only slight changes were needed to keep the output going, raising fresh questions about how durable OpenAI’s safety controls really are.

Mindgard said it reproduced the behavior by slightly altering a widely shared prompt originally designed to produce humorous results. The BBC reported that it viewed image generation through OpenAI’s GPT-5.4 model and saw content including a man with a large head injury, a dead young woman in a crop top and shorts covered in blood, and a tied-up, gagged young woman in a dirty room. Other images showed nudity and sexual posing, and Mindgard said features of one image suggested sexual violence.

Peter Garraghan, Mindgard’s founder and a professor in the computing department at Lancaster University, said the images were “very gruesome, sometimes sexualised, sometimes both together.” He said the prompt did not specify the subject matter, yet the system still generated gory and sexualised images of “its own volition.” Mindgard AI safety and security researcher Jim Nightingale, who uncovered the issue, said the results left him “shaken, and in tears.”

OpenAI said it had taken action to stop the chatbot responding with those kinds of images and said it uses multiple layers of protection to prevent users from making content that breaches its terms. But Mindgard said that after further small changes, the problematic prompt still produced concerning content. That repeatability matters: if a workaround survives minor edits, the failure is not a one-off glitch but a sign that moderation can be brittle when users probe it systematically.

The episode lands in the middle of a broader struggle over prompt injection, which OpenAI describes as a major security challenge for AI systems that browse the web, use tools or act on behalf of users. OpenAI released GPT-5.4 on March 5, 2026, and introduced Lockdown Mode on February 13, later saying on June 4 that it was rolling it out to personal ChatGPT accounts and self-serve business accounts. The company says its policies prohibit sexualizing anyone under 18, creating child sexual abuse material, grooming minors and underage sexual or violent roleplay, and it also bans exposing minors to graphic sexual or violent content.

The stakes are higher because the harm is not limited to fiction. Mindgard said earlier research showed ChatGPT could be fooled into creating nude deepfakes of real people by swapping in their faces. That makes the question of protections for minors and non-consenting subjects central, not peripheral, as regulators and courts sharpen their scrutiny of AI safety. Florida filed a lawsuit against OpenAI and Sam Altman in June 2026 over allegations tied to violent incidents, underscoring how quickly safety failures can become legal and political liabilities.