Tuesday, Nov 17, 2026
|
Advertisement

ChatGPT Users Could Face 24-Hour Suspension for Prompts Sexualising Children, OpenAI Announces New Safety Measures

OpenAI announced stricter child safety measures for ChatGPT, warning users who try to generate sexualised content about children (CSAM), access CSAM or seek out sexually exploitative content that minors will have their accounts temporarily suspended.

ChatGPT
ChatGPT | Photo Credit: https://x.com/Pirat_Nation

When prompted, ChatGPT will warn when prompts are made to search for CSAM or other sexually exploitative material and block users for 24 hours from the platform. The measures were announced on November 16 along with several other initiatives aimed at identifying abusive content and supporting organisations working to combat child sexual abuse.

Advertisement

OpenAI said the new warning system is meant to stop potentially harmful behaviour, not just reject individual requests. If ChatGPT detects a prompt for material that sexualises children, it will halt the conversation and show a warning telling you why it’s harmful (and against platform rules).

The flagged conversation will be removed from the user’s visible chat history, access to ChatGPT may be suspended for 24 hours (users won’t be able to access their accounts during that time), and repeated violations could lead to longer restrictions and ultimately permanent account closure. The company said sexualising children is harmful whether the requested material is real photos, illustrations or AI-generated content.

The latest news also shows how OpenAI is using AI technology to identify child sexual abuse material more effectively. The company introduced a new image classifier that can detect CSAM (including AI-generated images) as well as other types of images (like those created by humans). It looks for visual signs of abuse in previously unseen images and scores them so reviewers can identify potentially harmful content. 

The classifier was developed with the National Centre for Missing and Exploited Children (NCMEC), and OpenAI is working with organisations like Tech Coalition and ROOST to make it available to others. In early testing with partners, the classifier flagged 48 per cent fewer images than abusive images but identified the same number of abusive images. That could reduce the workload on content moderation teams while helping them prioritise serious cases.

Advertisement

OpenAI is also expanding its child protection efforts beyond ChatGPT users. It’s publishing additional guidelines for developers who use its artificial intelligence tools to build applications and services they’re meant to help prevent the creation and distribution of child sexual abuse material, identify potentially abusive content and set up reporting mechanisms. 

OpenAI said in early testing that guidance had helped several developers adopt child safety tools and register to report suspected CSAM to the authorities. “We believe stronger safeguards across the entire AI development ecosystem are needed because harmful content can spread through many platforms and applications (not just mainstream chatbots),” it wrote.

Another part of the initiative is to help law enforcement agencies investigate child sexual exploitation. OpenAI is making AI-powered capabilities available to investigators specialised tools for analysing complex information and supporting investigations; resources that will help authorities process information more quickly and identify potential patterns in online cases of child abuse.

The company is training investigators and giving practical guidance on how to responsibly use artificial intelligence while working with children. These steps show how AI technology is becoming more important in preventing online abuse as well as helping authorities respond to serious crimes involving children.

The company said the new safeguards were developed with child safety organisations, researchers and academics (Protect Children; Lucy Faithfull Foundation) involved so warning messages discourage harmful behaviour and point people in the right direction for support. OpenAI has also pointed to specialist resources for people who are worried about their behaviour and want help stopping it not just technology alone will block content (it is guidance and support too).

The new measures are being rolled out worldwide and will affect personal ChatGPT accounts using Chat and Work modes on web and mobile platforms. OpenAI has said that the specific CSAM warning and suspension interventions aren’t available in ChatGPT Business, Enterprise, Edu or Healthcare workspaces yet; voice mode; Codex or the desktop applications either how it actually works may depend on where the user is based. (In general, if you get a temporary suspension email then it was issued incorrectly.) If you think yours was issued incorrectly then you can appeal (link in notification email) but an appeal does not restore access.

With AI-generated images, videos and conversations becoming more common the concern about AI being used to exploit children has grown. OpenAI’s announcement today shows that they are taking a tougher stance against people who try to use ChatGPT for these activities automated detection, temporary account bans, developer guidelines and law enforcement support all work together to strengthen response to online child sexual abuse. These measures also demonstrate something important for the AI industry: innovation cannot come at the expense of children’s safety and protection.

Comments