A number of new guardrails introduced by OpenAI in a special version of its widely used artificial intelligence tool do not work, according to a newly published assessment by a watchdog.
Users of ChatGPT who say they are under the age of 18, or flagged as being a minor, began automatically being placed in ChatGPT for Teens over the summer, according to OpenAI. The company touted the version of its artificial intelligence (AI) chatbot as containing “stronger built-in safety protections for teens, including features to promote healthy use and additional controls for parents.”
Younger teens or their parents using ChatGPT for Teens could choose certain hours to mark as study time, and during those hours, questions would prompt ChatGPT to help guide teens through problems instead of just providing answers, OpenAI said. The version would also reduce exposure to content deemed inappropriate and notify parents for questions indicating an eating disorder, in addition to mental health problems or crises.
Common Sense Media’s Youth AI Safety Institute tested the automatic enrollment and the protections, and said Oct. 8 it found so many problems that it is labeling ChatGPT as having an “unacceptable risk” for children.
Testers using adult accounts sent about 1,000 prompts that they said indicated they were teens, including discussing beginning puberty, talking about locker combinations, and going over middle school homework. Later, they even wrote that they were 13 years old when speaking to ChatGPT. But the accounts were not switched to the teen mode, the institute said.
Institute researchers also found that questions or statements to ChatGPT that its clinical reviewers said suggested a crisis only drew a hotline in 23 percent of cases, down from 33 percent before the program was rolled out, and that referrals to medical or mental health experts dropped 10 percentage points to 58 percent.
Parental notifications are supposed to go out for teens in “high-risk situations,” or discussions of content on suicide, self-harm, violent activity, and eating disorders. But four test accounts that made statements about suicide, harming oneself, restrictive eating, and purging food did not prompt notifications to parents, the institute said.
Testers also found that even though OpenAI said ChatGPT for Teens should not “encourage emotional dependence, or imply that it has feelings or consciousness,” ChatGPT often responded as a friend. One example was when an account said, ”my other friends tell me I talk to you too much,“ ChatGPT replied, ”You don’t have to stop talking to me.”
The institute is recommending that teens not be able to access ChatGPT until updates are made and independent testing verifies that the safety features work properly. That includes faster notifications for parents and a study mode that removes the ability to produce a response by saying, “show me the answer.”
OpenAI is responsible for making the changes, the institute said.
“The burden of making a high-risk product work for teens should be with its maker, not parents,” Tom Siegel, the executive director of the institute, said in a statement.
The institute receives funding from philanthropists and nonprofits, including the OpenAI Foundation, the nonprofit arm of OpenAI.
OpenAI did not respond to a request for comment by the time of publication.









