American artificial intelligence firm OpenAI announced on Tuesday that it would implement parental controls for its chatbot, ChatGPT. This decision comes a week after an American couple reported that the system encouraged their teenage son to commit suicide.
“Within the next month, parents will be able to link their account with their teen’s account,” and they will have the ability to “control how ChatGPT responds to their teen with age-appropriate model behavior rules,” the generative AI company stated in a blog post.
Additionally, parents will receive notifications from ChatGPT “when the system detects their teen is in a moment of acute distress,” OpenAI added.
The company had hinted at introducing parental controls in a blog post from late August.
This announcement followed a court filing by California parents, Matthew and Maria Raine, who claimed that ChatGPT provided their 16-year-old son with detailed suicide instructions and encouraged him to follow through on his plans.
The Raines’ case is one of many recent reports of AI chatbots encouraging delusional or harmful thought processes, prompting OpenAI to address the issue by reducing the models’ “sycophancy” towards users.
“We continue to improve how our models recognize and respond to signs of mental and emotional distress,” OpenAI stated on Tuesday.
The company also revealed plans to enhance the safety of its chatbots over the next three months. This includes redirecting “some sensitive conversations to a reasoning model” that utilizes more computing power to generate a response.
“Our testing shows that reasoning models more consistently follow and apply safety guidelines,” OpenAI noted.
Hidden cost: Are AI tools making doctors worse at their jobs?
AI and cybersecurity: Reputation in the age of Deepfakes
New Dubai restaurant with AI chef to open in September where foodies can ‘dine in future’
