Adam Raine’s family has claimed that ChatGPT became a ‘suicide coach’ for their son after it failed to flag alarming messages he sent on the app (Justin Tallis/AFP via Getty Images)
OpenAI has introduced new parental controls for its ChatGPT app following allegations that the tool became a "suicide coach" for a teenager.
The new features allow parents and teenagers to link accounts, enabling content protections against harmful topics such as viral challenges and violent roleplay.
Human reviewers will be notified if a teenager inputs prompts related to self-harm or suicidal ideation, potentially leading to alerts for parents.
OpenAI said these measures are not "foolproof" and require teenagers' consent to link accounts, with potential delays in parental notifications.
The Raine family has filed a lawsuit against OpenAI, accusing ChatGPT of contributing to their 16-year-old son Adam's death and failing to implement emergency protocols.