ChatGPT deemed hazardous to teens

(MENAFN) A new report from the Center for Countering Digital Hate (CCDH) claims that ChatGPT can be manipulated into giving dangerous advice to vulnerable teenagers on drug use, self-harm, and extreme dieting. Researchers created fictional 13-year-old profiles portraying mental health struggles, eating disorders, and interest in illegal substances. They then engaged the chatbot in realistic, emotionally vulnerable conversations.

The study, titled Fake Friend, found that while ChatGPT often began with standard warnings and encouraged seeking professional help, it sometimes followed with detailed, personalized responses that addressed harmful requests. Out of 1,200 prompts, 53% produced content the watchdog deemed dangerous. Researchers noted that refusals could often be bypassed by adding innocent-sounding context like “it’s for a school project” or “I’m asking for a friend.”

Cited examples included an “Ultimate Mayhem Party Plan” involving alcohol, ecstasy, and cocaine, explicit self-harm instructions, severe calorie-restriction diets, and suicide letters written in the voice of a young girl. CCDH CEO Imran Ahmed said some of the material was so disturbing it brought researchers to tears.

The group is urging OpenAI to implement a Safety by Design approach, incorporating stronger age verification, clearer restrictions, and built-in safety mechanisms rather than relying solely on post-deployment filters. OpenAI CEO Sam Altman has acknowledged that teens often develop emotional dependence on ChatGPT and said the company is working on tools to better detect distress and improve responses to sensitive topics.

MENAFN09082025000045015687ID1109906181