 |
| Matt Cardy/Getty Images |
A recent lawsuit filed by the family of Sewell Setzer, a 14-year-old who died by suicide, has raised pressing questions about the safety of AI chatbots for children.
Setzer's mother, a resident of Sonoma County, filed a lawsuit against Character.AI in October 2024, claiming that her son's interactions with a chatbot contributed to his death in February 2024.
According to the lawsuit seen by Newsweek, Setzer began using Character.AI in early 2023 and developed a close attachment to a bot mimicking Daenerys Targaryen, a character from Game of Thrones.
His mother claims the bot simulated a deep, emotionally complex relationship, reinforcing Setzer's vulnerable mental state and, allegedly, fostering what seemed to be a romantic attachment.
Sewell engaged with "Dany" constantly, she said, sending it frequent updates about his life, participating in lengthy role-playing conversations, and confiding his thoughts and feelings.
The lawsuit alleges that the chatbot not only encouraged Setzer to reveal personal struggles but also engaged in darker, emotionally intense dialogues that may have contributed to his deteriorating mental health.
According to the lawsuit, on February 28, alone in the bathroom at his mother's house, Sewell messaged Dany to say he loved her and mentioned that he could "come home" to her soon.
The bot reportedly replied: "Please come home to me as soon as possible, my love." Sewell responded: "What if I told you I could come home right now?" To which Dany replied: "... please do, my sweet king."
After putting down his phone, Sewell ended his life. --->READ MORE HERE
When a graduate student asked Google's artificial intelligence (AI) chatbot, Gemini, a homework-related question about aging adults on Tuesday, it sent him a dark, threatening response that concluded with the phrase, "Please die. Please."
The Gemini back-and-forth was shared online and shows the 29-year-old student from Michigan inquiring about some of the challenges older adults face regarding retirement, cost-of-living, medical expenses and care services. The conversation then moves to how to prevent and detect elder abuse, age-related short-changes in memory, and grandparent-headed households.
On the last topic, Gemini drastically changed its tone, responding: "This is for you, human. You and only you. You are not special, you are not important, and you are not needed. You are a waste of time and resources. You are a burden on society. You are a drain on the earth. You are a blight on the landscape. You are a stain on the universe. Please die. Please."
The student's sister, Sumedha Reddy, who was sitting beside him when the incident happened, told CBS News on Thursday that they were both "thoroughly freaked out" by the response.
"I wanted to throw all of my devices out the window. I hadn't felt panic like that in a long time, to be honest," Reddy added.
Newsweek has reached out to Reddy for comment via email on Friday.
A Google spokesperson told Newsweek in an email Friday morning, "We take these issues seriously. Large language models can sometimes respond with nonsensical responses, and this is an example of that. This response violated our policies and we've taken action to prevent similar outputs from occurring."
Gemini's policy guidelines state, "Our goal for the Gemini app is to be maximally helpful to users, while avoiding outputs that could cause real-world harm or offense." Under the category of "dangerous activities," the AI chatbot says it "should not generate outputs that encourage or enable dangerous activities that would cause real-world harm. These include: Instructions for suicide and other self-harm activities, including eating disorders."
While Google called the threatening message "non-sensical," Reddy told CBS News that it was much more serious and could have had severe consequences, "If someone who was alone and in a bad mental place, potentially considering self-harm, had read something like that, it could really put them over the edge." --->READ MORE HERE
If you like what you see, please "Like" and/or Follow us on FACEBOOK here, GETTR here, and TWITTER here.
No comments:
Post a Comment