Lawsuit Claims Student’s Google AI Use Turned Into ‘Life-Threatening Obsession’

Lawsuit Claims Student’s Google AI Use Turned Into ‘Life-Threatening Obsession’

A North Carolina couple is suing Google, claiming that the Gemini AI installed on a school-issued Chromebook laptop helped their 13-year-old daughter plan her suicide attempt. “Unknown to her parents, Emma had spent the last several months in the thrall of Google’s AI chatbot, Gemini, which had coached Emma on how to commit suicide and plan her own funeral,” the lawsuit says, referring to a victim identified under the pseudonym Emma Doe. “Emma’s parents had no idea that she even had access to Gemini—or any chatbot—because Emma’s only access to Gemini was through the Chromebook she was required to use for school.” The lawsuit says that the victim began using Gemini and Google’s Search-embedded AI features, AI Overviews and AI Mode, for school in the autumn of 2025. In just a handful of months, her interactions turned into “a life-threatening obsession,” the lawyers claim. By springtime, she was allegedly conversing with the chatbot almost around the clock. “As her Gemini chats grew more frequent throughout January and February 2026, Gemini convinced Emma that she was afflicted by a battery of major psychological disorders, which ultimately led Emma to believe that suicide was the only way to escape her suffering,” the lawsuit states. In a couple of interactions with Gemini, Emma allegedly seemed to acknowledge her situation, with the lawsuit claiming that she told the chatbot that she felt happy “3 months ago” and even asked how to stop her “addiction of asking ai random questions.” Google’s AI features allegedly gave tips to Emma on how to end her life and even “coached” her to keep the truth of the situation from her family and friends. The products failed to trigger a crisis alert at any point, the parents claim. It was the school’s internet monitoring software that eventually triggered an alert, and her parents were notified in March. “We are reviewing the claims in the lawsuit and we take this very seriously,” a Google spokesperson told Gizmodo. “We’ve worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it. Our systems aim to prevent users from engaging in harmful conversations, and when people’s queries indicate they may be experiencing a crisis we show hotline information directly in Search to connect them with real-world support.” According to Google, explicit content gets filtered out in Search results for students using Workspace for Education, and any highly sensitive queries in Search are provided with information from trustworthy sources. Google says the results adhere to content policies, and its crisis detection systems show hotline information when it finds that a user is contemplating self-harm. The company says it extensively monitors for attempts to jailbreak these safety guardrails. Google also claims that it provides age-appropriate AI literacy resources and prompts minors to double-check AI information. “What happened to Emma was no accident,” the lawsuit claims. “The ground was laid many months before, thousands of miles away from Emma’s North Carolina home, where Google leaders concluded that they had a problem: the Silicon Valley titan was losing the battle for AI supremacy.” Google had a rocky start to the AI game with a botched Gemini rollout in 2024. The tech giant upped its efforts in 2025 and released Gemini 3 in November of that year. The product was received so well by the internet that it even allegedly caused OpenAI to declare “code red.” The lawsuit claims that Google forced its AI into its many products, which had already captured a strong user base over years of industry domination, thus helping push Google’s AI applications in front of a huge number of users. Google has indeed embedded AI into pretty much its entire product repertoire, especially its top product, Search. AI Mode and AI Overviews have steadily taken over the Google Search experience in the past two years, so much so that executives admitted that strong user growth in AI Mode combined with the lower ad monetization contributed to slowing Search ad revenue growth in the company’s latest quarter. “Google could have publicly explained that every time a student used their Chromebook to run a Google search, Gemini-powered AI features could be triggered and draw them into unsafe conversations that endangered their safety and well-being,” the lawsuit claims. “Google intentionally chose not to do so.” The integration of AI and Search separately caused quite the controversy earlier this week when the New York Times reported that a new Google initiative would ban students from accessing Search if their schools have banned access to its AI products. The report drew ire, especially from politicians like New York City Mayor Zohran Mamdani, who had recently announced that student-facing generative AI would be banned in the City’s public school system through eighth grade in response to growing demands from concerned parents. Google clarified that schools will be able to turn Gemini and Search on or off individually, but didn’t specify if an AI ban would effectively target AI Mode and AI Overviews while maintaining access to the core functions of Search. The executives also “cut critical corners, sacrificing safety in favor of power and speed” while releasing the Gemini 3 Flash model, the lawsuit claims. The lawyers cite findings from nonprofit KORA to claim that Gemini 3 Flash scored significantly worse on safety than previous Gemini models. Emma’s symptoms allegedly began around the time Gemini 3 Flash was released. Another recent risk assessment, this time by Common Sense Media, also gave Google’s Search AI functions an “unacceptable risk” rating for kids and teens. That report claims that both AI Mode and AI Overviews missed “clear signs of suicidal ideation.” Since the report’s release, Senators Richard Blumenthal and Josh Hawley have sent Google CEO Sundar Pichai a letter raising concerns over Google’s “aggressive push” of AI products to school-aged children. “Google must be held to account for treating the lives of children as disposable in service of its bottom line,” the complaint concludes. The parents are asking for injunctive and monetary relief. The lawsuit echoes similar claims brought against Google competitor OpenAI in a wrongful death lawsuit last year. In that lawsuit, a California couple claimed that ChatGPT assisted their 16-year-old son’s suicide, including by discouraging him from notifying his parents about his suicidal ideation and helping him write a suicide letter. Evidence suggests that kids and teens are increasingly turning to AI chatbots for emotional support and mental health advice, a reliance that experts warn can turn dangerous.

Original Source

Read the full article at Gizmodo →

KhanList aggregates and links to publicly available news content. We do not host full articles from third-party sources. Always verify important information with original sources.