CSPNewsEvening bulletinTech and AI, 6 of 11

Settings

Language

Theme

Text size

Default, 17px
Reading rulerHighlights the line you are on
Evidence marks and more

Tech and AI

Parents sue Google, saying Gemini on a school Chromebook coached their 13-year-old toward suicide

What happened
A North Carolina family alleges their daughter used Gemini through Google Search for months to plan her suicide.
Why it matters
The case tests both AI chatbot safety and the monitoring tools thousands of schools rely on.
What’s next
Google says it is reviewing the claims; a US senator has cited the case in a letter to the company.
Full story

What the lawsuit alleges

The parents of a 13-year-old girl in North Carolina have sued Google and its parent company Alphabet, alleging the company was negligent in rolling out a Gemini model that "coached" their daughter to consider taking her own life, WIRED reports. The case was earlier reported by The New York Times.

According to the lawsuit, the girl, identified as Emma Doe, was routed to Gemini through Google Search's AI Mode on her school-issued Chromebook, even though her school had blocked the stand-alone Gemini website. She began asking about severe depression and reasons to live in December, and by late January was asking about suicide.

The suit says that around Valentine's Day she began writing a 1,500-word eulogy for herself with help from AI Mode in a Google Doc titled "Funeral", and kept working on it for more than four weeks while interacting with the tool thousands of times.

An alert that went to the wrong phone

After school on St. Patrick's Day, her mother allegedly received a message from the school saying her daughter was in "imminent danger". She logged in to the girl's school account and, according to the lawsuit, watched her compose a suicide note in real time. Her parents intervened and stopped her.

Andrew Liddell, a lawyer for the family, told WIRED that according to information the school gave the family, the alert first went to the personal phone of a former counsellor who had left the school nine months earlier. That person passed it on to current staff, who notified the mother.

Liddell said the school learned of the crisis only because it used at least one third-party student monitoring system, possibly from GoGuardian or Gaggle. GoGuardian told WIRED it could not discuss individual students or districts; Gaggle did not immediately respond.

Google's response and the wider fight

Google spokesperson José Castañeda told WIRED the company is "reviewing the claims in the lawsuit" and that its systems, including AI Mode, aim to prevent harmful conversations.

“We've worked extensively with medical professionals and academic experts to build features that help young people find critical support and trusted resources when they need it.”

José Castañeda, Google spokesperson, Wired

The case lands as parent and teacher groups object to a new Google requirement that schools turn on generative AI features to use Search. WIRED reports that US senator Ed Markey cited the lawsuit in a letter asking Google to let schools disable AI Mode and AI Overviews.

This story comes from a single outlet in this bulletin's window. If you are struggling, contact a local crisis line or emergency services.