Opinion: Children are turning to AI for advice. We have terrifying examples of where this can lead

With the recent focus on social media and how safe it is for our children, we are in danger of losing sight of a potentially bigger concern: AI chatbots. This year’s report from CyberSafeKids shows a sharp rise in children’s use of AI chatbots. In our 2024 report – the first year we tracked this – just 5 per cent of younger children (aged 8-12) were using AI chatbots. This year, that figure is 55 per cent: an elevenfold increase in two years.It’s not hard to see why either since AI chatbots are now embedded in the platforms children use most – from WhatsApp and Snapchat to Roblox, as well as stand-alone apps such as ChatGPT.And while the headlines focus on Meta’s latest $18 billion settlement of a lawsuit brought by over half of US states over social media harms, and TikTok coming under fire for failing to adequately moderate harmful content, we’re missing the point entirely on generative AI’s impact on children’s health and wellbeing.Take it from the children themselves. One 13-year-old told us in an online survey done in schools: “I can easily access ChatGPT and other AI websites and you guys are not doing anything to help the kids but you all still allow AI use in the school. Genuinely one of the worst things I’ve seen online is people killing themselves or hurting themselves online because of AI, like Character.ai and more.” It’s hard to know whether this child is referring to cases they’ve read about or prompts they may be giving to a chatbot, but either way, it raises alarm bells.READ MORESo to this child’s point: what are we doing about it? Not enough, according to our latest data, with 50 per cent of younger children and 45 per cent of the older children (aged 12-15) telling us that no one has talked to them about AI at all, never mind how to use it safely.More worrying still, given how conversational, self-affirming and persuasive they are designed to be, is that 16 per cent of under-13s and 20 per cent of over-13s are talking to chatbots to “chat” and “get advice” and that 10 per cent of eight- to 12-year-olds and 8 per cent of 12 – to 15-year-olds find it easier to talk to AI than to people they know in real life.We have terrifying examples of where this can lead. There have been high-profile cases in the US and the UK in which children are alleged to have taken their own lives after seeking advice from chatbots.This is all the more concerning given that chatbots are largely unregulated. The EU AI Act, while significant in many ways, treats chatbots as “limited risk” rather than the more stringent “high risk” category.[ The Irish Times view on the impact of AI: a timely warning from Bill GatesOpens in new window ]The only regulatory obligation – and this is only new as of last month – is to tell users they’re talking to an AI, a requirement that did not apply to the thousands of children we surveyed that were engaging with these chatbots over the past year. Even then, there is a waiver if it’s already obvious to “a reasonably well-informed, observant and circumspect person” that they’re talking to AI, given the context. Yet another “get-out” clause that will be hard to prove in the inevitable court cases to come – like the recent Meta litigation.Despite the scale of the settlement, I fear little has changed beyond a small financial hit to Meta’s financial juggernaut: up to $18 billion, payable over 10 years, is a mere 13 per cent of Meta’s targeted AI investment for 2026 alone – and comes with many strings attached.Some have lauded the outcome as a “win” for children, but many experts are far more sceptical. These include one of the chief witnesses, Arturo Béjar, a former Meta whistleblower. He said: “The limitations … in the agreement are the equivalent of saying: ‘Well, you can smoke as many cigarettes as you can in two hours a day’. It doesn’t make the cigarettes any safer.”That’s a crucial point. “Social media” (and I’m using that term in the broadest sense here to include gaming and the often-embedded AI chatbots) in its current form is simply not safe by design for children, because it relies on a business model that is designed for profit over everything else, including safety.This doesn’t stop the companies behind these platforms from claiming that they are indeed safe for teens. The phrases “safe by design” and “no trade-offs where children’s safety is concerned” are so overused they have become meaningless cliches. I’ve even heard one member of a Trust and Safety team say at a safety event that keeping children safe online was “the reason I get up in the morning”.Yet the reality is so far from the PR spin. All of these companies – Meta, TikTok, YouTube and Snapchat – are still facing thousands of US lawsuits alleging their platforms were designed to addict and harm children’s mental health. Snapchat is facing scrutiny and a growing backlash over claims the platform’s disappearing messages are implicated in drug sales to minors. Roblox, the most popular online environment for a child under the age of 13 in Ireland today, is accused of failing to prevent grooming.CyberSafeKids’ report findings further underline the fact that children are far from safe on these platforms. The regulations we have are not making the difference they need to. The outcomes for children are no better this year than last – we still see the same number of underage users (70 per cent) accessing platforms with a minimum age restriction of 13, despite an Online Safety Code that has been in effect for over a year and European legislation that requires more robust age gates to prevent underage access.[ ‘Humanity needs to wake up’ to AI dangers, says Anthropic chiefOpens in new window ]Despite new regulation and new court settlements, children are not fundamentally safer in the online environments they inhabit. There is still no legal definition in Europe of “safe by design”. There is no requirement to prove a product meets this bar before it reaches children. The only chance to enforce legislation comes after harm has been proven to have occurred – and even then, it’s mired in years of legal challenge, as we’re already seeing with the Data Protection Commission fines and the large-scale European Commission investigations. And while all those drag out, we need to ask ourselves at what cost to society, to our children’s health and wellbeing?We need to stop waiting for these companies to do better and start setting the rules of engagement ourselves.Alex Cooney is chief executive of Ireland’s children’s online safety charity CyberSafeKids. Its latest report is available at www.cybersafekids.ie