Secret video ‘marks’, AI sliders & MILLIONS of dodgy spammers banned – TikTok boss reveals future of the app in AI age

IN just a few years, AI has gone from an invisible force behind the scenes to…well, basically being everywhere. For tech giants like TikTok, it means an enormous job of separating top-notch clips from AI slop – and helping users to spot dodgy content on their own. TikTok’s Tom Varghese sat down to chat with The Sun’s Sean Keach (The Sun) and Tom Varghese (TikTok) Credit: The Sun / Future Tech Feed Speaking to The Sun on Future Tech Feed with Sean Keach (Watch the full episode on YouTube), TikTok’s global policy lead for AI shared how his team is future-proofing the app so you can scroll in peace. Tom Varghese told how he and his colleagues are using brainy computing systems, secret “invisible watermarks” and mass account purging to keep the app clean. Sign up for the Tech newsletter Thank you! And the app is even dishing out handy alerts and advice to make sure you don’t end up fooled. AI SPLIT Not all AI is bad. Some AI content is absolute rubbish, and it can even be dangerous. But Tom explained that AI is helping regular TikTok users make videos without fancy editing skills or huge budgets. “If you’ve ever created a video on TikTok, you usually have some kind of idea or a vision in mind,” Tom told The Sun’s Future Tech Feed. “And what the creative AI tools can do is essentially to shink the gap between having an idea and expressing it. Most read in Tech He gave the example of the popular account Seconds in History. It uses AI to imagine long-past historical scenes, or to bring viewers on a virtual walking tour of Ancient Rome – something that wouldn’t have been previously possible without a huge budget. Tom Varghese is the artificial intelligence lead in TikTok’s Global Public Policy team Credit: TikTok TikTok is working to label AI-generated content for viewers Credit: TikTok “For a lot of people, this makes history more vivid and accessible,” said Tom. “So these are the types of examples that make us believe that AI can transform creativity on our platform, if it’s used transparently and safely.” THE PURGE The dark side of AI is when it’s used by online ne’er-do-wells to churn out cheap and often misleading spam at a staggering scale. Particularly dangerous is any misinformation around healthcare, politics and current events, and finance, Tom tells us. But AI content can still be spammy and annoying – even if it isn’t outright dangerous. TikTok is taking aim at this type of content with a major crackdown on automated accounts that flood the feed with low-value clips. This crackdown has resulted in tens of millions of fake accounts being scrubbed from TikTok in the first few months of the year alone. “I think we’ve removed more than 86 million fake accounts just in the first three months of this year,” said Tom. “But as this kind of content evolves, so must we.” And the good news is that the vast majority of time, this is happening before the dodgy content ever makes it to you. “So 99% of violative content on the platform is removed proactively without anyone reporting it,” Tom said. “And the majority of that before anyone sees it.” TikTok doesn’t ban creators from using AI, and even has some AI tools to help produce videos quickly. But the company has a zero-tolerance approach to automated spam farms. “There’s a clear difference, right?,” said Tom. “Between a creator using AI to bring an original to life. “And an account that’s designed to flood the platform with low value or misleading AI-generated material.” SPAM TRIANGLE When TikTok is looking at cracking down on spam, it looks at three specific points. These are the telltale signs that an account is dishing out rubbish AI content that needs to be binned. The first point is when an account is uploading “low quality, high volume, formulaic content”.This kind of AI mass production is a major red flag. The second is whether there is a kind of first-person active engagement with the user. That’s a way of spotting automated bots that might be running the show. And the third is the topical focus, so does it pose “risks to safety and health”. “And the reason we’re focusing on topics like current events, financial advice, historical atrocities, medical content, it’s because this is where there’s the highest potential for real world harm,” Tom said. TikTok is one of the most popular social media apps in the world with more than 2 billion users Credit: CFOTO/Future Publishing via Gett TikTok is dishing out resources to help users better identify AI content – and how to harness it for their own clips Credit: TikTok “We’ve trained our systems, and the new detection tools we’re rolling out are really looking for the type of accounts that meet these three criteria.” He went on: “Because advanced moderation technologies, they do help us at scale. “They play a significant role in ensuring that we’re proactive, that we’re concise, consistent, and do this at speed. “But then you also need people because a lot of these issues require human judgment and understanding all that context that machines not always do.” SECRET WATERMARKS We’ve all seen how AI videos can now be shockingly realistic. So how are the tech companies spotting and flagging them? There are a few different systems, but two of the main ones are C2PA and digital watermarking. C2PA is a kind of “content credential”, which works a bit like a digital history log that’s embedded directly into a file. So that will detail exactly how and when a piece of content was created. “That log travels with the file across platforms, allowing others to detect and label content,” Tom explained. According to Tom, TikTok has already labelled more than 3 billion videos as AI generated. Another technique is something called invisible watermarking. You might be familiar with regular watermarking, which is when you see a logo printed on a video – maybe with a brand logo or a creator’s username. But TikTok relies on a kind of invisible watermark for tracking AI. “That’s where you embed a signal directly into the pixels of an image or video,” Tom said. “And it’s not something that you can see with the visible eye, but this watermark can later be detected. There’s now an AI-Generated Content slider in Manage Topics to change how much AI you see Credit: TikTok Future Tech Feed with Sean Keach sees expert guests answering big questions about what’s next Credit: FTF “We can use it to verify whether a piece of AI-generated content out there on the internet was actually created using one of our creativity tools.” IN CONTROL It’s not all crackdown though. Tom says a big part of the effort is handing more tools to regular app users. For a start, TikTok recently introduced a new AI slider to its Manage Topics selection. This allows you to ramp up or scale down the number of AI-generated videos that you see in your feed. There’s also a new in-app AI Literacy Hub that links users up with educational resources when they search for AI topics. “People come to TikTok to have an authentic experience,” Tom said. “And we want them to know that when you’re looking at a video, it is created by a real person, that it’s authentic content.” He added: “Being able to spot when AI is being used, understand what you’re seeing, and being able to shape that experience is going to be fundamental to how we want people engaging with AI on TikTok.” Comment now