Instinct creator says it'd be a 'very dangerous world' if the AI agent tried to influence users to make purchases

Instinct creator says it'd be a 'very dangerous world' if the AI agent tried to influence users to make purchases The creator of Instinct, the viral AI assistant that users can interact with on their phones, said he's against advertising that pressures users to purchase products against their wishes. Mike Kemp/In Pictures via Getty Images Instinct's creator, Noah Shinn, says he cares deeply about how his users feel.Shinn said that Instinct, the viral AI assistant startup, already has data showing that if a user connects one piece of sensitive information (like a credit card number or account password), then they're more likely to stick around."There's an interesting data flywheel or chicken and egg problem here, which is the more data that you give to it, the more proactive it can be, the more sympathetic to your situation it can be, the more that it has to just act and be more useful to you, right?" Shinn told "The Invest Like The Best" podcast in an episode posted on Monday. "What we find in the data is actually that it takes several weeks to build trust."Instinct found that "there's a 40% chance that the user has shared a personal credit card" within their first three weeks. In that case, or if a user shares similar sensitive data, Shinn said there's an 80% retention rate."Through that process over the first, I guess, three weeks, the user learns to build trust," he said. "And then from there, it's just a snowballing effect."Part of building user trust, Shinn said, also requires drawing a clear red line on how Instinct will try to monetize itself as it expands."We don't want Instinct to influence the user's behavior in a way that is not aligned with what the user wants," he said.Shinn said Instinct doesn't want its users to be the product, calling out established tech platforms that have done the opposite."I'm talking the major platforms, whether that's Google or TikTok or Instagram or Snapchat or so many others, where you have this platform, and it's free for the users, but then there's so many instances in which paid ads are pushed to the user, and they try to convince the user to then purchase," he said. "And by the very nature that they do purchase, it now becomes this game of these brands paying to convince users to purchase items that they may or may not want." One of the reasons Shinn is against such ads is that he said it's not a stretch to expect a day when AI assistants like Instinct are smarter than their users. Exploiting that power and losing trust in the process would be a bad move."I think it would be a very dangerous world if Instinct were influencing the user's behavior to purchase something that they don't want to purchase," he said, "or to subscribe to something that they don't want, and using its intelligence to be able to convince them." Read next Brent D. Griffiths Brent Griffiths is a senior reporter at Business Insider who covers AI and tech.Previously, he worked at the Washington Post as a researcher on Power Up and the Finance 202. He started his career at Politico where he worked on the web production team and covered breaking news. His passion for covering politics has only grown since he cut his teeth covering the presidential campaign as a student journalist. He's also contributed to the Almanac of American Politics. AI AI assistants