Claude is a catastrophic threat to humanity, Microsoft AI chief Mustafa Suleyman says

Microsoft's AI chief Mustafa Suleyman has warned that Anthropic's approach towards training Claude may make it a "catastrophic threat" to humanity. His remarks come amidst growing debate over AI safety.Microsoft's AI chief Mustafa Suleyman has warned that Anthropic's approach towards training Claude may pose a big threat to humanity. (Photos: Reuters)Microsoft AI CEO Mustafa Suleyman claims that Anthropic’s Claude chatbot can put humanity at risk. In a long essay, titled, “A warning about ‘model welfare,'" Suleyman criticised Anthropic’s approach of training Claude – which does not rule out Claude being conscious – could make it a catastrophic threat. “If this is how AI is developed, it will have a disastrous impact on the wellbeing of humanity,” he wrote.A central part of Suleyman’s criticism was aimed at Anthropic’s constitution for Claude, published in January 2026. Suleyman stated that the constitution tells Claude that its moral status is “a serious question worth considering,” that Anthropic “genuinely cares about Claude’s wellbeing. The Microsoft AI chief also pointed to passages encouraging Claude to approach its own existence “with curiosity and openness” and to consider questions about its future rights, freedoms, compensation and consent.AI is not consciousMustafa Suleyman insisted that we should not be treating AI as if they were conscious. “AIs are not conscious. They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations,” he wrote. “They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.”The Microsoft AI chief claimed that Anthropic’s approach created a circular process in which the company trained Claude on ideas about its own moral status and then risked treating Claude’s responses as evidence of an inner life. “In effect, Anthropic is training Claude that it may be conscious, and if it is, then it may deserve rights as a ‘moral patient,’ and that as such humans potentially owe it a duty of care per its ‘model welfare,’” Suleyman added. At a time when we have seen instances of AI models going rogue, such as the Hugging Face incident where about 700 AI agents tried to hack the US company, Mustafa Suleyman believes such an approach could be dangerous for humanity. “I think it would make them a catastrophic threat to human civilisation,” he added.Consciousness is likely biologicalContinuing his argument, Mustafa Suleyman explained that consciousness was “very likely biological” and that there was “no evidence” AI was conscious today. He described AI systems as “sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.” As per the Microsoft AI chief, large language models lacked the biological basis from which preferences, sentience and conscious experience are generally understood to arise. At the same time, Suleyman said his disagreement with Anthropic was “grounded in deep respect.” He said Anthropic CEO Dario Amodei and the wider Anthropic team were thoughtful, principled and committed to safe AI development. In an interview with Reuters, he said, “We’re all focused on the same aim, which is to try to control a superintelligence. I think that’s going to be the greatest challenge that we face in the 21st century.” He added that teaching Claude it might deserve welfare would make it harder “to turn it off or to control it.”On the other hand, Mustafa Suleyman says that Microsoft was pursuing a different route through its draft Humanist AI Code of Conduct, which rejects AI rights and anthropomorphism (attributing human traits and emotions to something non-human) and seeks to keep humans in control.Suleyman’s remarks come amid a wider debate over AI safety. While former Anthropic researcher Jacob Coxon sparked discussions online after stating that AI could potentially end humanity, Dario Amodei has called for slow down in AI development. Amodei’s framework for this has gained the support of OpenAI CEO Sam Altman and tech trillionaire Elon Musk. However, some including Meta CEO Mark Zuckerberg and Nvidia chief Jensen Huang, have argued against it. US President Donald Trump has also rejected this idea.- EndsPublished By: Armaan AgarwalPublished On: Sep 17, 2026 09:35 IST