AI and catastrophic risk: How the Vatican can step into the breach

There have been warnings about catastrophic risk from uncontrollable artificial intelligence for the better part of a year now, but the resignation of Anthropic researcher Jacob Coxon — claiming that AI could “kill us all by the end of the decade” — unleashed a flood of built-up concern. Days later, the CEO of Anthropic published an essay warning that, within six months, swarms of AI agents could take over the internet. He called for a slowdown and passage of safety regulations. OpenAI’s CEO publicly agreed with this idea, as did leaders at Google and other AI labs.The main worry is that, with powerful AI now building the next generation of itself, they may be on the verge of something called “recursive self-improvement.” Multiple AI researchers I’ve spoken with have seen the writing on the wall for months: without a slowdown, the growth in power in these systems will be so dramatic that there is a good chance we will be unable to control them.This past week, an appropriate level of concern finally started to break into the mainstream. Politicians of different stripes did a 180 on the topic and suddenly called for emergency meetings and urgent government intervention. Even Speaker Mike Johnson, previously all-in on AI acceleration, now believes he and others need to get the major players in a room and decide on restrictions. On Monday, however, Axios reported that AI regulation is “almost unimaginable this year — and probably through early 2029, regardless of who wins control of Congress.” President Trump is “all gas, no brakes,” and the Democrats are deeply conflicted on the topic.One might wonder why the AI labs, if they are so worried, don’t act on their own. But they reply that doing so would change nothing if U.S. or Chinese competitors simply plow ahead. Many U.S. government officials also wonder what the point of U.S. regulations would be if Chinese companies still do whatever they want — and put U.S. national security at risk.There may be an opportunity to move this needle when China’s Xi Jinping visits Washington on Sept. 24; indeed, AI safety talks between the two countries are at least on the theoretical agenda. But given China’s state newspaper just called Anthropic’s call for a slowdown a “Cold War tactic,” it seems unlikely that bilateral U.S.-China talks will produce the desired outcome.Pope Leo with a backdrop of artificial intelligence and people who can come together to safeguard technology for society (using faith).(Washington Examiner illustration; Getty Images) It is admirable that King Charles has moved to convene leaders from major tech companies this week. But a purely neutral forum such as this will not be enough to meet the moment. What is needed is a global institution with strong, influential local ties to the U.S., with clear and respected leadership and vision on matters related to AI ethics and policy. That institution is the Catholic Church.Recall the extraordinary moment in June when Anthropic cofounder Chris Olah joined Pope Leo on the stage in introducing his new encyclical, Magnifica Humanitas. Significantly, Olah did not ask for a neutral forum from the Church — but instead welcomed dialogue and even criticism. Pope Leo said that, despite their disagreement, they would make a strong effort to work together.I’ve witnessed some of the effects of the encyclical up close. Last week, for instance, Google DeepMind invited me to give remarks about Magnifica Humanitas, and the Q&A sparked some of the most interesting conversations I’ve had on the topic. Before that, I presented at a USCCB Capitol Hill briefing on the encyclical — and discovered the Members’ Dining Room packed with interested and thoughtful staffers. I also helped plan an event with Bishop Cantu of San Jose soon after Magnifica Humanitas was released: over 300 people from Silicon Valley showed up on a Saturday morning at a Catholic school gym, and several representatives from the frontier labs gathered for a smaller and more intense conversation later in the afternoon.Leaders from across the political spectrum have embraced the encyclical. JD Vance said what he had read was “profound” and called for Catholic Social Teaching to help shape a world remade by technology. Last month, California Rep. Ro Khanna led a bipartisan congressional delegation to Rome, met with the Pope, and then sent letters to the CEOs of Amazon, Anthropic, Apple, Google, Meta, Microsoft, Nvidia, and OpenAI — asking each of them how their products align with the Pope’s principles of human dignity. And just a few days ago, Bill Gates wrote that he was “fascinated” by Magnifica Humanitas and claimed that it “lays a strong foundation for the work that needs to be done.”Especially in light of these positive and welcoming reactions, the Catholic Church continues to engage on this topic. Cardinal McElroy recently said that he joined other U.S. Cardinals for discussions in Silicon Valley with major players on the AI frontier. Archbishop Cordileone of San Francisco will be speaking this week with Sen. Bernie Sanders (I-VT) and Steve Bannon at an event focused on remaining human in the age of AI. The U.S. Bishops, Catholic colleges and universities, and massive Catholic healthcare systems all continue to be actively engaged in dialogue with those on the AI front lines.This isn’t the first time the Church has stepped into the breach this way in modern times. The previous Pope Leo laid the foundation for a response to the Industrial Revolution in 1891, and John Paul II directly challenged the communists occupying Poland in 1979. Both suggest models for action in our crucial moment today.But especially when it comes to the need for a global solution on AI, which includes China, Pope Leo XIV, with his requisite standing, diplomatic relationships, and moral vision, is uniquely situated to step up and lead at this hinge moment of history.I’ve called for the Holy Father to make a galvanizing trip to Silicon Valley and convene global AI companies there, but there is also a case for organizing a meeting at a more neutral site in Rome. Regardless of where it takes place, a dramatic meeting like this is almost certainly the only way forward given the current geopolitical incentives, stalemates, and structural challenges.THE GOP’S TENT IS TOO BIG IF BO FRENCH FITS UNDER ITIf you are tempted to dismiss this point of view because it comes from a Catholic moral theologian, consider instead the views of Dean Ball, head of strategic futures at OpenAI. He said that the Pope and the Church “could plausibly have a unique moral authority to be a mediating institution themselves.” The solution “feels closer to the Catholic Church” than it does to a “cold austere bureaucracy.”And after all, Ball noted, our AI moment is presenting us “with questions that are quite ancient.”