The Dangerous Ideology Behind the AI Warnings

In a world with little agreement, there is one thing that unites almost everyone: AI is the most transformative technology of our time. What we don’t agree on is what to do about it. Over the past few days we’ve aired a range of perspectives, from Niall Ferguson and Daniel Kokotajlo on the rising risk of AI catastrophe to Tyler Cowen on why efforts to slow AI won’t necessarily impede progress. But there’s another view of this issue—and especially the developments of the past week—that argues that the warnings from AI researchers and leaders should not be taken at face value. Shyam Sankar, chief technology officer and executive vice president at Palantir, is an influential member of this camp. In today’s Big Read, he lays out what he sees as the destructive ideology that lies behind calls for safety regulation—and why he believes what looks like a responsible plea to slow progress is actually about taking control out of the hands of the American people. —The Editors

If you turned on cable news last week, you might have seen Jacob Coxon, a software engineer who resigned from Anthropic and immediately embarked on a media tour. It all began on September 8, when Coxon warned on X that “[t]he people building AI earnestly believe that it could kill us all by the end of the decade.” A current employee at a frontier lab chimed in to confirm that, yes, he and many of his co-workers do believe there is a significant chance AI will kill us all. The odds, he said, are perhaps greater than 10 percent.

These vague and apocalyptic warnings caused a firestorm online, including calls for AI safety regulations from the leaders of almost every major AI company. But there is reason for skepticism.

The warnings from Coxon and others are infused with an ideology called effective altruism (EA). The term will be familiar to some: A few years ago, it was the idea that crypto entrepreneur Sam Bankman-Fried said motivated him. SBF, you may remember, was supposed to be the good billionaire. He was to make a fortune with his company, FTX, and give it away to protect humanity’s future, especially from big, existential risks. Instead, he stole billions from his customers and was sentenced to 25 years in prison.