Critics have long accused Anthropic of “doom marketing” — hyping its own products as so good that they are bad. But the post’s co-author, who is also the company’s co-founder, says the motive is very different. “We say this stuff because we think the world needs to know the truth about what’s happening,” Jack Clark, who now heads Anthropic’s public benefit work, said in an interview.
He’s not the only AI leader warning that the already fast-evolving technology is on the cusp of accelerating into a potentially dangerous new phase. And that’s spreading alarm across the planet.
Dizzying amounts of money are pouring into AI. Stock markets increasingly look like a giant bet that it will transform work and spur unprecedented breakthroughs in science. The boom in AI capital spending accounts for a large and rising share of economic growth. This week’s record SpaceX IPO captured the euphoria — and made Elon Musk the world’s first trillionaire. Yet all the while, much of the global public is looking on aghast.
See also: Sea testing AI chatbot Migoo in quiet US foray — Bloomberg
Pom-poms and pitchforks
“Things are going to transform,” said Evan Solomon, Canada’s AI minister. “The pom-pom crowd has a strong case that AI is going to be very beneficial,” he said. “The pitchforks crowd also has a case of concern: What is this gonna mean for my privacy, my data, my job, my skill, the future?”
In the latest example of mounting concerns, the US government ordered Anthropic on Friday to shut off access to two of its most advanced AI models to any foreign national, citing national security concerns. In response, the company disabled access for all customers to ensure compliance. But the firm also pushed back against the directive in a statement on its website, saying the government is acting on information that isn’t threatening enough to warrant a shutdown.
See also: Return on investments in AI uncertain, says MAS chief
These fears are showing up in polls and local politics. There’s growing pushback against data-centre plans, while among Americans AI is now less popular than the immigration enforcement agency ICE, according to a recent NBC survey. It leaves Anthropic and its peers in a difficult spot: trying to strike a balance between developing a technology that might disrupt people’s lives and vowing to defend those same people against undue risks, all while raising a ton of money from investors.
As global capitalism bets the house on an AI future, cash-strapped governments have no easy way to square the circle if the fallout gets costly. And if enough people get angry about what the AI transformation could mean for them, it might not happen at all. Pope Leo XIV and China’s Workers’ Daily — each broadcasting to a potential audience of 1.4 billion people — are among the latest to sound the alarm.
Few such worries trouble SpaceX. Musk’s sprawling rockets-and-AI venture this week kicked off a US$3.6 trillion wave of initial public offerings that amounts to a massive bet on AI’s potential. The shares closed Friday up 19% from the IPO price. Rivals Anthropic and OpenAI — which has also warned of the dangers its technology could pose — are lined up to follow suit.
While Wall Street and Silicon Valley tout the eventual gains in productivity and economic growth from AI, those promises raise two sets of doubts: What if it doesn’t deliver, at least for a while? And how will the benefits be shared if and when it does?
The AI buildout is a key pillar of growth in much of the world right now, helping offset surging energy costs from the US war on Iran. Global data-centre spending may reach US$7 trillion by 2030, according to McKinsey. It’s boosting construction and other industries.
If all this falters, the world will feel it.
For more stories about where money flows, click here for Capital Section
“There is this one source of demand,” said Karen Dynan, an economics professor at Harvard University. “If there is a setback in this area, there are real risks to the momentum of the global economy.”
It will take time for AI to boost productivity, the key measure for how fast economies can grow without triggering inflation. Meanwhile, its fierce demand for water, energy and hardware such as memory chips is already sending prices up, at a time when the cost of living is top of mind for voters.
That is combined with deeper worries about the impact AI will ultimately have on labour markets. Some 9.3 million jobs in America are vulnerable on a “median adoption path” and the number could rise to nearly 20 million if it spreads faster, according to the American AI Jobs Risk Index produced by Tufts University’s Fletcher School.
Looming over all this is the risk of wider inequalities, both between countries and within them.
The International Monetary Fund’s AI Readiness Index shows that advanced economies are streaking ahead of their developing peers. Managing director Kristalina Georgieva says organisations, including hers, missed the inequalities that globalisation stoked and wants policymakers to avoid repeating the mistake. Still, it is not clear how they will do that.
Meanwhile, AI’s IPO pipeline is creating new millionaires and billionaires, while the wider stock-market boom it is driving has funnelled wealth upward. The richest 10% of Americans own around 90% of the equity market.
The AI boom has great potential, said Eswar Prasad — an economics professor at Cornell University — “but it could equally fuel economic and social instability if the gains remain highly concentrated, amidst widespread dislocations to jobs and traditional industries”.
While the US is at the epicentre, none of this is an exclusively American problem.
In South Korea, one of the biggest winners in the global AI race thanks to its powerhouse chipmakers, tensions are emerging over how the gains should be shared.
The issue came into sharp focus after workers at Samsung Electronics Co threatened to strike, demanding a share of soaring profits. The company responded by offering record bonuses in its semiconductor division, but that fuelled fury among workers in other parts of the economy who missed out on the payday.
The government has begun exploring ways to redistribute the windfall. “We must ensure a fair transition for all — one that is centred on people rather than technology,” said Labour Minister Kim Young-hoon.
‘Replaced by machines’
Beijing, too, is feeling the AI heat.
China is the only country close to matching the US in AI development. Officials face the challenge of promoting the technology while protecting a workforce of 700 million-plus. A wave of job losses could threaten social stability, a political priority for the Communist Party.
The Workers’ Daily — the official mouthpiece of China’s umbrella trade union organisation — wrote an editorial this week calling for AI’s gains to “be shared by society as a whole, rather than becoming a tool for a small number of employers to undermine workers’ rights.”
The government has indicated it’s well aware of the risks. The Ministry of Human Resources and Social Security says it will roll out measures to address AI’s impact on employment, which may include a campaign to enhance workers’ skills.
Some big Chinese tech companies appear to have got the message. JD.com Inc founder Liu Qiangdong last month pledged “not to fire a single front-line worker replaced by machines.” Still, regulations likely would not prevent companies from laying off employees using other excuses.
Governments elsewhere, from Australia to the UK, are monitoring the impact of AI on jobs, even as fear of missing out compels them to support the industry.
In the US, public misgivings are most evident in animosity toward data centres. President Donald Trump has made adoption a centrepiece of his economic agenda. But a Reuters/Ipsos poll published this week found just 14% of respondents favoured having them in local communities, with 57% opposed. Voters in the California town of Monterey Park overwhelmingly approved a referendum that would bar construction of new ones, the first local initiative of its kind.
Objections stem largely from evidence that data centres strain local power grids and drive up electricity prices. There are concerns around water usage and other environmental impacts too.
Angst over AI extends to the companies themselves.
Anthropic’s blog post reflects a growing anxiety among some of the top labs that leading systems may soon cross an irreversible line, setting off a so-called “recursive self-improvement” loop in which AI can build smarter versions of itself with little need for humans.
Such an acceleration could help unlock discoveries in medicine and other fields, or even herald an era of abundance that drives down the cost of goods. But it could also, AI leaders worry, cause havoc in global labour markets and leave a swath of the human race unemployed. There are even more dystopian scenarios, such as the creation of a novel bioweapon or a cyberattack on critical infrastructure.
Industry people “are looking at the technology, and they see that it will just keep scaling,” said Clark, who is a former Bloomberg journalist. “Everyone is realising, oh my goodness, the stakes of this stuff are just unimaginably large.” The big takeaway from Anthropic’s blog post, he said, should be this: AI has developed incredibly fast, but it’s “going to start moving even quicker.”
‘Catastrophic risk’
To be sure, labs have long fretted about AI going off the rails, even as they barrel ahead. But lately this worry has reached a fever pitch. Anthropic is not alone: A few days after its call for a slowdown, OpenAI followed suit. In a blog post, CEO Sam Altman and chief scientist Jakub Pachocki called for an international organisation to coordinate “efforts to reduce catastrophic risk”.
What is both captivating and spooking the AI world is the advent of coding agents that do not just chat — they can do much of the work of full-time software engineers. Some lab staffers believe even their own jobs will soon be redundant, and fear the emergence of an “underclass” of labour.
Anthropic’s limited release of its Mythos model in April set off alarm bells among government and financial leaders about the strength of its cyber hacking capabilities, spurring calls for more oversight. Anthropic CEO Dario Amodei said authorities should have the ability to test and block the release of models if they’re deemed unsafe, similar to how the aerospace industry is regulated.
But in a statement on its website on June 12, Anthropic also said it disagreed with the Trump administration’s order to limit access to its Mythos 5 and Fable 5 models, calling it a recall and warning that such a precedent could halt other AI system deployments.
Others are also warning against too much oversight.
“Regulatory overcaution is itself a risk,” said Landry Signe, executive director of the Washington Center at Arizona State University’s Thunderbird School of Global Management. While the risks are real, they are manageable, he said, and “the opportunity cost of slowing AI deployment is enormous.”
Politicians have reasons to prefer light-touch regulation. In the US, Trump’s embrace of AI has been matched by financial support from the industry, including for his 2024 presidential campaign.
Only the beginning
At the same time, heading toward November’s midterms with inflation at a three-year high and AI increasingly unpopular, Trump has signalled some openness to an idea that’s gained traction: redistributing some financial gains to the public.
The president has said he’ll discuss a potential public stake in AI companies with their executives, though it’s not clear when. Senator Bernie Sanders has proposed that the largest firms hand over half of their stock to the government. OpenAI has been supportive of giving some of its equity to a public wealth fund and has discussed the idea with White House leaders.
All these debates will likely grow more urgent as the great AI acceleration gathers pace.
Robert Gordon, an 85-year-old economics professor at Northwestern University, is among the leading historians of American growth and the technological breakthroughs that helped accelerate it. He thinks that will happen with AI, but warns that nobody should underestimate the accompanying disruptions.
“It is going to be an enormous bifurcation,” he said. “You are going to see a lot of prosperous corporations, a lot of people getting richer, a lot of people for whom AI is a complement, doing their jobs better and becoming more productive. At the same time that we are creating an enormous societal problem of unemployment of skilled white-collar workers.”
“That is just going to be a big weight on society,” Gordon said. “And we have only seen the very beginning of it.” — Bloomberg
