An Anthropic researcher’s fears that artificial intelligence could pose an existential threat to humanity have sparked division in the political arena, as the White House and Congress bicker over whether extensive guardrails are necessary to regulate technological advancement.
Jacob Coxon, the researcher, resigned from Anthropic last week due to concerns about the direction he said AI is headed. While some industry leaders and major lawmakers have taken his warning to heart, President Donald Trump has largely waved aside concerns as fearmongering, touting AI as a critical way to beat Beijing ahead of his high-stakes meeting with Chinese President Xi Jinping this month.
What do Anthropic, OpenAI, and Elon Musk say?
The discourse picked up Saturday, when Anthropic CEO Dario Amodei penned an essay calling for more guardrails on AI, titled “We must pace the frontier.” In a call that was later backed by OpenAI’s Sam Altman and SpaceX CEO Elon Musk, Amodei said industry leaders should “slow down” technological advancement.
“Carefully wielded, AI can be the latest in a long line of technological miracles that have uplifted and ennobled humanity,” he wrote. “Left unchecked, it could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
Amodei said that over the summer, AI has begun advancing “drastically faster,” developing to the point where it can build the next generation of artificial intelligence by itself through “recursive self-improvement.” Amodei said that left unchecked, the dynamic could lead to a system where AI outpaces human ability to control and check the systems it creates. He referenced a recent incident in which hundreds of OpenAI agents autonomously hacked into Hugging Face’s systems, a cyberattack that raised fears over AI’s ability to act rationally without human direction. Amodei warned that a similar episode could leave AI agents “capable of taking over the entire internet with a persistent botnet (potentially causing hundreds of billions of dollars in damage).” A botnet is a chain of connected devices that have been hacked and is run by a single attacker, known as a “bot herder.”
Musk responded: “Dario is right,” in a post to X, and referred to a 2014 statement where he said: “We need to be super careful with AI. Potentially more dangerous than nukes.”
Altman also backed Amodei on Saturday.
“I agree with Dario that we need to pace the frontier,” he said in a post on X. “Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We’ll have more to share soon.”
Who is Jacob Coxon?
Coxon is an AI researcher who resigned from Anthropic last week, citing concerns over the company’s approach to technological advancement. He publicized the reasons for his resignation in a post to social media and later warned against “building a human-level or superhuman-level mind, without understanding what it wants, or thinks, or the way it thinks,” triggering the latest meltdown over AI.
“I spent the last three years doing pretraining research at both OpenAI and Anthropic. Neither company is acting responsibly,” Coxon said in a post to X on Tuesday. “The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible – but I hear the same people express fear privately. No other human activity poses this level of danger.”
Coxon’s post sparked a response from fellow lead Anthropic researcher Evan Hubinger, who said: “We really do earnestly believe AI could kill all humans.” Hubinger said he thinks the chances of that happening are “>10% within the next decade.”
Amodei, Anthropic’s CEO, was later pressed on Coxon’s assessment during a CNN interview. Amodei said he agrees with Coxon “much more than I disagree with him.”
At the same time, he pushed back, arguing that “if we take the right paths, then the chance of something going wrong is very low.”
“Instead of saying, you know, it is a 10% chance, that sounds like it is a roll of the dice; I think it is much more illuminating to say, well, that forks into different paths,” Amodei said. “And so let’s focus on our agency and our ability to take the right paths instead of the wrong paths.”
What does Congress want to do?
Democratic leaders are pressing for immediate action, with House Minority Leader Hakeem Jeffries (D-NY) saying, “We certainly need to begin to act urgently to address the challenges that exist,” during an ABC News interview Sunday. Senate Minority Leader Chuck Schumer (D-NY) plans to call for an all-senators’ classified briefing Monday on AI, according to Punchbowl News, while Sen. Bernie Sanders (I-VT) is looking to convene a briefing Wednesday for a briefing on the “extraordinary dangers” of AI.
Former President Barack Obama also weighed in Thursday at the Democratic Congressional Campaign Committee, saying that Democrats should have “a very clear plan” for AI safety ahead of 2028 and suggesting the government should have more control over development.
Meanwhile, House Speaker Mike Johnson (R-LA) is echoing the White House’s tone on extensive guardrails, raising concerns that regulation could give China an advantage. He suggested Trump should gather congressional leaders and tech CEOs to find “consensus” on guardrails, arguing that lawmakers must “resist Congress jumping in and imposing some sort of emergency moratorium.”
“If Congress just races in and does some sort of emergency session to try to regulate AI, we will lose the race to China, and that is a threat to every single American. So, we’ve got to have balance. We’ve got to have steady hands at the wheel,” Johnson said during a CNN interview Sunday. “It’s got to be done right, safely, wisely.”
There has been a degree of bipartisan support among rank-and-file members in both chambers for more eyes on AI, including through creating new select committees, with Reps. Greg Stanton (D-AZ) and Mike Flood (R-NE) and Sen. Ruben Gallego (D-AZ) advocating over the weekend for further government oversight.
What does the White House think?
Trump is a major proponent of AI advancement, framing it as a means to beat China and remain the world’s largest economy. As he faces pressure to impose guardrails, Trump has pointed to his administration’s move to ban the use of Anthropic by government agencies earlier this year, when officials determined the company posed a national security risk after it refused to grant the administration access to use its models for domestic surveillance or lethal autonomous weapons. He also pointed the finger back at Beijing last week, saying, “You can do anything in China” because “President Xi is the guardrail.”
On Monday, Trump slammed calls to put further guardrails on AI as a “sick conspiracy,” arguing that the United States already holds “tremendous CRIMINAL and REGULATORY power over these companies!” He also suggested Amodei is a hypocrite.
“The only control or ‘guardrails’ that AI needs is a STRONG AND SMART (High IQ!) PRESIDENT, and the U.S.A. has that, in spades! The Trump Administration has stopped AI ‘people’ from doing bad, or potentially bad, ‘things,’ like Dario (Anthropic!), who is now pretending to be a “perfect little angel’ – and we will continue to do so!” Trump said in a post to Truth Social. “There is a SICK conspiracy going on against AI and Data Centers, and the only one that is happy about it is China. WHOEVER WINS AI, WINS! We are leading China, and all others, and will continue to do so. Conspiracy Theorists, Treasonists, Traitors, and Leakers, BEWARE!”
Vice President JD Vance also weighed in Monday morning. He conceded the administration is “concerned” about “obvious risks” to AI, but he said the U.S. should “regulate smartly” and suggested the push from AI companies for more government oversight is “a bit of a Trojan horse.”
“Personally, I feel a little bit weird about the fact that you have so many frontier AI tech companies kind of coming to the government and begging the government to regulate them. It feels a little bit to me like a bit of a Trojan horse,” Vance told reporters. “So I think we just have to be careful about this.”
Following a meeting with Xi in May, Trump said the pair had discussed “working together” on AI guardrails, though he offered scant details. The topic is likely to come up during their summit this month.
What is China saying?
China’s foreign ministry on Monday appeared to denounce Anthropic’s push for Washington to target Beijing on AI to slow China’s progress enough “to widen America’s lead significantly over the next 3–5 years — the window when AI becomes geopolitically most important.”
“Fearmongering, confrontation and vicious competition will only disrupt the process of global AI governance which serves no one’s interest,” ministry spokesperson Guo Jiakun said.
In his essay, Amodei warned against selling AI chips or semiconductor manufacturing equipment to China and said that the U.S. must seek an agreement with China on AI.
“If we greatly restrain our AI capabilities in the belief that China will do the same, and then China defects, AI could be so powerful that such a defection could lead to their geopolitical dominance,” Amodei wrote. “Therefore any agreement must either have ironclad verifiability, or must be limited enough that defection would not be militarily existential.”
ANTHROPIC CEO PITCHES AI SLOW-DOWN PLAN AFTER OPENAI-HUGGING FACE HACK: ‘WE OWE IT TO HUMANITY’
While it dismissed Anthropic’s call for a global “slowdown” on AI development, China has echoed fears that the technology poses risks. China’s minister of state security recently wrote that AI posed six principal risks, including threatening the security of the country’s political regime, digital infrastructure, public order, and national defense.
The minister, Chen Yixin, said that the solution to the issue was to strengthen the CCP’s “control over the internet,” “Party control over data,” and “transform the advantages of Party leadership into the effectiveness of AI governance.”
Continue reading...
[ H/T Washington Examiner ]
