The former artificial intelligence researcher who resigned from his role at Anthropic and subsequently called for enhanced safety guardrails will testify alongside the leading AI lab’s safety representatives at a New York City council meeting on Monday.
Jacob Coxon, 27, did pretraining research at OpenAI and Anthropic for three years before abruptly and publicly leaving the private sector, sparking a nationwide debate about the importance of “pacing the frontier” and slowing AI development. Coxon’s comments were augmented by various supposed hacking incidents, the OpenAI-Hugging Face incident being the most prominent, and the public’s broadly negative view of the industry.
“Neither company is acting responsibly,” Coxon said in the Sep. 8 X post announcing his resignation. “They are racing straight to self-improving superintelligence and gambling with our lives.”
Coxon is appearing at the request of Council Speaker Julie Menin, who sought to bring together AI whistleblowers and industry representatives as city officials weigh bills aimed at establishing safeguards, according to Bloomberg. Safety and policy researchers from Google, OpenAI, Anthropic, and Meta will be in attendance. The council reportedly subpoenaed SpaceX’s AI team after it failed to respond to a request to appear at the meeting.
“The people building AI earnestly believe that it could kill us all by the end of the decade,” Coxon also said in his announcement. “This is not a marketing stunt. If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.” Coxon largely framed his concerns during media appearances around fears that systems that could “hack anything” will one day exist.
ANTHROPIC RESEARCHER QUITS OVER EXISTENTIAL FEAR OF AI: ‘GAMBLING WITH OUR LIVES’
Much of Menin’s motivation for holding the hearing stems from AI companies’ prominence in New York City, though most AI policy debates exist at the federal and state levels. A bipartisan backlash against data center construction largely spearheaded by state governors faded from public view once Coxon made the announcement in early September. Anthropic CEO Dario Amodei subsequently published an essay titled “We Must Pace The Frontier,” outlining his perspective on how to manage AI’s risks. Amodei joined President Donald Trump and other AI leaders last week at the White House to sign a voluntary, non-binding commitment to self-regulation.
One proposal before the council is a Menin-sponsored bill that would give whistleblowers who report violations a cut of the recovered fines or penalties. Coxon wasn’t the only Anthropic employee to quit that week in early September, as the then-head of Scalable Operations also left the company.
Continue reading...
[ H/T Washington Examiner ]
