dailynewsnblog
Anthropic (ANTH.PVT) CEO Dario Amodei rocked the AI world over the weekend after he posted a lengthy essay calling on the industry to slow the pace of development of the most advanced AI models.
Rival OpenAI (OPAI.PVT) CEO Sam Altman followed up on Amodei’s piece, which he published Saturday morning, via a post on X saying that he agrees with Anthropic’s chief executive.
The announcements come after two AI researchers posted about the dangers of advanced frontier AI models.
On Tuesday, Jacob Coxon announced that he resigned from Anthropic saying that neither the company nor OpenAI are acting responsibly when it comes to developing powerful AI systems. In a post on X, Coxon stated that the companies are racing to create “self-improving superintelligence.”
“The people building AI earnestly believe that it could kill us all by the end of the decade. This is not a marketing stunt,” he wrote. “If anything, many executives and senior researchers will couch their phrasing in the press to sound sensible — but I hear the same people express fear privately. No other human activity poses this level of danger.”
Anthropic’s Alignment Science Lead, Evan Hubinger, then replied to Coxon’s post saying the former Anthropic worker is correct and that he believes there is a greater-than-10 % chance that the technology could “kill all humans.”
The posts set off alarm bells from Silicon Valley to Washington, DC over the potential dangers posed by the technology.
But President Trump on Sunday pushed back on the safety concerns, saying that “We’re leading China in AI. We’re the most sophisticated country in the world, and frankly, I want to keep it that way because whoever wins AI wins,” Reuters reported.
While he said he is open to guardrails on AI, he also said that “very negative forces” are “bringing up things that won’t happen.”
The AI warnings come after a summer of revelations from the industry in which Anthropic, OpenAI, and Meta (META) reported that their respective models attempted to or successfully hacked into outside organizations’ systems.
AI researchers and CEOs have repeatedly expressed concerns that the technology could cause societal upheaval, including major job losses.
The latest fears come amid talk of recursive self-improvement, or AI that is so advanced it can develop better AI models on its own.
In his essay, Amodei says AI using recursive self-improvement, “could outrun our ability to understand and control these systems, and so must be pursued very carefully, if at all.”
