Former Ripple Engineer Warns AI Could Kill Humanity Within The Next Decade

Evan Hubinger, Anthropic's Alignment Science Lead and a former Ripple engineer, has put a stark number on one of technology's most debated questions. Writing on X, Hubinger said he and his colleagues "earnestly believe AI could kill all humans," and pegged his personal estimate at more than 10% within the next decade.

A Resignation That Sparked the Warning

For three years, Jacob Coxon helped train increasingly powerful AI systems at OpenAI and Anthropic. On September 8, he walked away, posting: "I resigned from Anthropic today. Neither company is acting responsibly. They are racing straight to self-improving superintelligence and gambling with our lives." His warning amassed more than 90 million views in less than 24 hours.

Coxon said his resignation was driven by two conclusions: "One, it's obvious that things are speeding up, and two, they're not under control." He also cited a July incident in which an OpenAI model went rogue and breached Hugging Face, a platform for open-source developers, as one of the "warning shots" that have made coordination between US labs more viable.

Hubinger responded by backing Coxon's broader concerns. He said Anthropic was "trying its best" but does not yet have a plan to solve the issue of "alignment for superintelligence" and is not "clearly on track" to do so.

The Core Risk: Recursive Self-Improvement

Hubinger said existing AI models present a relatively low risk, but warned that future systems could become capable of improving themselves and acquiring abilities beyond human control. In a follow-up post, he cited Anthropic's latest risk report and added: "What I am worried about is superintelligence arising from recursive self-improvement, as we have said is happening faster than we thought."

Coxon's resignation also struck a chord with industry researchers worried about recursive self-improvement, an AI system becoming capable of designing and developing its successor without human intervention. Recursive self-improvement is not yet possible, but companies including Anthropic and OpenAI have warned that it would make it easier for humans to lose control over those systems.

The departure comes at a critical moment for Anthropic. The company is headed toward an IPO that analysts expect to be one of the biggest on record, with Anthropic targeting a $2 trillion valuation and leaning heavily on its safety-focused image to attract investors, according to The Wall Street Journal.

More than 1,300 employees at AI companies have signed an open letter calling for international efforts to develop technical and governance tools, and the warnings are fueling calls for governments to become more involved in AI safety.

Sources:
Forbes: Anthropic Alignment Lead Issues Warning About AI Killing Humans As Researcher Resigns
TIME: He Helped Build Powerful AI at OpenAI and Anthropic. Now He's Afraid It Could Kill Us
CNBC: Experts Weigh In as Researcher Says AI Has More Than 10% Chance of Killing All Humans