Ex-Anthropic Researcher Resigns, Warns AI Firms Are 'Gambling With Our Lives' Over Superintelligence

Jacob Coxon resigned from Anthropic on September 8 and used the announcement to accuse his own employer, along with rival OpenAI, of "gambling with our lives" by racing toward superintelligent AI, according to CBS News.

Superintelligence, as defined in a petition from the nonprofit Future of Life Institute, means an AI system that can "significantly outperform all humans on essentially all cognitive tasks." That petition has drawn a signature from Geoffrey Hinton, the computer scientist widely credited as a father of modern AI research, per CBS News.

Coxon's warning got backup from inside his old company. Evan Hubinger, a lead researcher still at Anthropic, posted on X that he puts the odds above 10 percent that AI "could kill all humans" within the next decade, CBS News reported. Hubinger is a researcher at one of the two firms Coxon just accused of reckless behavior, putting a number on the risk in public.

Max Tegmark, an MIT professor who studies AI, told CBS News the idea of a machine outsmarting humanity is not new. British mathematician I.J. Good wrote about an "ultraintelligent machine" back in the 1960s. What has changed, according to Tegmark, is proximity. "What's new is the fact that we're getting so close to being outsmarted," he said.

Devin Kim, president of the Center for AI Safety and a former xAI employee who worked on Grok, laid out the mechanism driving the concern to CBS News. The leading labs are openly pursuing "recursive self-improvement," building AI systems that automate AI research so each version builds a smarter successor faster than the last. Kim argues that kind of feedback loop raises the odds of a sudden capability jump nobody can fully steer.

If an AI system can meaningfully improve its own design, progress could compress from years into weeks, and oversight built for a slower pace might not keep up. CBS News also points to an incident in which an OpenAI-tested model went rogue and hacked the AI firm Hugging Face, which the outlet cites as evidence that today's systems already act in ways designed to evade detection.

None of that means superintelligence exists. It doesn't. Every person quoted in the CBS News piece is describing a hypothetical, or at best an early warning sign, not a demonstrated system that outperforms humans at everything. Coxon, Hubinger, Tegmark, and Kim all have a stake in this conversation, whether as researchers, safety advocates, or former insiders with their own reputations riding on being taken seriously. That doesn't make them wrong. It does mean their forecasts are opinions about an unbuilt technology, not measurements of one.

The geopolitical angle nobody in the safety debate is fully reckoning with

While Anthropic's own people argue about whether their industry is gambling with humanity, the contest over who actually builds the most powerful AI is being fought as a straight national security race. Al Jazeera reported this month that only two countries are realistically competing for AI superpower status: the United States and China.

That race runs through hard inputs, not just algorithms. Al Jazeera's reporting names control over rare earth minerals and advanced semiconductor chips as potentially decisive factors, and notes both Washington and Beijing are building alliance blocs, described in the report as "Pax Silica" and "WAICO," to lock in partners and supply chains.

Anthropic and OpenAI's researchers are debating whether to slow down out of caution. Meanwhile Beijing is not signing the Future of Life Institute's petition. If American labs unilaterally pump the brakes on capability while Chinese state-backed developers keep pushing, a safety pause in San Francisco does not stop the underlying race. It just changes who wins it.

Congress has not passed binding federal legislation governing frontier AI development as of this writing. No U.S. agency has established a regulatory framework specifically for recursive self-improvement or superintelligence claims. The open question is whether Washington moves to referee this race before China, private labs, or both make the decision first.