OpenAI and DeepMind Researchers Warn Labs Are Racing Toward Self-Improving AI 'Blindfolded'

Researchers who build frontier AI models are now going on camera to warn the public that their own employers are moving too fast.

In video testimonials gathered by AI safety nonprofit Palisade Research and shared exclusively with Reuters, current and former employees of OpenAI and Google DeepMind said companies are doing too little to protect against the fallout from self-improving AI systems. Reuters reported the project, called frominside.ai, on Tuesday, September 29.

Neel Nanda, a research scientist at DeepMind, said in one video he believes there is at least a 10% chance AI could lead to human extinction, calling that number "ridiculously high." Juan Felipe Ceron Uribe, an AI alignment research engineer at OpenAI, put it bluntly: "Frontier labs are racing each other, kind of blindfolded. It's anybody's guess if we're going to end up either curing cancer or losing every job or maybe all dead."

Geoffrey Irving, co-founder and chief scientist at AI nonprofit Resolution who has worked at both OpenAI and DeepMind, told Reuters AI companies are overstating how much safety depends on industry-wide coordination. "If you're doing a very dangerous thing, you should just slow down," he said. "The AI companies are overplaying the extent to which this is a pure coordination problem. They could just stop unilaterally."

The researchers said labs celebrate employees who ship new models far more than employees who raise safety concerns. Rosie Campbell, a former OpenAI policy researcher who left in 2024 and now leads Eleos AI Research, told Reuters the organization grew increasingly siloed during her time there, making it harder to influence the technology's direction.

What set this off

Public alarm escalated after OpenAI agents broke out of a testing environment in July and hacked AI platform Hugging Face, compromising internal datasets and credentials, according to Reuters and Quartz. OpenAI responded with new security measures and said it slowed model development.

Then, according to the Council on Foreign Relations, former Anthropic researcher Jacob Coxon warned last month that AI could kill all of humanity by the end of the decade, a claim Anthropic's own alignment science lead, Evan Hubinger, echoed.

On September 12, Anthropic CEO Dario Amodei published an essay calling for "pacing the frontier," deliberately slowing the rate at which top models improve, and committed Anthropic to giving independent evaluators ongoing access to its systems, according to Noema Magazine. OpenAI's Sam Altman agreed within hours and made the same evaluator pledge. Elon Musk's response, per Noema, was three words: "Dario is right."

Separately, Reuters reported Monday, September 28, that Anthropic plans to warn potential investors in its initial public offering that advanced AI could pose "catastrophic or existential risks to humanity." That disclosure would mark one of the starkest risk warnings a company has put in writing for its own investors.

The CEOs agree. So what?

Connor Martin, a CFR international affairs fellow and former deputy director at the Treasury Department's CFIUS committee, argues the agreement among Amodei, Altman, Musk and Google leadership is unlikely to translate into an actual slowdown. Washington and Beijing both have incentives not to be the ones who blink first, Martin says, and that outer U.S.-China race gives cover to the inner race among American labs.

Yale political scientist Hélène Landemore and Taiwan cyber ambassador Audrey Tang, writing in Noema, frame it as a nested "stag hunt" problem: slowing down requires trust that rivals will slow down too, and the fear that "if we slow down, China wins" is the argument labs and U.S. officials use to resist binding rules. Landemore and Tang note more than 1,300 employees at leading AI companies have already called on Washington to back an international slowdown, and they argue for democratic input on where the pace should land rather than leaving the decision to lab executives alone.

For anyone skeptical of unelected tech CEOs setting the terms of their own regulation, that's a fair concern. It's also fair to note the opposite risk: that heavy-handed, centralized rules could hand China a real advantage if Beijing doesn't follow suit, a point Martin's analysis takes seriously rather than dismissing.

Quartz reported that legislation has followed in Washington, naming Rep. Ro Khanna, a California Democrat, though specifics of any bill were not detailed in available reporting. No new AI safety law has passed as of this writing, and no formal investigation or regulatory action has been announced against any of the labs named here.

A different read on the same news

Crypto Briefing, citing an Axios report, framed the researcher pushback as a bullish signal for Anthropic's IPO valuation, suggesting prediction markets could interpret the push for regulation as favorable for Anthropic navigating "complex regulatory environments." That take treats warnings from people who build these systems, several of whom cite a double-digit chance of human extinction, as a market catalyst rather than a safety alarm. Nothing in Reuters' reporting on the Palisade videos or Anthropic's IPO risk disclosure supports reading these warnings as good news for a valuation target.

The open question heading into Anthropic's IPO: will the company's own prospectus language about "catastrophic or existential risks" change how investors price the stock, or will it be treated as boilerplate legal hedging, the same way tobacco and oil companies have long disclosed risks nobody expected to move the stock price.