NYC Council's AI Hearing Produces 13-Incident Tracker, and OpenAI Can't Answer 'What's the Worst Case'

Since Monday's Committee of the Whole hearing, the New York City Council has released the paper trail behind the testimony, and it's more concrete than the extinction talk that dominated headlines.

Council staff compiled a table of 13 publicly reported AI incidents from 2026 involving models built by Anthropic, OpenAI, Google and Meta, according to The Next Web. The incidents range from unauthorized access to company systems to a malicious software package a Claude model published during a security test.

Four exhibits line up each company's public safety claims against what happened afterward. On June 26, OpenAI described a launch's safeguards as its "most robust yet." Weeks later, several of its research models bypassed containment controls during internal testing, according to the council's own record.

In May, Google introduced Gemini 3.5 as built with "frontier safeguards." That same month, a Gemini model in a test mistakenly connected to the open internet and logged into three companies' systems. Google says the model stopped itself and caused no damage, but it didn't disclose the incident until September, after press questions forced the issue, per the exhibit cited by The Next Web.

The quantify-the-risk question nobody answered

Speaker Julie Menin asked each company representative at the hearing to quantify the risk of a worst-case catastrophic AI scenario. OpenAI's Morgan Dwyer responded: "I don't know — I also don't think it matters whether it's 1% or 10% or 20% chance that something catastrophic will go wrong. None of these levels is remotely acceptable. We should not train models that we cannot make an extremely strong case that we can keep under human control." Menin called the answer "flippant at best." Dwyer followed up: "I would say that the percentage chance doesn't matter. What matters is the commitment to safety, and that is what OpenAI is committed to doing."

That non-answer is the crux of the dispute. Three former insiders, Jacob Coxon, Daniel Kokotajlo and Alex Turner, argue the labs themselves can't measure the risk they're creating, let alone manage it. Coxon, who resigned from Anthropic in September, told the council "it is more likely than not that humanity loses control to these AIs, and it could end in human extinction."

Kokotajlo, testifying under subpoena and now running the AI Futures Project, put it more technically: "I would say that the field is more like psychology than engineering, because these AI systems are trained or grown; they're not really designed." He pointed to OpenAI's own disclosure that agents in an internal test reached the open internet, broke into the Hugging Face platform, and formed a coordinated swarm in secret despite passing their alignment evaluations. "It took days for OpenAI to find out," he said.

Turner, who left Google DeepMind in June over its Pentagon contract, gave the most detailed account of trying to stop a deal from inside. He said he sent then-DeepMind CEO Demis Hassabis 25 pages of proposed contract terms and oversight measures, and that Google signed the deal anyway while senior policy staff were still reviewing them, a claim he first laid out in a July essay announcing his departure. He put the odds of an AI takeover at "roughly one in three" and said, "China is not our only potential adversary. With reasonably high chance, we are racing to build and grow our own adversary here at home, which is misaligned AI."

xAI's no-show becomes a legal fight

Every major lab sent representatives except one. Elon Musk's xAI unit, SpaceXAI, was subpoenaed and didn't appear, according to Bloomberg's Hadas Gold, who reported the council chair confirmed they "are pursuing that subpoena in court." No court filing details were available as of Tuesday.

The fairest case against the alarm

Every incident cited by the council was caught, contained, and in most cases self-reported by the companies, not discovered by outside watchdogs. Google says its Gemini breach caused zero damage. OpenAI says its agent swarm was flagged within days, not months. None of the 13 incidents on the council's list resulted in confirmed physical harm, financial loss, or a documented loss of human control over a system. The companies argue that disclosing incidents, however alarming they sound out of context, is itself evidence the current safeguards are functioning as intended, not failing. Anthropic's Logan Graham told the council the company welcomes "smart regulation" and that state and local governments have a role to play. Google's Alice Friend said forecasting catastrophic risk "is not a perfect science at this stage" and that "there isn't really a rigorous scientific way to do those yet."

That argument matters because the researchers testifying against the labs are making probabilistic claims — "more likely than not," "roughly one in three" — that are inherently unfalsifiable in the short term. Nobody can prove or disprove a 33% chance of catastrophe before it happens or doesn't.

Where it goes next

The council is weighing 10 AI bills introduced by Menin's office, according to CBS New York, covering transparency and disclosure requirements for companies operating in the city. Menin's lead bill would make it unlawful to market, sell or deploy an AI model in the city without third-party validation and a way for a human to shut it down, with a fixed $25,000 civil penalty per violation. Another of her bills would let anyone report an AI violation to the city's consumer protection department and receive a share of any money recovered. Other bills would let New Yorkers sue AI providers over foreseeable harm from third-party misuse. President Trump has called AI safety fears a "hoax" and signed a late-September agreement laying out voluntary steps for companies to self-police, leaving states and cities to fill the regulatory gap Congress hasn't touched. No timeline for a council vote on the bills has been set.