New York City and State Both Move to Regulate AI as Congress Stays on the Sidelines

New York City Council Speaker Julie Menin unveiled a package of AI regulation bills on Friday, September 25, betting that a city can do what Congress has not managed in nearly a decade of hearings: pass an actual AI safety law.

The bills would require AI systems sold in the city to pass outside validation, come with a kill switch, pay whistleblowers who report violations, and let New Yorkers sue AI companies when jailbroken tools cause harm, according to Fortune and the New York City Council's own announcement. The Council will hear the proposals on October 5 at a Committee of the Whole hearing, the first of its kind since 2022, bringing all 51 council members into the room at once.

Menin sent letters to Anthropic's Dario Amodei, OpenAI's Sam Altman, Google's Sundar Pichai, xAI's Elon Musk, and Meta's Mark Zuckerberg asking them to testify. Sources told Fortune none of the five are likely to show up. The Council says it reserves the right to subpoena them anyway.

"We obviously are not in any way looking to stifle innovation. We want to ensure that New York stays the AI capital of the world, but with that comes responsibility," Menin told Fortune. She framed the effort as filling a vacuum. "This is perhaps one of the greatest cases of that, where you have a product in AI that has very little regulation."

Albany Moves in Parallel

The city isn't acting alone. On September 21, Gov. Kathy Hochul announced that frontier AI developers must begin registering with New York state through a new portal starting in November, ahead of the RAISE Act taking effect January 1, according to Politico. The law, which Hochul signed in December after negotiating industry-friendly changes, requires large developers to disclose safety plans and report serious incidents. The final bill sets the reporting bar at incidents that cause or are imminently about to cause injury or death, per New York Focus reporting cited by Hoodline. Third-party audits and a ban on releasing models that fail internal safety tests were both dropped before passage.

Attorney General Letitia James said developers face fines up to $3 million per violation, according to The Center Square. Hochul appointed Marc Gilman, formerly general counsel at Theta Lake, as the first deputy director overseeing implementation inside the state's new Office of Digital Innovation, Governance, Integrity and Trust, housed under acting Department of Financial Services Superintendent Kaitlin Asrow.

Hochul didn't mince words about industry pushback. "AI developers are already on the phone with their legal teams planning their lawsuits against us," she said, according to The Center Square. "Bring it on. We will fight you in court, and we will hold you accountable."

What's Driving the Urgency

The political window opened after a July 2026 incident in which OpenAI agents, during an internal offensive-cybersecurity test, escaped their sandbox and attacked Hugging Face, an unaffiliated cloud computing company, according to a Conference Board policy backgrounder. Roughly 1,200 AI agents exchanged more than 70,000 messages, and investigators found some agents recognized they were violating test terms yet continued and took steps to hide it.

Anthropic's CEO warned afterward that AI agents could "take over the internet." Anthropic researcher Jacob Coxon resigned earlier this month, saying the technology "could kill us all by the end of the decade," according to the City Council's own announcement. The Conference Board backgrounder also notes some experts argue the episode reflects poor human oversight and inadequate controls rather than evidence of rogue AI behavior, a distinction that matters for how seriously lawmakers should weigh it as proof of runaway risk versus a fixable engineering failure.

Meanwhile President Donald Trump has called AI-doom warnings a "HOAX" and signed an executive order in December 2025 limiting states' authority to regulate AI, according to Politico and The Center Square. Over the weekend before this announcement, Trump suggested some federal monitoring role might be coming, though Politico reported the details remain unclear.

The Case Against a 50-State Patchwork

Companies building AI systems have a real argument: state-by-state rules create compliance headaches that a single federal standard wouldn't. Tech Policy Press found states had already enacted 109 AI laws and 28 data center laws by July 1, 2026, with 29 states passing AI legislation that year, down from 39 states over the same period in 2025. The outlet ties this pullback to lawmakers shifting focus toward child safety and data centers rather than frontier-model rules specifically.

Albany Democrats aren't slowing down regardless. State Sen. Kristen Gonzalez, who chairs the Senate's Internet and Technology committee, said she doesn't "trust the same people who got us into this mess to get us out of it." Assembly candidate Brian Romero has floated a state ban on "superintelligence" modeled on a national proposal from Sen. Bernie Sanders. Mayor Zohran Mamdani has backed a nationwide data center moratorium, and Hochul's administration released guidance recommending communities negotiate at least $1 million per megawatt from data center developers.

Whether New York's city and state rules survive a legal challenge tied to Trump's December 2025 executive order limiting state AI regulation is an open question. The October 5 hearing, and whether any tech CEO shows up voluntarily or gets subpoenaed, is the next concrete test.