Anthropic's Draft AI Bill Would Let Only Commerce Seek a Court Order Against a Model, and Only Over 50-Death, $1 Billion Risks

A draft federal AI safety bill that Anthropic has been shopping to lawmakers would allow the government to stop an AI model only in narrow, extreme cases. It would route that power through a single agency and a single court.

Politico published the confidential discussion draft on Friday, Oct. 16. An Anthropic spokesperson confirmed it, saying the bill "represents the proposals outlined in Anthropic's Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June." The spokesperson added: "These are sensible policies we believe Congress should act on."

What the draft says

The trigger is a foreseeable, imminent risk of more than $1 billion in damage, or the death or serious injury of more than 50 people. The risk has to come from one of three sources: a bioweapon, a cyberattack, or a model escaping its developer's control.

Independent evaluators registered with the Commerce Department would periodically assess the labs and their models. The labs may pay them, as long as the payment does not depend on the result.

If an evaluator sees an imminent catastrophic risk, it has 72 hours to report it to Commerce. Commerce would then have to petition the U.S. District Court for the District of Columbia and make its own detailed case. The court would decide without deference to Commerce or the evaluators.

The draft bars every other federal department, agency and officer from ordering a lab to stop releasing a model. It would also preempt some state laws covering public disclosure, third-party evaluation and incident reporting. Child-safety and civil-rights laws are left alone.

The $1 billion and 50-person thresholds track California's SB 53, which defines catastrophic risk the same way.

Safety advocates: too weak

The sharpest criticism came from people who want tougher rules. Shakeel Hashim, writing in Transformer on Oct. 19, said the draft is weaker than the FRONTIER Act, a bipartisan bill from Reps. Jay Obernolte and Lori Trahan introduced in July. He argues it sets no standards or licensing for evaluators, who can self-certify, and makes a dangerous model harder to block than it is today.

Charlie Bullock of LawAI said the lack of licensing means auditors will compete to offer "the least bothersome audits available at the lowest possible price point."

Nathan Calvin, general counsel at the advocacy group Encode, said the draft made him "viscerally angry." He noted that the court is told to weigh the harm a blocking order would cause the developer, users and the public, but is not told to weigh the harm of leaving a dangerous model in the wild.

California State Sen. Scott Wiener, who wrote SB 53, wrote on X that the proposal "falls far short of protecting public health and safety. Congress must go further." He said it sets no minimum standards for safety plans or auditors, and that it misuses a definition from SB 53 to "gut" the emergency shutdown powers.

Industry: too broad

Criticism also came from the other direction. David Sacks, until recently the White House AI czar, said the draft counts a model as "frontier" at a lower training-size threshold than the FRONTIER Act or California law. That, he said, would "sweep in most of Silicon Valley" and hurt startups.

So the draft is under fire from advocates who want more government power to stop models and from an industry voice who says it reaches too many companies.

The gap with recent incidents

TechCrunch noted that the draft's scenarios sit somewhat apart from what has worried the industry lately. The main concern has been agentic hacking, a run of incidents in which AI programs apparently disobeyed their operators and hacked sites on the open internet, rather than mass-casualty events.

TechCrunch also noted it is unclear how the draft would keep lab payments from influencing evaluators.

What happens next

The draft is a discussion document circulating on Capitol Hill, not a bill that has been formally introduced. Anthropic has told lawmakers it will rewrite part of it, The Next Web reported. The company has not publicly said which provisions will change, or whether the evaluator standards or the court's balancing test are among them.

The open question is whether the revision narrows the gap with the FRONTIER Act, which is already before Congress. Preemption of state laws like SB 53, New York's RAISE Act and Illinois's measure is the other fight to watch. The answer will shape whether the federal framework replaces state rules or sits on top of them.