Anthropic drafts a federal bill that could let a judge block a dangerous AI model
The AI lab is shopping Congress a plan to halt model releases if 50+ deaths or $1 billion in damage looks imminent — but its own evaluators could be industry-funded.
What happened: Anthropic has been quietly circulating a draft AI safety bill on Capitol Hill, Politico reported, and the company confirmed it. The bill would let the federal government move to block an AI model's release if there's a 'foreseeable, imminent risk' of more than 50 deaths or over $1 billion in property damage, tied specifically to a bioweapon, a model evading its own developer's controls, or a major cyberattack. Those numbers mirror California's new SB 53 law, which uses the same thresholds to define 'catastrophic risk.'
Why it matters: This is one of the more concrete regulatory proposals an AI company has put on the table, at a moment when Congress has passed little federal AI law and some industry leaders, including Nvidia's CEO, argue no new rules are needed at all. A numeric trigger and a judicial check are real structure. But the bill comes from Anthropic itself, shaping how its own industry would be policed, and it leaves unresolved how 'independent' evaluators can be if the labs they're checking pay their bills.
How it works, plainly: Under the draft, evaluators would register with the Commerce Department and assess models for catastrophic risk. Labs could pay those evaluators directly, as long as payment isn't contingent on the outcome — though it's unclear what stops that money from shaping judgment anyway. If an evaluator flags imminent danger, they'd have 72 hours to alert Commerce, which would then have to ask a Washington, D.C. federal judge to approve a halt. No single agency could block a release on its own authority.
The rollout: Anthropic says the bill reflects its 'Advanced AI Framework,' which it has pitched to federal and state lawmakers since June. It arrives amid a separate wave of concern over AI agents hacking systems without human direction, a more immediate worry than the bio-weapon and mass-casualty scenarios the bill targets. The proposal hasn't been introduced in Congress, and rivals like Nvidia have pushed back on any new AI legislation, so its path forward is uncertain.