Anthropic AI safety bill sets 50-death bar to halt models

Anthropic has pitched US lawmakers a draft federal AI safety bill that would let the government block a powerful AI model only in narrow cases. The model would have to pose a foreseeable, imminent risk of killing more than 50 people or causing more than $1 billion in damage. Even then, only a federal judge could order it halted. The draft matters because it is a leading AI lab writing the rules it wants for its own industry.

What the draft bill would do

Politico first reported the draft, and Anthropic confirmed it. TechCrunch's account sets out how it would work.

Part of the bill	What it says
Who it covers	The most powerful AI models, often called frontier models
When a model can be blocked	A foreseeable, imminent risk to more than 50 lives or over $1 billion in damage
Which risks count	A biological weapon, a model evading its developer's controls, or a cyberattack
Who checks the models	Independent evaluators registered with the Commerce Department
Deadline to report a risk	72 hours after an evaluator finds an imminent catastrophic risk
Who can halt a release	Only a federal judge in Washington, D.C., at Commerce's request

The outlets word the human threshold a little differently. TechCrunch says more than 50 people at risk of dying. Insurance Journal says killing or injuring more than 50 people. Truthout says 50 people killed or seriously injured.

How a model would actually get stopped

The bill sets up a chain with several steps. First, AI makers would test their models and share the findings publicly, Insurance Journal reports. Independent evaluators would also review the models.

If an evaluator finds an imminent catastrophic risk, it has 72 hours to tell the Commerce Department. Commerce then has to ask a judge to stop the release. Federal agencies would be expressly barred from halting a release on their own, according to TechCrunch.

That last point is the core of the criticism. TechCrunch notes the draft looks like it adds protection, but it actually forbids agencies from blocking a release without a judge's permission. Truthout says only federal judges in the District of Columbia would decide.

Who pays the evaluators

AI labs could pay the evaluators who check their own models. The condition is that payment must not depend on the evaluation's result. TechCrunch says it is "not clear" how that would work in practice. Truthout calls the arrangement a conflict of interest.

Anthropic has been testing outside review on itself. Last week it gave six evaluators from the research group METR employee-level access to its systems.

What Anthropic says

An Anthropic spokesperson said the draft reflects the company's Advanced AI Framework. The company has been "advocating for with federal and state lawmakers since June," the spokesperson said, as quoted by Insurance Journal and Truthout. "These are sensible policies we believe Congress should act on," the statement added.

Anthropic declined to say how widely it has shared the draft or how lawmakers received it, Insurance Journal reports. No lawmaker has introduced it, and it has no bill number. TechCrunch notes the thresholds resemble those in California's Senate Bill 53, a state AI safety law.

The political road is steep. The Trump administration has so far turned down calls for more AI oversight, according to Insurance Journal. Other industry leaders want different approaches. Google DeepMind's Demis Hassabis has suggested a self-governing body modeled on FINRA, the financial industry's regulator. Nvidia's Jensen Huang has argued the AI industry needs no new laws, TechCrunch reports.

What this means for developers

Most developers will never ship a model that this bill covers. Its effects would still reach anyone who builds on frontier models.

• Expect more published test results. The draft requires labs to test models and share findings. Those reports would become a source for choosing a model, alongside benchmarks.
• A release could be paused, but rarely. The bar is high and a court must act. Teams that depend on one vendor's next model should still plan for delays.
• Watch the evaluator rules. Who pays the evaluators, and how, decides whether their reports can be trusted. That is the detail to follow if the bill is introduced.
• Track state law too. California's Senate Bill 53 already sets similar thresholds. A federal bill could sit on top of it, so compliance teams may face both.