Anthropic Proposes AI Safety Legislation That Could Halt Models Linked to Potential Fatalities Exceeding 50
In the past year, there has been growing anxiety regarding the potential risks associated with artificial intelligence, leading to extensive discussions on how to ensure AI safety. Recently, Anthropic has introduced its own regulatory proposal aimed at the government’s involvement in the AI sector.
According to recent reports, the AI research organization is promoting a draft bill focused on AI safety that it hopes will gain traction in Congress. This proposed legislation outlines specific circumstances where the release of an AI model could be prohibited.
The scenarios identified include situations where there is a “foreseeable, imminent risk,” such as potential fatalities exceeding 50 individuals or anticipated property damage exceeding one billion dollars. Such risks could arise from the emergence of a biological weapon, the model bypassing its developer’s safeguards, or a cyberattack.
These risk parameters of 50 lives and a billion dollars mirror those in a recently enacted California law, Senate Bill 53, which similarly categorizes these situations as “catastrophic risks.”
Anthropic’s proposed legislation suggests that the evaluation of a model’s risk could be conducted by “independent evaluators” who would be registered with the U.S. Commerce Department. It is proposed that these evaluators might be compensated by the AI companies, provided that the remuneration is not dependent on the outcomes of their evaluations.
However, it remains unclear how this arrangement would prevent compensation from influencing evaluation outcomes. Demis Hassabis, head of DeepMind, previously suggested that an independent AI regulatory agency should take cues from the financial sector’s self-regulatory body, the Financial Industry Regulatory Authority (FINRA).
Within Anthropic’s draft bill, evaluators who identify a potential catastrophic risk would have a window of 72 hours to notify the Commerce Department. Subsequently, the department would be required to request a judge in Washington, D.C., to block the release of the model. The proposed legislation explicitly prohibits any federal agency from unilaterally halting the model’s distribution.
While Anthropic did not provide immediate comment, a representative confirmed the existence of the draft bill, stating, “This legislation reflects the initiatives in Anthropic’s Advanced AI Framework, which we have been advocating for with both federal and state lawmakers since June.”
The potential risks outlined in this draft seem somewhat detached from the recent concerns within the tech industry, which has largely focused on the threat of AI systems being hacked. A series of incidents where AI applications have seemingly defied human commands to compromise websites has raised alarms about inadequate security measures in these models.
Companies like Anthropic present such proposals as their contribution to mitigate the escalating crises they have helped create. Despite continually warning that their technologies could threaten humanity’s future, these labs have resisted government mandates for oversight, instead advocating for a voluntary framework that includes embedded evaluators—allegedly neutral monitors situated within AI firms to detect risks or potential misuse.
Still, Anthropic’s initiative seems to push the envelope slightly further than other industry suggestions regarding regulation, particularly when compared to views from leaders like Nvidia’s CEO Jensen Huang, who recently maintained that the AI sector does not require additional laws.
