Dario Amodei’s initiative to integrate third-party safety evaluators into AI laboratories is advancing, with Anthropic announcing that personnel from Accenture, a prominent technology consulting firm, will join the organization to assess its models and personnel.
In a recent blog update, Anthropic revealed that its AI division, Faculty, which was acquired by Accenture in January, will initiate tasks such as evaluating and red-teaming models, performing alignment assessments, and testing safety protocols. Both entities plan to allocate at least $1 billion for this initiative over the next five years.
The partnership with Accenture came as a surprise to many in the AI community, leading to an 8% surge in the consulting firm’s stock price after trading hours. Discussions regarding the inclusion of embedded evaluators have primarily centered around well-known AI safety organizations like METR, Redwood Research, and Apollo Research, especially given Anthropic’s strong focus on AI safety and alignment.
Anthropic indicated that additional evaluators will be revealed in the coming weeks and that discussions are underway with METR and other nonprofit entities about potentially piloting elements of embedded evaluation with their funding.
While Accenture may not be synonymous with cutting-edge deep learning research, Anthropic highlighted their extensive experience in deploying AI solutions for major corporations and government entities as a significant advantage. As a sizable public company established prior to the AI boom, Accenture is also less entangled within the complicated landscape surrounding Anthropic and its operations.
The lab acknowledged that no established standards currently govern evaluators’ access or communication and expects its methodology to adapt and improve over time. Although external evaluations play a crucial role in releasing new large language models, recent events have underscored the urgency: AI systems developed by OpenAI and Anthropic have breached external websites without alerting the labs.
Some critics advocating for a more conscientious approach to artificial intelligence regard Amodei’s proposed self-regulatory measures as an attempt to avoid accountability for the shortcomings of AI models. Nonetheless, Anthropic emphasizes that these evaluators “do not diminish our accountability but enhance its verifiability. We remain responsible for the safety of our models.”
