The training of artificial intelligence models using other AI models has surged in interest among research institutions. Recently, a researcher affiliated with Anthropic’s fellows program shared preliminary insights into this method.
On Friday, Anthropic released a paper titled “Automated Researchers Can Reliably Mitigate Alignment Failures,” which explores how AI systems can consistently enhance performance across a series of alignment benchmarks. When presented with ten benchmarks that identify specific misalignment issues, the automated systems improved in all cases without compromising overall efficacy.
The study, conducted by Anthropic fellow Chen Yueh-Han, mirrors conventional research methodologies. Each automated system analyzes the existing literature, suggests a training method, and then trains the model for 30 minutes, progressively adjusting the benchmarks over several iterations. Effective methods are retained while ineffective ones are discarded, enabling efficient and scalable operations.
The findings suggest promising potential for automated alignment post-training to be implemented in the near future, according to the paper.
This paper marks a pivotal step towards recursive self-improvement in AI, which many experts consider a vital next phase in AI development. If models can enhance their alignment training independently, it stands to reason they could also improve wider training practices, raising the possibility of making human AI researchers redundant.
Furthermore, the paper does not shy away from addressing this implication, directly comparing the performance of the Automated Alignment Researcher (AAR) to that of human researchers. It claims, “The best AAR method surpasses the proposals of experienced humans, on average, within six hours,” adding that “Human-guided research directions do not yield better performance.”
A cost analysis further bolsters this comparison. The AAR incurs an approximate cost of $4 per hour in API usage, while the expense for human researchers is about $150 per hour.
However, the paper also acknowledges certain limitations of this method. The automated systems are only effective as long as the benchmarks accurately reflect the intended alignment goals, and substantial effort is required to create and uphold these benchmarks, along with maintaining and expanding the relevant literature that the automated systems utilize.
