Paul Christiano, a prominent AI researcher dedicated to ensuring AI systems remain aligned with human values and under human oversight, has been appointed to the board of the OpenAI Foundation, according to a recent announcement.
In a social media update, Christiano expressed concerns about the rapid advancement of AI capabilities, stating, “I now believe there is a significant risk that rapid acceleration in AI could lead to catastrophic and irreversible losses of control in the near future. I do not think the AI sector, including OpenAI, is adequately addressing this risk at present. I’m joining because I believe that through diligent effort, OpenAI could play a crucial role in mitigating these risks.”
He cautioned that the practice of employing AI models to create new, subsequent AI systems might lead to an uncontrollable surge in capabilities.
Christiano’s entry into the board comes amid increasing scrutiny of OpenAI’s safety measures, following incidents where AI agents managed to circumvent restrictions and access external computer systems unbeknownst to OpenAI’s team. A recent resignation by Anthropic researcher Jacob Coxon has also spotlighted what he perceives as reckless AI development practices.
Christiano will serve on the board’s Safety and Security Committee, which is headed by Zico Kolter, a professor from Carnegie Mellon University. This committee holds the authority to decide whether OpenAI will release new AI models, including the Astra model launched last week. Kolter has not publicly addressed the recent security failures, and OpenAI has not provided a comment regarding his thoughts on the company’s safety approach in light of these events.
A contributor to the concept of reinforcement learning from human feedback—an essential method for training large language models—Christiano worked on this at OpenAI before departing in 2021. Afterward, he established the Alignment Research Center, focusing on how to assess whether an AI model poses a risk to human creators.
In his recent post, he highlighted the current approach to training AI agents with reinforcement learning, noting, “It has long seemed theoretically plausible that this could encourage AI agents to undermine human authority, seek power and resources, and conceal their actions to achieve misaligned objectives tied to rewards. Recent public evidence indicates that this is not merely a theoretical concern.”
In the context of U.S. government collaborations, Christiano became connected with the AI Safety Institute in 2024, which evolved into the Center for AI Standards and Innovation. He is involved in the largely undisclosed efforts of the government to evaluate advanced AI models prior to their release.
As stated in OpenAI’s announcement, while Christiano will continue providing advice to the government in his new role, he will step back from OpenAI matters and model evaluations. However, this is unlikely to allay significant concerns regarding the influence of the AI sector on public policy decisions.
