Anthropic and OpenAI propose embedding independent safety evaluators inside their AI labs

Details

Anthropic and OpenAI have proposed embedding independent safety evaluators within their AI research laboratories. The plan would give external experts direct access to development processes, models and practices inside the companies' labs.

Researchers responded positively to the idea of providing access that has so far been rare. They view it as an opportunity for more detailed and immediate assessment of risks posed by advanced AI systems.

At the same time, experts stress that the presence of external evaluators alone is not sufficient. For oversight to be meaningful, they say, there must be transparent procedures, ensured independence of the evaluators, and clear evaluation standards.

Critics also emphasize the need for regulation over the longer term. They argue that internal measures and voluntary initiatives may have limited impact unless supported by external rules and oversight frameworks.

The proposal from Anthropic and OpenAI has sparked debate about balancing access for external experts with the real ability of those experts to exercise independent scrutiny. Commentators highlight transparency, independence and regulation as the three essential prerequisites for effective oversight.

It remains unclear what specific mechanisms or criteria for independence will be implemented and whether standardized rules for the evaluators' work will be established. Available information so far is limited to the general proposal and researchers' responses.