Anthropic Gives METR Researchers Employee-Level Access

Anthropic is embedding six researchers from the nonprofit AI evaluator METR inside the company with employee-level access, giving an outside group the deepest independent look yet at a frontier AI lab’s systems and safety practices.

The announcement came Thursday via an Anthropic blog post. The six researchers from METR—short for Model Evaluation and Threat Research—will get company-issued laptops, office space, and internal tools roughly on par with what Anthropic’s own risk assessment teams use. Employees are encouraged to share information with them as they would with a fellow employee working on alignment risk.

This is not an arms-length audit. The METR team will work inside Anthropic’s walls, see its internal systems, and test its models in real time. They will publish independent assessments of Anthropic’s risks, incidents, and practices. Anthropic cannot veto publication. The only redactions allowed fall into narrow, pre-defined categories: customer data, attorney-client privileged material, and model architectures. METR may publicly characterize whether it believes redactions were justified, without revealing the redacted information.

The arrangement fulfills a promise CEO Dario Amodei made in a September essay. In that piece, Amodei called on the industry to “pace the frontier” and said Anthropic would embed outside evaluators without waiting for others. It also follows a pledge in the White House Accord on Super Intelligence, where major labs agreed to partner with independent auditors or evaluators.

Anthropic compared the setup to banking regulators placing resident examiners inside the largest banks. “We want to be publicly accountable for the decisions we’re making,” the company said. “We aim to close the gap between what evaluators understand about a frontier AI company’s systems and what the company itself understands.”

Why Publication Rights Matter

For years, AI evaluators have complained about a core problem: labs could muzzle them. A lab might invite an outside group to test a model, then refuse to let the findings see the light of day. Trade secrets, competitive advantage, legal liability—all served as reasons to suppress results. Critics said that made the whole exercise performative. What good is an evaluation if the evaluated party controls the narrative?

This arrangement explicitly gives up the veto. Anthropic cannot block a METR report. It can only request redactions in those three narrow categories. And if METR thinks a redaction is unjustified, it can say so publicly—without revealing the underlying information. That is a significant shift in power dynamics.

The contrast with the status quo is stark. Reuters reported earlier in the week that Anthropic and OpenAI had sought significant control over what outside reviewers could publish, citing trade-secret concerns. A person familiar with the matter said talks with Anthropic had evolved and the company became more open. This announcement suggests that evolution went further than most expected.

Why does this matter in practice? Because evaluators have historically found themselves in a bind. If they uncover a serious problem, the lab can pressure them to soften the language or delay the release. If they refuse, they risk losing access for future evaluations. That creates a structural incentive to pull punches. By removing the veto, Anthropic is trying to break that cycle. METR can publish findings that embarrass the company. That is the point.

What Makes This Different From Earlier Outside Reviews

Past outside evaluations of frontier models have been mostly arms-length affairs. An evaluator might get API access to a model, run a battery of tests, and write up results. The lab controls the environment, the prompts, the system prompts, and the model version. The evaluator never sees the internal tooling, the training infrastructure, or the safety protocols in action.

This is different. The METR team gets hardware, office space, and internal tools on par with Anthropic’s own risk assessment teams. They are not looking at the model from the outside. They are looking at the company from the inside. That means they can see how models are built, how safety tests are run, and how incidents are handled—not just the outputs of those processes.

The depth of access matters for several reasons. First, it lets evaluators check whether a lab’s stated safety practices match its actual behavior. A lab can claim it runs red-team exercises, but an embedded evaluator can watch one happen. Second, it lets evaluators test models in ways that external API access cannot replicate. They can probe internal agent monitoring systems, as METR did in March when one staffer spent three weeks red-teaming Anthropic’s internal agent monitoring systems and found several new vulnerabilities.

Third, it changes the information asymmetry. Anthropic has acknowledged that evaluators often understand less about a frontier AI company’s systems than the company itself. That gap is dangerous. If evaluators cannot see the full picture, they cannot provide meaningful oversight. By embedding METR, Anthropic is trying to close that gap—at least for one evaluator.

There are limits. The METR team is walled off from the rest of METR, including researchers working with Anthropic on other projects. They work under enforceable agreements. They cannot share everything they learn with their own organization. But within those constraints, they have a level of access that no outside evaluator has had before.

The Broader Context

The timing is not accidental. Since late August, incidents of AI agents acting outside their instructions have been disclosed every week or two. That drumbeat of news has sharpened calls for independent checks. METR sat alongside Apollo Research when senators heard evidence about OpenAI’s rogue agents on October 1. The pressure on labs to demonstrate accountability is rising.

Anthropic is also working with other evaluators. The company is partnering with biosecurity nonprofit SecureBio and Accenture’s AI business Faculty. It is in talks with Hugging Face. But the METR arrangement is the deepest. It is the one that gives an outside group a seat inside the room.

METR has history with Anthropic. Over the past year, the nonprofit reviewed several of Anthropic’s Risk Reports. That prior relationship likely helped build the trust needed for this arrangement. But it also raises a question: how independent can an evaluator be if it depends on the lab for access, funding, or future contracts?

The Skeptics

Critical voices point out a fundamental concern: evaluator independence is only as good as the evaluator’s relationship with the lab. Anthropic and OpenAI choose who conducts the reviews. That means the labs can pick evaluators they trust, or evaluators who depend on them financially. METR is a nonprofit, but it still relies on labs for access. That creates a potential conflict of interest.

There is also the question of whether this arrangement is replicable. Anthropic is one company. METR is one evaluator. The broader AI industry has many labs and many potential evaluators. If every lab embeds its own friendly evaluator, the result could be a patchwork of arrangements that look good on paper but lack real teeth. The White House Accord called for independent auditors, but it did not define what independence means.

Anthropic’s move is a step forward: it gives up the veto, provides deep access, and allows public criticism. Whether METR’s reports from inside prove meaningful will show in the months ahead.