Hugging Face Policy Chief Warns Embedded AI Evaluators Won't Be Easy

AI labs can't yet meaningfully compare their model evaluations to each other — and that gap is the core problem facing the industry's push for embedded third-party evaluators, according to Hugging Face's chief policy officer, Irene Solaiman.

Speaking at TechCrunch Disrupt last week, Solaiman laid out a blunt case: the benchmarking systems labs stake their reputations on are methods that are "complicated and still maturing." Testing, she said, needs to become a more consistent science before the industry can lean on it.

"They are an evolving science — where we don't know how to meaningfully compare them to each other," Solaiman said.

Can the results be trusted?

That's the practical question for anyone buying or deploying AI on the strength of a benchmark score. Solaiman's answer hinges on access.

"Trust is such a weighted term," she said. "In terms of whether we can trust results, one big part [of that] is can you reproduce the results of an evaluation? Do you have the level of access that you need to the model itself — this is where open weights come in in a big way."

Reproducibility has long been one of the stickier parts of benchmarking, and it remains a live debate across the industry. If you can't rerun the test yourself, you're taking the lab's word for it. Open-weight models change that equation.

Benchmarks also age fast. "They start to get ineffective relatively quickly, and we don't have as much of a thriving ecosystem as we need," Solaiman added.

What does this change for embedded evaluators?

The embedded evaluator model — placing supposedly "neutral" third-party observers inside AI organizations — has gained traction over recent months. Labs embraced it after a rash of safety incidents raised concerns about potential harms. Hugging Face itself was famously the target of one of those incidents.

Solaiman's warning cuts at the premise. "There's this hunger for an embedded evaluator ecosystem now, but the qualifications that determine who is an effective evaluator across what tasks need a lot more standardization," she said.

The industry has already seen how messy that question gets. Anthropic's decision to hire Accenture as its first evaluator drew considerable backlash, with critics mocking the idea that the consulting firm was qualified for the job. If nobody agrees on what makes an evaluator effective, the neutrality of the whole arrangement is up for grabs.

The testing target is shifting

There's a second, quieter shift underneath the debate: evaluations are moving from testing the model alone to testing the entire system around it — the harness and the agent included.

The harness, the tool a user works through to reach the model, matters more than most buyers assume. A weaker model paired with a better harness could outperform a frontier model, Solaiman shared.

That reality reshuffles procurement logic. A leaderboard ranking tells you less than it appears to if the score reflects only the model in isolation.

The problem is cost. "Agentic evals are really hard and really expensive," Solaiman said. "We have just more evaluations for models. … We don't have as many evaluations for harnesses, and we are developing better evaluations for the holistic agent — because that's what a lot of folks are getting value out of."

Where this leaves the market

Strip away the jargon and the situation is simple: evaluation is currently a free-for-all. For benchmarking to stay useful to industry leaders, Solaiman's implication is clear — the sector needs a uniform, industry-wide approach where everyone works from the same page.

Until standardization arrives, practitioners should treat benchmark claims as directional, not definitive — and weight reproducibility and system-level testing accordingly.