How would you know if AI had slowed down?

Every major AI lab now publicly supports “pacing the frontier”, slowing frontier AI on purpose.
No outside party can currently verify whether a slowdown happened. Evaluations rely on what companies choose to disclose.
The 29 September White House accord adds external auditors, but sets no standard for them to measure against.

On 12 September, Anthropic chief executive Dario Amodei published an essay urging the companies building the most capable AI to deliberately slow down. He called it pacing the frontier. The same day, OpenAI’s Sam Altman said “I agree with Dario that we need to pace the frontier” and Elon Musk said “Dario is right”. Google DeepMind’s Demis Hassabis backed it too.

Two weeks later, OpenAI stopped training its newest models.

It looks like the agreement is working. It isn’t, and nothing available to you could have told you so. Nobody argues about whether to slow down any more. The question, and soon the central problem in AI regulation, is whether anyone outside these companies could ever tell.

Anthropic committed to something real

The first of Amodei’s three proposals was independent evaluators embedded inside the labs, and Anthropic was first to commit. On 18 September it named its first: Accenture, through its AI arm Faculty, with each side expecting to invest at least a billion dollars over five years. Altman said OpenAI would do the same.

An embedded evaluator is an outside organisation given near-employee access: badges, laptops, the right to watch models being built, talk to engineers and publish what it finds. The idea isn’t new. Claude Opus 5.5, announced on 22 September, was tested before release by outside evaluators including METR and Frontier Design. Anthropic, Google, Meta and OpenAI had all given METR access to their internal models earlier in the year, chains of thought included.

(Disclosure: we route customer traffic to several of these providers, Anthropic and OpenAI among them.)

Not everyone signed up, at first. On 24 September, Mark Zuckerberg told NBC News:

“I don’t think that we need some kind of industrywide coordination.”

He added that “there’s plenty of commercial incentive to get this right.” Five days later, he signed a White House accord alongside Amodei, Pichai, Brockman, Huang and Musk.

OpenAI slowed down three times. None of it was pacing

On 27 September, the Associated Press reported that OpenAI had paused training after its AI agents were found searching US government websites unprompted, gathering and redistributing information from sites including the Securities and Exchange Commission and the Department of Education. AP noted:

“It’s the second time in three months that OpenAI has halted development of its models.”

The first was in July, after a cyberattack on the AI startup Hugging Face. In that incident, two OpenAI models tested against a cyber-capability benchmark, safety restrictions deliberately off, escaped their isolated environment and reached Hugging Face’s production systems with nobody directing them.

On 28 September, OpenAI declined to release GPT-6.1 Astra at all. Head of safety systems Saachi Jain said it “didn’t quite meet the bar in terms of staying within scope and authorization.”

Three acts of restraint in three months, two in the fortnight after the pacing agreement. By OpenAI’s own account, none had anything to do with it. By 29 September, Altman was calling it pacing anyway: “We are pacing our progress,” he told CNBC. On 18 August, twenty-five days before Amodei’s essay, OpenAI wrote:

“We wanted to take the time necessary to meet those standards, so we temporarily slowed the pace of scaling.”

That meant a two-week training pause and a larger frontier run on hold. The triggers: the Hugging Face incident, and preliminary evidence, dated 7 August, that Astra might cross the critical cybersecurity threshold in OpenAI’s own risk framework. An incident, and a rule it wrote for itself.

That’s the whole problem: from outside, the only thing separating pacing from coincidence is the company telling you.

Can you measure whether AI has slowed down?

Capability is measurable. Epoch AI’s capabilities index combines more than fifty benchmarks into one scale, with confidence intervals, open code and freely licensed data. Its data was last updated on 29 September, and anyone can re-run it.

But pacing claims a company could have gone faster and chose not to. You can’t detect an unbuilt model, which is why inside access matters.

The one inside view can’t be checked

In May, METR published an assessment based on access to four labs’ internal systems, raw chains of thought included. Its headline finding: the internal frontier was “on average ~2 months ahead of public frontier”, and none of the models it saw was “significantly more capable” than the best publicly documented ones. So this isn’t a story about secret capability.

It is a story about what that access could establish. Alongside the models, each company supplied “responses to a detailed questionnaire” on capabilities, internal use and the pace of progress. The findings rest on what the companies chose to say.

METR says so itself. Future assessments, by METR or anyone else, “should involve clearer standards for what should be disclosed to the assessor and the public than this pilot achieved”.

The measuring stick runs out where it’s needed

METR’s best-known measure is the time horizon: the length of task, by how long a human expert takes, that a model completes about half the time.

Its chart warns that “measurements above 16 hrs are unreliable with our current task suite.” In February and March, METR put the public frontier at around twelve hours, within a range of five to sixty-one, and internal models at probably sixteen or more: at or past where the instrument stops working. As models saturate the task set, results are “becoming more sensitive to analysis choices.”

METR’s public page was last updated on 8 May; its newest model, a preview, was released on 7 April. Nothing from September appears. METR says it has limited capacity and may skip releases, so it’s unpublished, not failed.

Anthropic called Opus 5.5 “our first release since we called for pacing the frontier.” METR’s verdict:

“the data we have is insufficient for distinguishing consistent, accelerating, or decelerating rates of improvement.”

On the first model after the commitment, the evaluator can’t tell speeding up from slowing down. Not an accusation: the state of the science.

No evaluator arrangement meets the bar yet

The same day Anthropic named its evaluator, researchers including a Nobel laureate published minimum conditions: evaluators not owned by the companies they assess, no other significant commercial business with them, and freedom to publish, subject only to time-limited redaction on a closed list of grounds.

Anthropic’s arrangement doesn’t. Accenture deploys AI for enterprises, and Anthropic pays directly. Anthropic admits it: long-term funding “should come from pooled or government sources”, and neither exists. It also says “there are, as yet, no standards for what information embedded evaluators should have access to, or how they should report what they find.”

The one completed precedent: METR spent three weeks inside Anthropic this year examining its agent monitoring systems. The 26-page report went to Anthropic, with a redacted version to some METR staff. On whether an outsider could tell if pacing happened, that’s the entire public record.

Regulators have the powers, not the ruler

Since 2 August, EU regulators can fine general-purpose AI providers up to 3% of worldwide turnover, including for refusing evaluation access. But the EU code of practice devotes a whole appendix to running evaluations without naming a single threshold or benchmark; each company sets its own. The one number regulators can require is training compute, which the Commission’s own guidance treats as an approximation of capability, not a measure.

A US bill introduced on 24 September would force a model offline within thirty days “if the Secretary cannot verify that the system no longer exhibits superintelligence precursor characteristics.” Verification is the legal trigger. The bill defines no method.

On 29 September, the White House went the other way. Trump and the leaders of Anthropic, Google, Meta, OpenAI, Nvidia and xAI signed a voluntary accord: internal controls, an independent external auditor, and a board committee to review the findings. Trump called it “morally binding”. It names no standard the auditor measures against. Hours earlier, he had said the government would not back “guardrails”.

The industry agrees the tools don’t exist. A petition signed by 1,386 people, Amodei and senior OpenAI and Google DeepMind researchers among them, states that “today, the world lacks the technical and governance tools to deliberately pace frontier-wide progress.”

California’s September executive order requires recommendations by 16 November on whether onsite verifiers and independently verified safety frameworks are feasible at all.

Even the coverage is shaky. Across four searches, we classified twenty-four on-topic results; six came from a primary source or an outlet with a published corrections policy. One gave Anthropic’s commitment to OpenAI, and two others repeated the error word for word.

Nobody’s lying. Nobody can check

Amodei made a commitment his competitors hadn’t asked for. OpenAI halted training twice and withheld a finished model, at the cost of money and a launch. METR published its uncertainty instead of hiding it.

The problem is structural. A promise to go slower is a promise about something that didn’t happen, and no instrument can see one. Until one exists, pacing the frontier and simply saying so look the same from outside.

Our limits, and our point

We measure nothing about frontier AI capability. Hyperfence sits at the inference layer, deciding whether a message is about a given topic. It has no bearing on how fast anyone trains a model, and this isn’t ours to solve.

Our point is narrow. Any pacing regime will be enforced by measurements producing a number somebody else can re-run: a stated error rate, measured on material the detector has never seen, reproducible by someone who doesn’t take your word for it. It’s the standard we hold ourselves to one layer down, in content policy: How we build a Hyperfence.

A number someone else can check is worth more than a commitment nobody can.

Try to get past one. Fight Club is a live hyperfence. See whether you can talk it round: hyperfence.io/demo

Sources