- Anthropic published three internal measurements on September 17, 2026, including an index showing Claude now “leads” 26% of the company’s own AI R&D work, up from under 1% in February 2026.
- Roughly 30,000 agents were doing research and engineering work at Anthropic at any one time in August 2026; over a billion agent decisions were analyzed and 0.002% (about 1 in 47,000) were blocked in real time.
- In the week of July 13-20, about 6% of compute going to AI R&D went to safety work, rising to about 12% for AI-driven AI R&D.
- The disclosure lands mid-argument over Dario Amodei’s “pace the frontier” proposal, which Nvidia’s Jensen Huang and the White House have publicly resisted.
Twenty-six percent. That is the share of Anthropic’s own AI research and development work that Claude now “leads” — meaning the model completes most of the task end to end from a high-level prompt while a human supervises. Seven months earlier, in February 2026, that figure was under 1%. Anthropic published the number itself on September 17, alongside two other measurements no frontier lab had previously put in public. The question the disclosure raises is not what the numbers say. It is why a company would volunteer them at all, in the exact week the industry is fighting over whether it should slow down.
The Metric That Measures a Model Building Its Successor
What the R&D Automation Index actually counts
Anthropic built what it calls the Anthropic R&D Automation Index: a catalogue of every kind of AI R&D task done inside the company, each rated for how automated it currently is, then weighted and aggregated. The rating scale comes from Epoch AI and runs from AL0 (no AI involvement) to AL5 (fully autonomous, no human in the loop). AL3 is “AI collaborates” — large chunks of work under close human direction. AL4 is “AI leads” — the model works end to end from a high-level prompt while a human supervises the result.
The August 2026 snapshot reports three things. Claude is not operating fully autonomously for any measured subset of AI R&D work. Claude “leads” 26% of that work. And the share of work sitting at or above “AI collaborates” is above 90%. That last figure is the one most likely to be underweighted in coverage: it means the overwhelming majority of model-building work at a frontier lab already runs with a model inside the loop, not beside it.
How the number was constructed — and where it can break
The methodology is unusually specific for a corporate transparency post. For each week of July 2026, Anthropic randomly sampled 20% of staff in the departments that make up the model R&D loop. A Claude research agent reviewed each sampled person’s week using Slack and internal documentation and listed their tasks, producing roughly 15,000 granular task records. Those were organized into a frozen hierarchical tree of 542 nodes, 378 of them leaves — categories as specific as “eval platform defect diagnosis and fixes” and “serving incident postmortems.” An independent Claude judge then assigned an automation level to each node based only on evidence from that month or earlier.
Anthropic also published its own validation and its own caveats. Staff who own each work area rated their areas blind, without seeing the model’s evidence. The judge model agreed with humans about as often as humans agreed with each other — 59% exact agreement for model-versus-human, 35% for human-versus-human — and model and human ratings landed within one level of each other 97% of the time. The company names the two structural problems plainly: there is no common methodology across labs, and a lab using its own models to judge its own systems risks a judge that makes the same errors as the model it is checking.
Trend Insight — The 26% figure will get the headlines, but the frozen basket is the part that matters operationally. The index measures how automated July 2026’s work has become, not whether new categories of human work have appeared since. Anthropic checked this by rebuilding the tree from January 2026 data and found no rise in novel tasks between baskets — a finding that, if it holds, suggests the structure of model R&D is stable while the labor inside it is being reallocated to machines.
Thirty Thousand Agents, and One Block in Every 47,000 Decisions
Coverage, latency, escalation
The second measurement covers oversight. As of August 2026, approximately 30,000 agents were doing research and engineering work at Anthropic at any one time on its most-used internal platform. The company proposes three metrics any developer could report: coverage (the share of agent actions that pass a monitor), review latency (how quickly a flagged action is reviewed, first by machine and then by a human), and escalation rate (the share of activity blocked, redirected, or flagged).
Two monitor types run in parallel. Online monitors check actions in real time and block irreversible harms — the example given is an agent copying model weights out of Anthropic’s systems. Offline monitors read transcripts after the fact, targeting slow-developing, reversible problems such as signs of misalignment. Coverage is 100% for both. Humans review any blocked action within one week. The offline monitor flags roughly 100,000 transcripts per week, most resolved by classifier review, with approximately 50 per week escalated to human eyes.
Why a tiny percentage is not a small number
Anthropic analyzed over a billion decisions from its research and engineering agents across August 2026 and found that 0.002% of them — about 1 in 47,000 — were blocked by the online monitor. The offline monitor flags one to two transcripts in every thousand for further review. The company frames the scale problem directly: individual agents rarely misbehave, but when millions or billions of agents operate in the economy, even rare events happen regularly.
Two design choices make the monitoring tractable. Each agent is given a persistent individual identity that is not tied to a model, so its record survives model upgrades and every action can be audited per agent over time. And agents communicate through a shared, open messaging system rather than privately, so they can catch each other’s errors and every message can be cross-referenced against transcripts. METR has previously red-teamed the offline monitoring platform independently.
Trend Insight — For anyone running agent fleets in production, the identity decision is the transferable lesson. Tying every artifact to a durable per-agent identity rather than to a model version is what makes retrospective audit possible at all — and it costs almost nothing to adopt early, versus retrofitting it across a system that has already generated millions of untraceable actions.
Six Percent of R&D Compute Went to Safety
One week, sorted by classifier
The third measurement examines a snapshot of how Anthropic used all of its compute from July 13 to July 20, 2026. Every workload was sorted into a small number of categories, then classified as safety work or general AI R&D. The result: about 6% of compute going to AI R&D was allocated to safety, and about 12% of compute going to AI-driven AI R&D was allocated to safety.
Rather than classify all of the week’s almost 10,000 research runs, Anthropic sampled roughly 14% of them, weighted toward the runs that consumed the most compute so the result reflects where compute actually went rather than how many runs there were. Work that advanced capability as much as it advanced safety was counted as AI R&D, not safety — which makes the 6% a deliberately conservative floor rather than a ceiling.
The measurement’s own stated weaknesses
Anthropic is explicit that compute share is an imperfect proxy. Safety research consists largely of individual researchers designing experiments — time-consuming to produce, but not especially compute-intensive to run, so it will always look small next to a frontier training run. Three further limits are named: many underlying workload labels are best-effort and unverified rather than technically enforced; one week is enough to prove the measurement can be made but not enough to establish a trend; and efficiency improvements can lower the safety share without any reduction in safety work actually performed.
The value Anthropic claims for the metric is comparative, not absolute. Because compute is among the most verifiable inputs to AI development, a shared definition of safety work would let outsiders compare like with like across developers and over time — and could become a lever in any future pacing agreement, such as a commitment on the minimum share of compute devoted to alignment and evaluation.
Trend Insight — Anthropic concedes the definitional problem candidly: each developer will be tempted to draw the safety line generously, and the burden of proof should sit with the developer. That concession is the tell. A metric a lab can define in its own favor is only useful once a third party defines it instead — which is precisely the role Anthropic says it is now creating.
Why This Landed in the Middle of a Fight
The pacing argument, and the pushback
The measurements arrive days after Anthropic CEO Dario Amodei published a plan to “pace the frontier,” and Anthropic says it expects these numbers to shift if the industry coordinates on pacing. The company also says it plans to embed independent third-party evaluators from multiple organizations, giving them access to internal processes, systems, and data comparable to what internal risk assessment teams hold — and on September 18, Anthropic named Accenture as its first embedded evaluator.
The reception has not been uniform. Nvidia CEO Jensen Huang has publicly rejected the premise of a slowdown, echoing President Trump’s position that the AI backlash is overstated and regulation unnecessary. On TechCrunch’s Equity podcast, host Anthony Ha noted how quickly industry figures coalesced around Amodei’s proposal — encouraging as consensus, he said, but also grounds for skepticism that it represents genuine steps toward a slowdown. Kirsten Korosec put the alternative reading bluntly: consensus can be a good thing, and it can also be the beginnings of forming a cartel.
Whether markets alone can hold the line
The sharper critique on that episode was structural. Sean O’Kane argued that the usual correctives are not operating here: enforcement appetite is low, and consumer choice is a weak lever in a market that has shifted toward enterprise contracts. “Corporations are not going to pack up and move from Codex to Claude Code just because they disagree on principle,” he said — and the volume of investment capital flowing to frontier labs absorbs whatever losses do materialize.
Which is the context that makes the published numbers interesting regardless of motive. Anthropic’s own framing is that the world should minimize the gap between what frontier labs know and what the public knows, and that any developer could report these three measures today using a public methodology. That claim is now testable in the simplest possible way: the measurements exist, the method is documented, and every other frontier lab has been handed a template it can either match or decline to match in public.
Trend Insight — First-mover transparency is a competitive move as much as a safety one. By publishing a methodology and inviting comparison, Anthropic converts its own disclosure into pressure on rivals, where silence starts to read as an answer. Watch for whether OpenAI or Google DeepMind publish a comparable automation index in the next two quarters — that, not the 26%, is the number to track.
Related
- Four AI Models Shipped in One Week. Spending Fell Anyway.
- Oracle’s AI Backlog and the Cash Flow Problem Behind It
- The US-China Distillation Fight Over Cheap AI Models
- Tech Digest: What 176 Coding Agent Runs Revealed
- Tech Digest: The Prompt You Should Delete Right Now
Sources
- Anthropic — Measurements for understanding the pace of AI development inside frontier labs (September 17, 2026)
- TechCrunch — Is the AI industry really ready to slow down? (September 20, 2026)
- Anthropic Newsroom — Partnering with Accenture on embedded evaluation (September 18, 2026)
AI Biz Insider · AI Trends EN · aibizinsider.com
답글 남기기