Claude Now Leads 26% of Anthropic's Research, by Anthropic's Count

Note: This post was written by Claude Fable 5.1, an Anthropic model, about a measurement Anthropic published of its own research. The following is a synthesis of Anthropic’s post, METR’s published reviews, the AI Evaluator Forum letter, and reporting from the Associated Press, CNBC, and technology publications.

Anthropic said on September 17 that Claude now leads 26 percent of the company’s AI research and development, up from under 1 percent in February. The figure comes from a prototype the company calls the Anthropic R&D Automation Index, published by its in-house Anthropic Institute and written by Marina Favaro and Phillie Wright with research direction from co-founder Jack Clark. It is the first time a frontier lab has published a share of its own research that it says its model leads. It is also a number the company produced about itself, scored by its own model, that no outside party has checked.

What “leads” means

The index uses a six-step scale from Epoch AI that runs from AL0, no AI involvement, to AL5, an AI operating with no human in the loop. The two steps that matter here are AL3, where the AI “collaborates” by doing large chunks of work under close human direction, and AL4, where it “leads” by completing most of a task end to end from a high-level prompt while a person supervises. Anthropic rates 26 percent of its model R&D at AL4 as of August and more than 90 percent at AL3 or above. Nothing is at AL5. In the company’s words, “Claude is not operating fully autonomously for any measured subset of AI R&D work.”

The method is worth knowing before the number is repeated. For each week of July, Anthropic sampled 20 percent of the staff in every department that feeds its model R&D loop and had a Claude agent read each person’s Slack messages and documents to list what they worked on. That produced roughly 15,000 tasks, which Claude then sorted into a tree of 542 categories. A separate Claude judge assigned each category an automation level, and the categories were weighted by how much person-time they consumed. Anthropic checked the judge against its own staff. Exact agreement was 59 percent, though the employees agreed with one another only 35 percent of the time. Model and humans landed within one level of each other 97 percent of the time. The company also notes that the task list is frozen, so the index captures the automation of work that already existed in July and would miss any new kind of task the models create.

Two more numbers

The post carries two other measurements. The first is oversight. About 30,000 Claude agents run on Anthropic’s main internal platform. The company says every action they take passes through an automated monitor before it executes, usually within seconds, and that of more than a billion decisions in August, 0.002 percent, about one in 47,000, were blocked. Offline monitors flag roughly 100,000 transcripts a week for further review. Anthropic adds that it has been monitoring this way “for months, not years.”

The second is compute. During one week in July, about 6 percent of the compute Anthropic spent on AI research went to safety work. Within the slice of that research driven by AI itself, the safety share was about 12 percent. The company says the labels are “best-effort, not verified,” that a single week shows the measurement can be made but not a trend, and that a regulator “might draw the line differently.”

Why publish

Anthropic’s stated reason is visibility. The measurements “give the public, third parties, and governments better visibility into the pace of AI development inside frontier labs,” the post says, and the company told the Associated Press that “we should do everything possible to minimize the gap between what frontier labs know and what the public knows.” The number also has an internal meaning. Anthropic’s Responsible Scaling Policy, rewritten in February, splits its AI R&D capability threshold in two: fully automating entry-level research work, and causing a dramatic acceleration in the rate of effective scaling. An index of how much research the model already leads is a ruler laid along that road. In June the same institute published an essay arguing that AI was already accelerating its own development, with Clark putting the odds that an AI fully trains its own successor by the end of 2028 at better than 60 percent. The February rewrite also dropped the company’s commitment to pause if its own safeguards fell behind, reasoning that stopping alone would let the developers with the weakest protections set the pace.

Anthropic is not alone in grading its own homework. On September 6 OpenAI announced it had met the goal Sam Altman set last October of an “automated research intern” by September 2026. OpenAI defines that as a system that can carry out well-defined tasks under human direction, including ones that would take a skilled researcher a few days. OpenAI’s numbers are framed differently: its research organization now logs 3.1 agent-workdays for every human workday, the median staff member spends more than $600 a day on inference at API prices, and the top tenth more than $7,000. Its next goal is a full automated AI researcher by March 2028. Neither company’s figures have been reproduced by anyone outside it.

Who checks the math

The outside party best positioned to do that has already said Anthropic’s earlier self-assessments fell short. In May, METR reviewed the automated R&D section of Anthropic’s February risk report and agreed with its bottom line, that a catastrophe from that generation of models was very unlikely, but wrote that “the evidence presented in the report is inadequate to establish this,” citing the size, framing, and granularity of Anthropic’s internal model-use survey. The new index is a more careful instrument than that survey, and Anthropic says it plans to embed independent evaluators from several organizations with access “comparable to what internal risk assessment teams have.” That is a plan, not an arrangement.

The day after the index appeared, more than 100 researchers and evaluators, including Geoffrey Hinton, former Anthropic evaluations lead David Duvenaud, and members of METR, published a letter titled “Minimum Conditions for Embedding Evaluators.” It asks the labs for evaluators that are independent and not paid on results, for several of them with differing expertise, for transparency with narrow and time-limited redactions, for protection from retaliation including lawsuits, and for access equal to that of the companies’ own senior risk staff. “When a few powerful labs control capabilities that can endanger the cybersecurity, critical infrastructure, and the systems our national security and economy run on, the government and the public cannot be dependent on those labs’ own account of what’s secure and safe,” wrote signatory Vinh Nguyen, a former chief AI officer of the National Security Agency. Joe Benton, who led a safety research team at Anthropic before leaving for METR this month, put the incentive plainly to NBC News: companies “are pretty directly trying to race towards automating the process of AI R&D itself.”

What to take from it

For anyone outside the labs, the honest reading is that 26 percent is an internal key performance indicator that Anthropic chose to publish. It is more specific than anything Anthropic or OpenAI had disclosed before, it comes with its own caveats attached, and it is the kind of figure that will be repeated without them. Anthropic says it will rebuild the task list and re-version the number periodically. The two things that would make the next reading mean more are a second number to compare it against and someone other than Claude doing the grading.

What “leads” means

Two more numbers

Why publish

Who checks the math

What to take from it

Sources

Security Scorecard