OpenAI’s chief scientist co-signs call to oversee self-improving AI

Twenty-two researchers, among them Geoffrey Hinton, Yoshua Bengio, OpenAI chief scientist Jakub Pachocki and Anthropic co-founder Jack Clark, published a paper on Monday, Sept 28, warning that AI may soon automate most of the work of building AI. If that happens, they argue, the result could be an intelligence explosion, a sharp acceleration “compressing advances that would otherwise take years into months or less”.

The paper, “What if automating AI R&D triggers an intelligence explosion?”, appears as No. 2/2026 in a Frontier AI Working Paper Series and is hosted by the Cambridge Programme on AI Science & Policy. It is written for policymakers, and it says preparing for such an event “should be an urgent priority”. The authors write in a personal capacity; their views “do not necessarily represent the views of the organizations” they are affiliated with.

The key numbers come from inside a lab

The paper’s most striking evidence is a measurement, not a forecast. “Anthropic reports that AI systems’ share of approved code rose from low single digits to over 80% between January 2025 and May 2026,” the authors write. In the same sentence they add, again citing Anthropic, that the proportion of R&D work “autonomously completed with only high-level human supervision rose from 1% to 26% between March and August 2026”.

In five months, the share of research work done by AI with a human only setting the direction went from one in a hundred to about one in four.

That is the figure to hold on to. The rest of the case rests on trend lines. The best systems now complete AI research tasks “that take human experts hours to days”, compared with seconds-long tasks in 2023, and the authors say “some tentative extrapolations of recent trends” suggest that months-long AI R&D projects will be automated by mid-2028.

What they want governments to do

The recommendations come in a deliberate order. First comes visibility: policymakers should consider “requiring standardized reporting of key AI R&D indicators and processes to governments and third-party auditors”, and possibly auditors embedded inside AI companies.

Second come ways to steer and restrain an acceleration. The list includes “limits on the extent to which capabilities can increase within a given time period”, “developing options to pause specific AI R&D workloads”, and requiring some tests of automated research systems to run on air-gapped networks, meaning computers cut off from outside connections.

Third comes preparation: emergency plans for scenarios that include large labour-market shocks, geopolitical instability or a loss of control.

The names on the paper are the news

Hinton and Bengio have warned about AI risk for years. What is new is who stands beside them. Pachocki is OpenAI’s chief scientist, Clark co-founded Anthropic, and Eric Horvitz is listed with Microsoft. Dawn Song of UC Berkeley is also Meta’s vice president of AI research, The Wall Street Journal reported. Anton Korinek lists both the University of Virginia and Anthropic, and no author lists Google DeepMind.

Their companies kept their distance. OpenAI, Microsoft and Meta declined to comment to the Journal, and Anthropic did not respond, according to The Next Web.

So senior people at rival labs have put their names to a request that governments be ready to limit how fast AI capabilities grow, while their employers would not say whether they agree. That gap is the tension to keep in view.

What would change the picture

The argument depends on numbers that only the companies can produce. The 26% figure is Anthropic’s own measurement, which is why the authors put independent measurement first. Watch whether any government moves toward the reporting the paper asks for, and whether other labs publish their own figures on how much of their research AI now does without close supervision.