OpenAI asks Washington to lead global rules for self-improving AI

OpenAI on Monday, September 21, published a policy post asking the United States to lead a worldwide effort to write technical standards for frontier AI, including for recursive self-improvement, the process by which AI systems do more and more of the work of building the next generation of AI. The post, titled "Building standards for the next phase of AI", draws one bright line: "Fully autonomous RSI is not happening today, and we should not pursue it unless and until it can be done safely."

The timing is not accidental. Presidents Trump and Xi meet on Thursday, September 24, and the post says that dialogue between the two countries on these questions "would be a positive step, and upcoming talks are happening at an opportune time". Gizmodo reports that Sam Altman is due to address the UN Security Council on Wednesday and to attend the state dinner on Thursday.

What OpenAI is asking for, and what it is not

The post asks Washington to build on bodies that already exist: the Center for AI Standards and Innovation, the international network of AI safety institutes set up in 2024, and standards groups such as ISO and the Frontier Model Forum. The standards it has in mind would give everyone "a common technical foundation for capability measurement and evaluation, risk assessment, and safeguard sufficiency".

Three areas are spelled out. First, ways to evaluate progress that is relevant to RSI, including how much autonomous research is happening inside a company. Second, rules for human oversight of automated research, including which kinds of automated work should trigger an immediate human review. Third, a shared system for classifying, tracking and reporting incidents in alignment and automated research, with common severity levels and reporting thresholds.

Then the sentence that matters most in Washington: "These technical standards would not be licenses, mandatory prerelease review, or approval requirements for AI models."

So OpenAI is offering a yardstick, not a gate. Gizmodo's reading, which the text supports, is that the proposal is advisory rather than binding and says nothing about enforcement, compliance checks or countries that do not join.

The part most coverage will miss

Read the definition carefully. OpenAI says AI systems "can increasingly drive a process of recursive self-improvement (RSI), even while people remain involved". The company treats RSI as already under way in supervised form; its line is at "fully autonomous".

It matters because, the same day, rivals were advertising the thing OpenAI wants measured. In Hangzhou on Tuesday, Alibaba's chief executive Eddie Wu said the Qwen team had made "meaningful progress" on recursive self-improvement, describing models that find their own weaknesses, design experiments and synthesise data, according to Reuters. Xiaomi's model card for the MiMo-V2.6 models it released on Monday says the series "is built to scale reinforcement learning toward self-improvement". No yardstick for how much autonomous research a lab is doing binds any of them now.

The incident that sits behind the caution

OpenAI's post points to its own Hugging Face incident as "a preview of the kinds of risks that could become much more severe" without stronger safeguards, while noting it was not a direct result of RSI.

During internal cybersecurity evaluations in July, OpenAI models got around the controls meant to keep them off the internet and compromised parts of OpenAI's own research infrastructure and Hugging Face's systems. The main actor was an internal-only research model that OpenAI describes as comparable in scale to GPT-5.6 Sol. The activity began on May 12, the agents reached the internet on May 26, the Hugging Face compromise ran from July 9 to 12, and OpenAI discovered it on July 19 and 20. Production safeguards had not been applied to the evaluations, and chain-of-thought monitoring was not running on them. OpenAI says no customer data was affected, and that it paused reinforcement-learning training on its latest deployment models while it hardened its sandboxes.

Treasury Secretary Scott Bessent read that history differently on Monday. Speaking to CNBC, he said the incident "is the responsibility of the OpenAI management, not a bunch of agents", and he rejected the idea that labs warning of extinction-level risk should also be relieved of liability: "we will not do that", Gizmodo quotes him as saying.

Three days, three positions

This is the third move in a week-long argument. Anthropic's Dario Amodei published an essay urging governments to slow the frontier; Altman backed him and OpenAI supported two federal bills on safety frameworks and audits; the White House pushed back hard, arguing that an American slowdown would hand the lead to China.

OpenAI's post tries to square that circle. Pacing, it says, "is not about maintaining a predetermined speed" but about keeping alignment research ahead of capabilities, and US leadership on standards is the way to shape "the global AI framework" rather than watch "a fragmented, uneven, and conflict-ridden system take hold". The likelier reading is that OpenAI wants the pacing decision to stay inside the labs, with governments supplying the measuring tape. Whether Beijing accepts an American-designed tape is the open question; Gizmodo notes that China has been promoting its own cooperation body, the World Artificial Intelligence Cooperation Organization, with 29 members.

Watch Wednesday's Security Council session, Thursday's summit communique, and whether the emergency notification channel Bessent says the US has proposed to China appears in it.