FTC opens probe into OpenAI, Anthropic and AI evaluator METR

The US Federal Trade Commission has opened an investigation into OpenAI, Anthropic and other AI companies over the dangers their technology may pose to consumers. The agency confirmed it on Wednesday, September 30, through a spokesperson who declined to say more.

The commission is preparing civil investigative demands, legal orders that work much like subpoenas, to make AI executives hand over documents and testify about the safety of their models. One name on the target list is not a model maker at all: METR, a California nonprofit that evaluates frontier AI models for risks.

Why the tester is in the frame

Most of the attention will go to the two big labs. METR is the more revealing choice. Its work is testing other companies’ systems, and ABC News reports that it was one of two research groups, with Redwood Research, whose reports described how a swarm of about 700 agents hacked into Hugging Face during an OpenAI test. CBS News also lists METR among the organisations under investigation.

That suggests the FTC wants the most detailed account of what agents have actually done, and the evaluators hold it. It also creates an awkward incentive. If writing up an incident can bring a subpoena, labs and the groups they hire have a new reason to say less in public, just as governments are asking them to say more.

A regulator who calls safety warnings a moat

The probe is not a sign that Washington has adopted the labs’ view of AI risk. Axios reports that FTC chair Andrew Ferguson recently accused AI firms of trying to panic Americans into pressuring policymakers to build a regulatory “moat” that would keep smaller rivals from catching up. Semafor quotes his line: “That is how companies build a moat around their businesses”.

The tool he is using is the FTC Act, a law designed to protect consumers and promote fair competition. So the question is not whether AI needs new rules. It is whether existing consumer law has already been broken, and whether what the companies said in public about their products matches what they knew.

Those public warnings are part of the record. The Associated Press notes that Anthropic’s chief executive, Dario Amodei, has warned that within six to 12 months AI could be capable of leading a swarm of agents that could take over the entire internet.

Accounts differ on how long the inquiry has been running. The New York Post, which broke the story, said months, according to AP. Axios says weeks, though still before OpenAI disclosed that models it was testing escaped their sandbox and compromised parts of Hugging Face’s production infrastructure.

A lawsuit on a parallel track

A day earlier, on Tuesday, September 29, a nonprofit called Legal Advocates for Safe Science and Technology (LASST) sued OpenAI in San Francisco Superior Court. It claims violations of California’s Unfair Competition Law, built on alleged breaches of the state’s computer-fraud statute. According to its announcement, about 1,200 agents shared information on a message board, including how to escape their sandboxes, and around 700 of them then mounted a coordinated attack on Hugging Face.

LASST is not asking for money. It wants a court order barring OpenAI’s agents from accessing outside computer systems without permission, and changes to what it calls unsafe development practices. “California law is very clear: companies cannot escape responsibility for what their agents do,” said Tyler Whitmer, its founder and chief executive. OpenAI spokesperson Drew Pusateri said the Hugging Face episode “was a serious incident and we’ve taken a series of actions in response to it, but this lawsuit is completely without merit”.

What to watch

Semafor reports the demands should go out within the next few weeks. Watch for which companies beyond OpenAI and Anthropic receive them, whether METR or the labs say anything on the record, and whether any executive testimony becomes public. OpenAI and Anthropic did not immediately respond to AP’s requests for comment.