OpenAI and Meta Tell Federal Judicial Panel Deepfakes Are Becoming Impossible to Detect

Two of the companies building the technology told a federal judicial panel Thursday that nobody should count on software to sort real recordings from fake ones.

At a meeting in Boston on Oct. 15, representatives of OpenAI and Meta Platforms briefed the U.S. Judicial Conference's Advisory Committee on Evidence Rules. The committee has spent three years, since 2023, studying how courts should handle evidence that may have been fabricated or altered with AI.

What the companies said

"AI images are only getting better, and I do not think that there will be a scientific way going forward to detect them," said Shira Anderson, OpenAI's policy counsel.

Devin Norder, Meta's director of product management, said media from the latest frontier models is indistinguishable to humans from authentic material. He added that making it is cheap and fast: "the cost and the time it takes to create this media really are quite trivial at this point."

Both companies have built tools meant to verify visual content. California and the European Union have passed laws requiring watermarking to flag AI-generated material.

Anderson said that doesn't solve it. Each company runs its own system, and the systems don't talk to each other, so no universal method exists to catch every watermark or verify every image. "I do not think it'll ever be bulletproof," she said.

Few cases so far

The numbers cut the other way. A March survey by the Federal Judicial Center, the judiciary's research arm, polled 931 judges. Only 15 had handled a case in which a party challenged evidence as a deepfake. That is about 1.6%.

Judge Jesse Furman of Manhattan, who chairs the committee, drew attention to the gap. "There's a lot of academic commentary and other commentary predicting that courts will be dealing with deepfakes, but we haven't actually seen a whole lot of that yet," he said. Furman has appointed a subcommittee to study the issue further.

The rule that exists only on paper

The committee has a draft ready. Proposed Rule 901(c) would set a two-step test for evidence challenged as an altered or fabricated AI product. The challenger would first have to produce enough evidence to support the claim. If that threshold is met, the side offering the evidence would have to show it is more likely than not authentic.

The draft has not been approved or adopted. A National Law Review summary of the committee's earlier work says members concluded an amendment was not necessary at that time, given existing authentication methods and the small number of deepfakes in court, and held 901(c) in reserve.

A separate proposal, Rule 707, would hold machine-generated evidence offered without an expert to the reliability standard for expert testimony. It is a different question from deepfakes.

A split on the bench

The committee is not united on whether courts need anything AI-specific. Judge Valerie Caproni of Manhattan argued they already have what they need.

"We've dealt with bad actors presenting fake evidence since Cain and Abel," Caproni said. "That's a longstanding problem." She said existing law already prohibits fraudulent evidence and that judges can determine whether something was forged.

That argument matches the committee's record so far. Its own caution about changing rules stems from the same reasoning: few cases, and existing authentication standards still in place.

The industry testimony points the other direction. If the people who build the generators say a trained eye and a detector will both fail, a judge looking at a convincing video has less to go on than the old assumption that a camera records what happened.

Where authenticity gets decided

The practical issue is the file, not the pixels. A recording that looks real is not necessarily authentic. Equally, doubt about a file does not prove it was manipulated. That pushes weight toward provenance: who created the file, when, on what device, and whether it was changed afterward.

OpenAI and Meta have a stake in this discussion. They sell or support the generation tools and the verification tools alike, and their representatives are now telling a federal committee that the verification side has limits.

The next step is the subcommittee's work. Whether it revives Rule 901(c) or leaves courts to existing authentication rules will depend on whether the 15-in-931 figure grows.