GPT-6-Astra Solves 49 of 50 Unpublished Physics Problems in 35-Author Benchmark Study

A 35-author preprint called TheoPhysEval, posted to arXiv on Oct. 9, 2026, reports that OpenAI's GPT-6-Astra solved 49 of 50 unpublished, research-level theoretical physics problems. That is 98%.

The authors come from institutions including the International Centre for Theoretical Sciences (ICTS) in India, Harvard, KU Leuven and Google DeepMind. Suvrat Raju of ICTS coordinated the effort.

How the test worked

The 50 problems span eleven subfields, from fluid dynamics and biological processes to particle physics and cosmology. Each came from an expert in that subfield and has a checkable numerical or symbolic answer. None had been published when testing began.

The authors say a leading specialist would need about a day or less for a typical problem. Specialists from neighboring fields would need far longer.

The researchers tested commercially available agents with no human hints. GPT-6-Astra ran inside OpenAI's Codex environment. Anthropic's Opus 5.5 ran inside Claude Code.

Each agent ran the set twice: once with web search, once with no internet access.

The scores

With web search, GPT-6-Astra solved 49 of 50. The full run cost about $116, or under $3 per problem on average.

Opus 5.5 came in second at 92%. Google's Gemini 3.8 Flash scored 68%. The open-weight models DeepSeek-V4.1-Flash and GLM 5.3 scored 54% and 42%.

With the internet cut off, Astra and Opus each scored 92%. The authors read that as a sign that leading models have absorbed much of the theoretical physics literature.

The weaker models made errors an expert would catch immediately. On one black hole problem, Gemini and GLM copied formulas from an older paper unchanged during the web-search run. Those formulas apply to five spacetime dimensions. The question required four.

Solving that problem means finding the older paper, adapting its equations to four dimensions and computing the result with custom code. Raju, a co-author of the older paper, needed about a day.

Answer keys and the one miss

The authors also report that follow-up checks with the experts found small errors in some of their own reference solutions. Once those were fixed, Astra's answers on those problems were correct.

One problem went unsolved by every model tested. It asked for an equation that produces a given curve. The authors admit the problem might seem "unfair," but say it is realistic, because researchers often have to infer the underlying physics from imprecise data.

What the authors say it does not show

The authors are careful about the limits. The problems are what they call "turn-the-crank" work: applying known methods to a slightly altered question. Contributors often changed a few values in a calculation from their own recent papers so the answer could not be guessed from the old result. The authors say tasks like these are often handed to PhD students as a first step into research.

The set also leaves out major open questions. The authors say some current models may already beat the best human researchers at concrete calculations, which they call a major milestone. But theoretical physics has not seen an AI-driven breakthrough the way mathematics has.

Their explanation is "taste," the hard-to-measure judgment about which questions are worth pursuing. They stress these are observations that cannot be quantified, not a forecast. They still say AI agents could do top-level research on their own in the near future.

Caveats on the number

This is a preprint and has not gone through formal peer review. The benchmark is 50 problems, a meaningful but finite sample. The paper's author list includes Google DeepMind researchers.

The 49-of-50 figure is a measure of reliability on bounded calculations with known methods. It is not a measure of discovery. Memorization cannot be fully ruled out. The authors point to the small gap between the web-search and no-internet scores as a sign the models have absorbed much of the literature.

The cost figure stands out. About $116 for 50 problems, each of which the authors say could take a top expert a day or less, is small next to the time human experts would spend. The authors' estimate that only a handful of people worldwide could once have done some of these problems, and only over several days, is part of why they call the result significant.

The problems and test setup are posted publicly on Hugging Face, so outside groups can rerun them. Whether other labs reproduce the 98% score, and whether any agent shows the taste the authors say is missing, are the questions the preprint leaves open.