The announcement yesterday by OpenAI of a solution to the Millennium Prize Navier-Stokes problem has been causing a great deal of discussion among mathematicians. There’s a lot to be said about this story and its implications, most of which I’ll leave to those better informed than me (for one thing, I understand basically nothing about the problem, or its solution).
While talking to people I started to realize that one thing that really bothers me about these kinds of announcements is that they’ve been coming with no information at all about how the result was achieved. Emphasis has been on the question of whether the proof is correct or understandable, but I now think people should start asking another sort of question: exactly how was this done?
Sam Altman yesterday explained that
we tried this because there were rumors on the internet last week that Anthropic’s models had solved a millennium problem and we were curious if ours could do it too.
The rumor of this kind circulating was this one, with first appearance on September 4. Altman appears to be claiming that OpenAI solved the Navier-Stokes problem over the weekend, starting just with the information that maybe Anthropic had a solution. Altman is famous though for being “unconstrained by the truth.”
Tristan Buckmaster and Levent Alpoge had been working for a year on Navier-Stokes with some AI agent help, see here. The rumor I’ve heard is that Levent told colleagues at Anthropic that he and Buckmaster were close to a solution, and that this news somehow got to people at OpenAI, possibly including some information about the methods being used.
The new AI agents have profound implications for mathematics research and I think the math community needs to start discussing specific ways to deal with these. Here’s a modest proposal: the math community needs to adopt a version of the ethical standards of experimental science. If you are using AI agents, you can’t just give a proof (formalized or not), but need to also provide a detailed explanation of how these agents were used to get the result.
In an experimental science you cannot just publish results of an experiment, you also need to precisely describe the experimental setup that produced the results. One reason for this is reproducibility: other scientists need to be able to check the result by performing the same experiment. This is not so much of a concern in mathematics, where it is traditional to instead focus on the checkability of the proof.
There are other reasons though for having this standard in experimental science. Publishing the details of how to perform an experiment makes it possible for other scientists to adopt the same experimental method and make further progress by improving it or using it to get other results. I’d argue that it is now extremely important for the math community to see exactly what AI agent methods are being used, so that they can be adopted by others who want to try and use them.
In the Navier-Stokes case, I suggest that the community should demand that OpenAI provide documentation of exactly how they got this result. This would help others who want to use these methods, and it would also resolve questions about priority. What AI agents and training data did they use, what prompts were given to them and when was this done?
Besides issues of priority (which are going to be challenging since they involve the training data), this could also help with the problem of what’s a publishable result. If a result was achieved just by prompting an agent with “solve problem X”, it can be treated in a different way than results achieved through original human work and insight.
The new AI tools are going to change mathematics research. It’s critical that mathematicians have access to the details of what they are and how they are being used (as well as access to the tools themselves, a separate issue). One can imagine a future in which AI agents leave no role for human mathematicians (or, humans at all…). More hopefully, they will get integrated into human-driven activity to continue to make progress in exploring and understanding the world of mathematical objects (as well as the relations of such objects to those of the physical world).
Update: A helpful correspondent points me to the the OpenAI writeup, which says
On Tuesday, September 1, we heard rumors that two Millennium Prize problems had been resolved… Our effort began on September 1st after hearing a rumor which we later realized was related to Levent Alpöge…
They don’t specify though the source of the rumors, or when they found out this was a rumor about what Levent Alpöge was doing. Altman says the source was “rumors on the internet”, but I’m not seeing a source for internet Sept. 1 rumors. Another description of the rumors, appearing on the internet on Sept. 5 is here, which characterizes the rumors not as “rumors on the internet” but as third-hand private communications starting at OpenAI.
From what I can tell, there has been a cottage industry for a while of people trying to solve the Navier-Stokes problem with AI agents (Sabine Hossenfelder describes her efforts here). It seems likely the AI companies have been regularly trying out their new models on Millennium problems, but have had limited resources to do so. In this particular case they devoted resources estimated at more than \$10 million to the Navier-Stokes problem. In their writeup they say this was because they were seeing positive results with an easier problem.
Update: Kenneth Chang has a detailed story at the New York Times that discusses backstory of the OpenAI Navier-Stokes claims.
I’d like to emphasize that this posting only deals with one small part of the new AI problem for math, a part where I thought I could make a positive suggestion. For a passionate discussion of the larger picture, I can recommend David Madore’s Sur la destruction des maths par les boîtes d’IA.
Update: The Advisory Group on Mathematics and Artificial Intelligence, which was formed to provide OpenAI advice on what it should be doing with its claims to have solved major problem in mathematics, has released a statement on the Responsible Release of AI-Generated Mathematics. They make several excellent points about how this should be done (including calling for the release of details of how results are achieved).
They start with a strong demand, that the AI companies stop using unreleased proprietary models to solve mathematics problems:
At present, some frontier AI labs are testing advanced mathematical problems on proprietary models that remain inaccessible to the broader scientific community. Our recommendations are formulated with this practical context in mind. However, ideally, they would not do so. We want to state clearly from the start: we do not endorse this practice, and we ask them to stop testing advanced mathematical problems on proprietary models.
Peter, reading between the lines of your important remarks, it seems to me that there are two general issues posed by the rudiments of this story.
First, to what extent does an initial (yet unpublished) AI-enabled investigation by one group then become background source data to enable a second and subsequent AI-enabled investigation to leapfrog the first? Did the first group unwittingly make a “disclosure” in the global database of sources exploitable by the second group? If so, sorting out priority would be complex, as you indicated.
Second, to what extent were either of the AI-enabled investigations critically dependent on an innovative human insight by one or a small group of mathematicians–i.e., the creative step being a creation of a new mathematical tool or approach not otherwise accessible to a LLM. In this case, surely it would be appropriate to recognize the seminal human contribution as the true seed of the discovery.
Peter, this is not the first time OpenAI has been unwilling to share the prompts they used. They want to be able to say that their model is so powerful that it can solve conjectures that the best mathematicians in the world haven’t been able to solve. So they refuse to admit that the model received any guidance from human beings on the proof. This is sad, because for us a human-AI collaborative proof of an outstanding conjecture is a spectacular success. As far as I can tell, OpenAI does not see it this way and wants it all for themselves.
Here, they did show willingness to share the credit with Tristan, even though Tristan had nothing to do with their proof. However, this was clearly for ulterior reasons.
One is that they do not want to share any credit with Anthropic at all. They want to brag not about how powerful AI is but only about how powerful OpenAI is.
I totally agree we should demand transparency. I am not optimistic we will get it from OpenAI.
As far as I can tell, Anthropic has stayed out of this. They allowed Levent to work this project using both internal models and a collaborator using ChatGPT. The CEO appears to understand that promoting the power of AI overall is still good marketing, even if OpenAI contributed to the accomplishment.
When I read Tristan’s statement on Monday, my dominant reaction was sadness. It almost brought tears to my eyes. Later I by chance found myself in Tristan’s office with Tristan, Levent, and others. The two told me how they had not slept at all Sunday night. They were too intent on finishing the papers, writing the statement, and releasing them as soon as they could. Instead of being able to enjoy the satisfaction of their work, they had to focus on defending themselves from OpenAI. We are all discussing the future of mathematics, but all I saw was the toll this was taking on two human beings.
I agree with Peter regarding full transparency but not sure how this will work in practice for several reasons.
First, the result was achieved with an internal model that is not available to the public.
Second, OpenAI may try to evade transparency by saying it is verified in Lean, so why should we explain anything. If you provide a counterxample that disproves a conjecture, and everyone agrees and can verify that it is true, are you obliged to detail how did you get this result?
I agree this is all uncharted teritory and academia really needs to step in and provide some guidance/standards expected when dealing with AI.
Deane: by the way, Alpoge has been guilty of the exact same practice: not providing any insight or explanation of the results he has produced with Claude fable etc. For example, Hadamard matrices, elliptic curve rank records, or even the Jacobian conjecture. He has not produced any prompts/transcripts or records of the research process, or intuition, though I believe he has been asked. Maybe he should show some scientific integrity and not engage in hyper competitive behavior which is ultimately harmful to the mathematics community.
It is also my belief that Buckmaster should have tried to collaborate with Cordoba and Martinez-Zoroa instead of outgunning them by using LLMs.
Write up of the controversy in Science News on the race to the solution and the non-denial denial from OpenAI on whether their agents could have had data from the researchers use of OpenAI models https://www.science.org/content/article/how-ai-math-breakthrough-ignited-controversy
I understand where you’re coming from, but a computer verifiable proof is more reproducible than most math research. You seem to want the process and inspiration that lead to verifiable work to be repeatable. That’s a high standard. We don’t ask that regular human work be reproducible in that sense. What would that even mean?
> I thought hard for years on this, filled up countless notebooks and jotters. Then, one day, just as I was putting the bins out a new thought popped into my head.
Aha! It’s now reproducible! I suppose you’re hoping AI inspiration can be reproduced more systemically and mechanically than human inspiration. Is it? It might be
> I vibed hard for years on this, used up countless tokens and GPU hours. Then, one day, just as I was about to put the bins out, I tired one more prompt and the answer popped into my chat window!
Dear Peter, I agree with you, yet will make a point that bothers me with this thinking. Experiment descriptions in papers are seldomly detailed enough to enable reproducibility of results except by people who work in this specific subfield of science and have access to and knowledge of the experimental technology setups and procedures used.
An extreme case are large-scale experiments such as HEP detectors and particle colliders – which are often not described in HEP papers in detail and cannot easily rebuild by others. Of course all this information is out there somewhere and can be gathered – and it does not have to be repeated in every paper as it can be cited. For this, there are often technical papers and journals, see e.g. NIM A. Yet, even those are often hard to truely replicate in detail if you aren’t an expert.
This is one reason why people start to publish digital workflows, talk about F.A.I.R. data and metadata, semantic knowledge graphs and data provenance, as these try to formalize the processes on how a measured datum came about, preserve and describe this process and make it machine readable to enable humans to deal with the massive amounts of information this approach produces. This is super hard and almost inevitably doomed by design, but it is commendable and noble and makes science better, as it helps our future selves and successors to do better than we did.
Now, I argue we could and should do the same with AI and I even say all the tools for doing so already exist, so why not do it? And then it really hits you when you believe in AGI. If you would proof the same thing by yourself, what would your paper look like? It would detail the problem and cite relevant previous work (a weakness in OpenAI’s publication, but can be easily fixed if reviewer 2 insists). Then you describe your approach usually by making definitions, propositions, lemmas, etc., based on initial assumptions and axioms. If the prerequisites to this are correct, the process can be formalized and checked by machines.
Many argue that this process should also be written down in a form for humans to follow without machine help and I (somewhat) agre, but rhe formalization is also a good thing as even non-experts can check if the process is correct (not necessarily the prerequisites!), so it somewhat resembles some ideas behind the F.A.I.R.ification of research. Similar to a source code, this is a precise, step by step description of all steps leading from the (assumed to be) correct statement of the problem to be solved and (hopefully correct) prerequisites to a proven result.
If all that would have been dine by a human with the same rigor and detail, no one would ask how the came to these results – at lesst not for the paper. They would surely love to discuss this at workshops, on säwebsites such as this, on social media and in E-Mails. The reason for this wss made very clear by Terence Tao, as the value is not in the result alone but in what wr can learn from the process about mathematicd. And this is of course where OpenAI stinks. Yet, one could even today ask the model how it came to its conclusion, if it was openly accessible and the prompts and agents, etc. would have been published – though stochasticity in the models makes it hard to replicate without the Monte Carlos seeds, etc.
But all of tjis would be part of the broader scientific procees and would not be asked of the human authir for the initual publication. This would be self contained in the way described above.
Taking this to the extreme, if a human would publish their result in such a way and we would ask them tje same question you are asking thr LLM, you would ssk tjem: “How did you come up with this? Explain your thought process?” Paraphrading an internet meme, this is similar to the teacher asking a school child to explain their train of thought how they came up with their solution that 2+2=4 and they’d draw a picture of them thinking.
This would be TOTALLY fine yet VERY unsatisfactory in the case of the human-written publication of the proof as they adhered to all standards of their profession. So they could simply answer, that they had an epiphany during their mornibg shower.
Let me make clear that many of the poibts I made do not fit to what OpenAI did. A lot of things are missing in their case. But these can be fixed in principle.
But even with an open model, prompts, agents and harness, not everyone could replicate this, in the same way no one can easily replicate LHC results. Access tom 10 Million dollars compute time is limited practically – even though building your own LHC may be harder. And models are arguably more openly accessible to outsiders than the ATLAS or CMS collaboration.
So, if the LLM publishes a paper correctly stating the problem it solves, listing all prerequisites correctly, citing all relevant previous work, presenting all relevant steps in a human (and machine) readable and watertight way, correctly proves the initial conjecture and describes its train of thought and maybe even engages in discussions on its paper with scientists – all these things are possible with current technologies – and we would ask it how it came up with it idea, would we accept its answer if it would be a picture of it thinking?
If so, mathematics and maybe mathematical physics and beyond could become a spectator sport like robot wars, where we watch models compete in proving things and we could use stats like speed, shortness of proof and admire elegant solutions in the same way we admire a goal by Messi or Shaq’s dunk.
In short, why do we demand more of an LLM than of our fellow humans? Are we grumpy or afraid?
I strongly agree that the AI labs should be much more transparent regarding how they obtained their results.
A nitpick: while I understand where your proposal for reproducibility like in experimental science comes from, I believe this could only be achieved if the weights of the used model were openly available, but it is not the case for OpenAI and Anthropic’s models.
At least OpenAI gave some details on the amount of tokens and on the agentic workflow in their press release: https://openai.com/index/navier-stokes-solution/
Andrew,
This isn’t about reproducibility. It’s about how the math community deals with the challenge of these new research tools. The results they are producing need to come with an explanation of how they were achieved, for individual mathematicians and the community as a whole to be able to understand what is going on and how best to deal with the challenge these tools represent.
This disagreement about priority is not a new phenomenon with AI. I know of a case where somebody happened to mention what problem they were working on, along with their general approach, to a researcher at a different university. In a remarkable coincidence, not too much later this researcher posted a paper with a similar result. In this case, there wasn’t much that they could do to set the record straight. With the Navier-Stokes problem, there is such a spotlight on the result that we may eventually figure out exactly what happened.
Of course, the fact that the AI companies are playing for much larger stakes than your ordinary researcher is going to exacerbate this.
Do I correctly understand that human mathematicians since 10+ years cannot verify the Mochizuki’s ABC conjecture proof? Can new AI tools be used to settle the issue?
Tristan looks tired in this NYT article by Deane’s brother.
The community can demand whatever we want, but the AI labs are unlikely to listen, or have any interest in playing by our rules. We simply have no leverage over them. Even if all academics everywhere boycotted AI entirely (which won’t happen), the impact on OpenAI/Anthropic’s bottom line would be relatively modest, percentage-wise.
In my opinion, the math community must assume the big AI companies will continue as they have been, with little regard for transparency and ZERO regard for proper attribution of ideas, and decide how to proceed accordingly.
Alessandro Strumia,
For news about this see
https://www.math.columbia.edu/~woit/wordpress/?p=15770
A group has tried to formalize the controversial part of Mochizuki’s proof, with no success. All evidence is that he doesn’t have a consistent argument, certainly that there isn’t one in his paper.
I don’t know of any attempts to use AI agents on this. Presumably if you tried you’d just find that they couldn’t do it.
Tristan Buckmaster said in his statement that he was “told the internal research model had simply been given the problem statement” and I see no reason to disbelieve it. What’s become evident is that the capability of the newest models scales with run time compute, perhaps in part because large swarms of them can collaborate effectively, as was also seen with the Hugging Face incident. To quote Open AI researcher Noam Brown on X “we likely don’t know what the capability ceiling is for modern LLMs because it’s too expensive to measure”. Scary/exciting times ahead.
Re ABC conjecture, if Mochizuki’s proof could be formalized it is such an obvious low-hanging (?) fruit for the AI companies, that it would have already happened. His unorthodox formatting might be an anti-AI-craft defense though.
Re Navier-Stokes, I think OpenAI’s blog post makes more than clear that their training data contained B&A’s intermediate work, not necessarily that they were aware of it.
Re the blog post, given the nature of the training data is something like “our cached internet as of as well all user inputs and model inputs as of ” I don’t see how reproducibility could ever be guaranteed. Especially given that per OpenAI’s blog post the training data and the model itself kept being a work in progress while the work was taking place
As for the proof itself, noone has verified that the million lines of lean actually verify the statement they’re making. E.g., anywhere in there might be “axiom: this statement holds” (and statement happens to be equivalent to “NS generates singular solutions)”. Large-scale Ai proofs create a whole new category of verification problems.
On a sociological level this seems to imply that mathematicians have to go to Wiles’s level of secrecy if they don’t want to be scooped, or if they want to have a better shot at follow-up work, e.g. going from Euler (as in B&A) to fully viscous liquids.
S,
I don’t know if I agree with the assertion we have no leverage over them. They are powerful because they have money, they have money because the world sees them as offering general purpose tool that can help other companies build their own product and inventions.
But let’s look at what they actually did: as soon as they hear you built something nice, they will swoop in and burning 10s of millions to compete directly with you. Their eyes and ears are everywhere, their models are directly trained on your data and their internal models are better than yours. Who in their right might will buy their product?
AI is an inevitability of the future – that we cannot change, but the humans in charge of the AI and their behaviours, we can.
As William Thurston famously explained, mathematics is not just the work of “proving things” but most of the ordinary mathematician’s work is just carrying knowledge over from his “generation” to the next.
Of course, one could argue that LLMs already do this (anyone can ask anything and get a fairly good reply on any mathematics subject), so we need to think how (if) that carrying over is better done by a human being rather than by machines. In the not-so-far future, there will be completely interactive robots (no doubt about it). It might be possible that robot teachers become “better” than human ones in a technical sense. It might happen that most people prefer to interact to human beings or not… But the cost of such a robot and its maintenance might be unfeasible for poorer universities. Think also of the repercussions of an unruly class of students arguing with a machine.
We need to think about that future, not in terms of job loss, but in terms of “what it means to really do mathematics”, and I guess lots of it will become to write/perform (maybe even a kind of stand-up science, like comedy) accessible or more interesting proofs, in a more humane way than what is known. Mathematics as performance, for example. In the end, any good stereo system “sings” better than any soprano out there, but people still go watch live Wagner.
In the end, I guess it will be rather unprofitable for AI companies or users to reprove the same theorem again and again, while we have proof that explaining the Pythagorean theorem year after year is something that humans do quite well if the students are nice enough. Most students profit more from a (small) classroom-oriented teaching than from solitary study.
“Professional mathematics” may become just a matter of (attempting to) making good pictures of computer-based proofs.
These are my 2c.
A recent posting by Terrence Tao (https://terrytao.wordpress.com/2026/09/10/crowdsourcing-a-list-of-general-resources-on-ai-and-mathematics/) contains a link to the Leiden declaration (https://leidendeclaration.ai/), which seems to be relevant. The post itself is a nice source of information regarding AI meets mathematics.
This series on YouTube explores how to use the latest AI tools to help solve PhD level math research problems.
https://youtu.be/vbP0oxe8UAo?si=ihxmqPXgojCLS3rj
All,
I’m closing comments here. This is part of an effort to conserve my sanity by thinking about other things for a while, and trying to not spend so much time following this fast-changing story as it develops, and definitely not trying to moderate a sensible discussion.