The announcement yesterday by OpenAI of a solution to the Millennium Prize Navier-Stokes problem has been causing a great deal of discussion among mathematicians. There’s a lot to be said about this story and its implications, most of which I’ll leave to those better informed than me (for one thing, I understand basically nothing about the problem, or its solution).<\/p>\n
While talking to people I started to realize that one thing that really bothers me about these kinds of announcements is that they’ve been coming with no information at all about how the result was achieved. Emphasis has been on the question of whether the proof is correct or understandable, but I now think people should start asking another sort of question: exactly how was this done?<\/p>\n
Sam Altman yesterday explained<\/a> that<\/p>\n we tried this because there were rumors on the internet last week that Anthropic’s models had solved a millennium problem and we were curious if ours could do it too.<\/p><\/blockquote>\n The rumor of this kind circulating was this one<\/a>, with first appearance on September 4. Altman appears to be claiming that OpenAI solved the Navier-Stokes problem over the weekend, starting just with the information that maybe Anthropic had a solution. Altman is famous though for being “unconstrained by the truth.” <\/p>\n Tristan Buckmaster and Levent Alpoge had been working for a year on Navier-Stokes with some AI agent help, see here<\/a>. The rumor I’ve heard is that Levent told colleagues at Anthropic that he and Buckmaster were close to a solution, and that this news somehow got to people at OpenAI, possibly including some information about the methods being used.<\/p>\n The new AI agents have profound implications for mathematics research and I think the math community needs to start discussing specific ways to deal with these. Here’s a modest proposal: the math community needs to adopt a version of the ethical standards of experimental science. If you are using AI agents, you can’t just give a proof (formalized or not), but need to also provide a detailed explanation of how these agents were used to get the result. <\/p>\n In an experimental science you cannot just publish results of an experiment, you also need to precisely describe the experimental setup that produced the results. One reason for this is reproducibility: other scientists need to be able to check the result by performing the same experiment. This is not so much of a concern in mathematics, where it is traditional to instead focus on the checkability of the proof.<\/p>\n There are other reasons though for having this standard in experimental science. Publishing the details of how to perform an experiment makes it possible for other scientists to adopt the same experimental method and make further progress by improving it or using it to get other results. I’d argue that it is now extremely important for the math community to see exactly what AI agent methods are being used, so that they can be adopted by others who want to try and use them.<\/p>\n In the Navier-Stokes case, I suggest that the community should demand that OpenAI provide documentation of exactly how they got this result. This would help others who want to use these methods, and it would also resolve questions about priority. What AI agents and training data did they use, what prompts were given to them and when was this done?<\/p>\n Besides issues of priority (which are going to be challenging since they involve the training data), this could also help with the problem of what’s a publishable result. If a result was achieved just by prompting an agent with “solve problem X”, it can be treated in a different way than results achieved through original human work and insight.<\/p>\n The new AI tools are going to change mathematics research. It’s critical that mathematicians have access to the details of what they are and how they are being used (as well as access to the tools themselves, a separate issue). One can imagine a future in which AI agents leave no role for human mathematicians (or, humans at all…). More hopefully, they will get integrated into human-driven activity to continue to make progress in exploring and understanding the world of mathematical objects (as well as the relations of such objects to those of the physical world). <\/p>\n","protected":false},"excerpt":{"rendered":" The announcement yesterday by OpenAI of a solution to the Millennium Prize Navier-Stokes problem has been causing a great deal of discussion among mathematicians. There’s a lot to be said about this story and its implications, most of which I’ll … Continue reading