Not Even Wrong

This is a guest post from Aravind Asok¹. If you have comments about this, you can contact him at asok@usc.edu. We’ll see if there’s some way to later post moderated comments here.

Recently, several symposia have been organized in which groups of mathematicians interacted with developers of various AI systems (specifically, reasoning models) in a structured way. We have in mind the Frontier Math Symposium hosted by Epoch AI and the Deepmind/IAS workshop. The first of these events received more coverage in the press than the second. It spawned several articles including pieces in Scientific American and the Financial Times, though both articles are currently behind a paywall.² Curiously absent from these discussions is any kind of considered opinion of mathematicians regarding these interactions, though hyperbolic quotes from these pieces have made the rounds on social media. Neither of these events was open to the public: participation in both events was limited and by invitation. In both cases the goal was to foster transparent and unguarded interactions.

For context, note that many mathematicians have spent time interacting with reasoning models (Open AI’s ChatGPT, Google’s Gemini, and Anthropic’s Claude among others). While mathematicians were certainly not exempt from the wave of early prompt-based experimentation with initial public models of ChatGPT, they have also explored the behavior of reasoning models on professional aspects of mathematics, testing the models on research mathematics, homework problems, example problems for various classes as well as mathematics competition problems. Anecdotally, reactions run the gamut from dismissal³ to surprise.⁴ However, a structured group interaction with reasoning models provides a qualitatively different experience than these personal explorations. Since invitation to these events was controlled, their audience was necessarily limited; the Epoch event self-selected for those who expressed specific interest in AI,⁵ though the IAS/Deepmind event tried to generate a more random cross section of mathematicians.
Much press coverage has a breathless feel, e.g., including coverage of comments by Sam Altman in, say, Fortune.⁶ It seems fair to say that mathematicians are impressed with the current performance of models, and, furthermore, see interesting avenues for augmenting mathematical research using AI tools. However, many mathematicians view the rhetoric that “math can be solved”, extrapolating from progress on competition-style mathematics viewed as a game, as problematic at best, and at worst presenting a fundamental misunderstanding of the goals of research mathematics as a whole.

Our discussion here will focus on the Epoch AI-sponsored meeting for concreteness, which was not “secret” in any dramatic or clandestine sense, contrary to some reports. The backstory: Epoch AI has been trying to create benchmarks for the performance of various released LLMs⁷ (a.k.a., chatbots like Open AI’s ChatGPT, Anthropic’s Claude, Google Deepmind’s Gemini, etc.).⁸ Frontier Math is a benchmark designed to evaluate the mathematical capabilities of reasoning models. This benchmark consists of tiered lists of problems. Tier 1 problems amount to “mathematical olympiad” level problems, while Tiers 2 and 3 are “more challenging” requiring “specialized knowledge at the graduate level.” Frontier Math sought to build a Tier 4 benchmark of “research
level” problems.

Building the Tier 4 benchmark necessitated involving research mathematicians. Earlier this year, Epoch reached out to mathematicians through varying channels. Initial requests promised some amount of money for delivering a problem of a particular type, but many mathematicians unfamiliar with the source of the communication either dismissed it as not credible or had no interest in the monetary compensation.⁹ To speed up the collection of Tier 4 problems, Epoch came up with the idea of hosting a symposium. The symposium was advertised on several social media outlets (e.g., Twitter) and various mathematicians were contacted directly by e-mail. Interested participants were sometimes asked to interview with Frontier Math lead mathematician Elliot Glazer and also to produce a prospective problem. Mathematics is a fairly small community so many of the people who attended already knew others who were attending; also the vast majority of attendees came from California. Participants did sign a non-disclosure agreement, but it was limited to information related to the problems that were to be delivered. Symposium participants also had their travel and lodging covered, and were paid a \$1500 stipend for their participation.

Participants were given a list of criteria for problem construction; problems must: ¹⁰

1. Have a definite, verifiable answer (e.g., a large integer, a symbolic real, or a tuple of such objects) that can be checked computationally.
2. Resist guesswork: Answers should be “guessproof,” meaning random attempts or trivial brute-force approaches have a negligible chance of success. You should be confident that a person or AI who has found the answer has legitimately reasoned through the underlying mathematics.
3. Be computationally tractable: The solution of a computationally intensive problem must include scripts demonstrating how to find the answer, starting only from standard knowledge of the field. These scripts must cumulatively run less than a hour on standard hardware.

The participants were divided into groups based on field specificity (number theory, analysis, algebraic geometry, topology/geometry and combinatorics) and told to produce suitable problems.

How did participants contextualize this challenge? In mathematics research one frequently does not know in advance the solution to a given problem, nor whether the problem is computationally tractable. In fact, many mathematicians will agree that knowing a problem is soluble can be game-changing.¹¹ Moreover, deciding which problems should be deemed worthy of study can be difficult. As a consequence, by and large, participants did not frame the challenge as one of producing research problems, but rather one of simply producing appropriate problems.

Unsurprisingly, ability to construct such problems varied from subject to subject. For example, one geometer said that it was quite difficult to construct “interesting” problems subject to the constraints. There are also real questions about the extent to which “ability to resist guesswork” truly measures “mathematical understanding”. Many participants were rather open about this: even if AI managed to solve the problems they created, they did not feel that would constitute “understanding” in any real sense.

While most participants had written and submitted problems before the symposium started, few people had an idea at that point of what would be “easy” or “hard” for a model. Most of the first day was spent seeing how models interacted with these preliminary problems, and the subsequent discussions refined participants’ understanding of the stipulation that problems were resistant to guesswork. Along the way, models did manage to “solve” some of the problems, but that statement deserves qualification and a more detailed understanding of what constitutes a “solution”.

One key feature of reasoning models was explicit display of “reasoning traces”, showing the models “thinking”. These traces displayed models searching the web and identifying related papers, but their ability to do so was sensitive to the formulation of the problem in fascinating ways. For example, in algebraic geometry, formulating a problem in terms of commutative ring theory instead of varieties could elicit different responses from a model. However, it is a cornerstone of human algebraic geometry to be able to pass back and forth between the two points of view with relative ease. In geometry/topology, participants noted that models demonstrated no aptitude for geometric reasoning. For example, models could not create simple pictorial models (knot diagrams were specifically mentioned) for problems and manipulate them.¹² In algebraic and enumerative combinatorics, models applied standard methods well (e.g., solving linear recurrences, appealing to binomial identities), but if problems required several steps as well as ingenuity models were stymied, even if they were prompted with relevant literature or correct initial steps.

When a model did output a correct answer, examining the reasoning traces sometimes indicated that happened because the problem was constructed in such a way that the answer could be obtained by solving a much simpler but related problem. In terms of the exam solution paradigm, we would probably say such a response was “getting the right
answer for the wrong reason” and assign a failing grade to such a solution!

Participants were routinely told to aim to craft problems that even putative future reasoning models would find difficult. From that standpoint, it was easy to extrapolate that a future model might behave in a more human way, demonstrate “understanding” in a human sense, and isolate the missing key ingredient. This created a pervasive fear that if reasoning traces indicated models seemed “close now”, then one should extrapolate that the problems would be solvable by future models.¹³ Participants did observe that if literature in a particular domain was suitably saturated, the models could identify lemmas that would be appropriate and generate relevant mathematics. This was certainly impressive, but one wonders to what extent the natural language output affects perception of the coherence of responses: it is easy for things to “look about right” if one does not read too closely! Eventually, participants did converge on problems that were thought to meet the required bar.

The language models that we worked with were definitely good at keyword search, routinely generating useful lists of references. The models also excelled at natural language text generation and could generate non-trivial code, which made them useful in producing examples. However, press-reporting sometimes exaggerated this, suggesting that reasoning models are “faster” or “better” than professional mathematicians. Of course, such statements are very open to interpretation. On the one hand, this could be trivially true, e.g., calculators are routinely faster than professional mathematicians at adding numbers. Less trivially, it could mean automating complicated algebraic computations, but even this would be viewed by most mathematicians as far from the core essence of mathematical discovery.

The participants at the meeting form a rather thin cross-section of mathematicians who have some interest in the interface between AI (broadly construed) and mathematics. The symposium Signal chat became very active after the Scientific American article was posted. Undoubtedly participants felt there were exciting possible uses of AI for the development of mathematics. There are also real questions about whether or when future “reasoning models” will approach “human-level”competence,¹⁴ as well as serious and fascinating philosophical questions about what that even means; this is a direct challenge for the mathematics community. What does it mean to competently do research mathematics? What is valuable or important mathematics?

Finally, there are important practical questions about the impact, e.g., environmental or geopolitical, of computing at this level.¹⁵ All these questions deserve attention: barring some additional as-yet-unseen theoretical roadblock, reasoning models seem likely to continue improving, underscoring the importance of these questions. As things stand, however—particularly when it comes to mathematical reasoning—caution seems warranted in extrapolating future research proficiency of models.

There are some unusual things going on here today. The central university computer system has been having outages since Monday night. No news on the source of these, but this just came in:

CUIT has been working around the clock to restore services following yesterday’s widespread IT outage. As of this morning, many UNI-authenticated services have been restored, and teaching and learning tools are operational, including Courseworks, Zoom, Lionmail, and Google Drive. Additional systems will continue to come back online throughout the day. We will continue to update you with additional information as it becomes available.

In a comment, Alessandro Strumia points to this Twitter post claiming hackers have broken in the Columbia admissions department computers. Just about everything on that site is nonsense, so no reason to believe this, but we’ll see…

I don’t want to try and moderate a discussion here of NYC politics, but will just point out that for the past few weeks, much of the campaign to stop Zohran Mamdani was based on the same kind of attacks as those leveled at Columbia anti-genocide protesters. Shai Davidai is leading an effort to have the IRS go after the student newspaper because it endorsed Mamdani. This kind of thing doesn’t seem to be working, with Mamdani last night decisively winning the NYC Democratic mayoral primary.

Recently I’ve been getting better insight into the point of view of the Columbia trustees. The depressing conclusion I’m drawing is that the reason they’re not fighting Trump is that many of them don’t want to. What I see as a Fascist dictator acting illegally they see as someone governing very much in the interests of the well-to-do, so no threat to them personally. Much of what he’s doing they have no problem with, so why not just go along with it? The taking of grant funding is a big problem for them, but they don’t see any easy way to get it back through the legal system and don’t want to fight with Trump, so they’re focused purely on reaching some sort of settlement with his people (which in practice may be Stephen Miller).

Fox News today has a remarkable opinion piece by one of the trustees, Shoshana Shendelman. It contains a version of various MAGA talking points about how terrible Columbia and other universities are:

Higher education is at a crossroads. As a scientist and a trustee of Columbia University, I have witnessed firsthand how an institution once hailed as a pillar of intellectual progress can be threatened not only by enemies from without, but also by an ideology from within—one that celebrates chaos over creation, grievance over growth, and destruction over discourse…

However, in the spring of 2024, my home was filled with students (my children and many of their classmates) who no longer felt safe on Columbia’s campus. Their fears didn’t stem from intellectual challenge or debate—it stemmed from intimidation. Protesters, many masked, others emboldened by institutional tolerance, took control of the physical environment of the university. They shut down classrooms, buildings and libraries…

The real Columbia—my Columbia—is a monument of advancement shaped by generations of human endeavor. Monuments, however, can be vandalized and dismantled. And these ongoing campaigns have not been acts of civil dialogue; they have been acts of desecration. Of blaming rather than solving. Of tearing apart instead of sewing together. The world is full of destroyers. Their work is quick, loud, and rarely lasting…

Let’s rebuild institutions like Columbia together to ensure that hatred and chaos do not dominate, and everyone feels included and respected. This is our generation’s moment to lead. Building necessitates bravery, and in turn our courage will spread across America. Let’s build again.

Shendelman was the target of this article at the Intercept describing her role at Columbia and problems at her company, Applied Therapeutics. Much of the opinion piece is a personal complaint about this:

After voicing my concerns regarding antisemitism on campus, I became a target of certain media voices determined to erase decades of my work, service, and scholarship with a few calculated, malevolent strokes of a pen.

Up until now, the Columbia trustees have acted in private, keeping discussions of what they are doing confidential. It’s disturbing that one of them is now going on Fox News, presumably with the approval of the rest (I’m finding it hard to believe she would do this without checking with the board) and putting out a picture of the university as threatened from within by antisemitism and an evil ideology, thus needing a house-cleaning and a campaign to make Columbia great again. Yes, there has been an ongoing campaign to destroy a great institution, but going on Fox News with accusations like hers is to join that right-wing campaign. The trustees should make clear that this is not their agenda.

Update: Realizing I didn’t understand what Shendelman sees as the main problem with Columbia now, I reread her piece carefully. Here’s what she says is the main problem facing the university now:

Do we permit this new form of campus intimidation to silence those with opposing views and destroy educational opportunities for an entire generation?

This is very weird. The Trump administration has taken a wrecking ball to the scientific research enterprise at Columbia, thus destroying educational opportunities in science for years to come. It also has launched a campaign to intimidate anti-genocide protesters, with threats of jail, deportation, suspension, etc. So, as an argument that Columbia needs to fight the Trump administration this makes perfect sense. But from context, the “campus intimidation” she is worried about is not from Trump, but from anti-genocide protesters and their supposed antisemitism, which makes her an ally of the Trump campaign against Columbia. I end up completely mystified by who is supposedly silencing who and who is destroying educational opportunities not just for today’s students, but for “an entire generation”. It sounds like it’s the protesters, but the only educational opportunity they’ve destroyed this semester is a few hours of study time in a library.

Since Shendelman says she is writing as a trustee of Columbia university, the trustees really should clarify who it is who is “silenc[ing] those with opposing views and destroy[ing] educational opportunities for an entire generation?” and what they intend to do about it.

Update: What the Columbia trustees should be reading.

First of all, three new books of interest:

• Jesper Grimstrup has written a book titled The Ant Mill: how theoretical high-energy physics descended into groupthink, tribalism, and mass production of research. There’s a lot more about the book at his substack. I’ve contributed a foreword.

  Grimstrup has spent a lot of time thinking about what is going on in the fundamental theoretical physics research community, partly based on his own experiences, together with looking at some data he has gathered and analyzed (with Jarl Sidelmann, see here). Twenty years ago Lee Smolin and I drew attention to these problems, which in many ways have gotten significantly worse since then. I hope Grimstrup’s book gets some people thinking seriously about what can be done.

• Niayesh Afshordi and Phil Halper have a new book out, Battle of the Big Bang: The New Tales of Our Cosmic Origins. If you’re in London this week you can hear them talk about it in person.

  The book covers a huge range of speculative ideas about the Big Bang, including a lot of stories based on Afshordi’s experiences as a researcher in the field. This is a subject I’ve never paid that close attention to, so I learned quite a bit from the book. It’s not a superficial overview, but fairly dense with information.

  While going through all this was interesting, it did leave me reminded of why I’ve never spent much time on these topics. There’s no real role for deep mathematics and the whole business has stayed unfortunately divorced from any successful confrontation with experiment. The situation is kind of like that in particle theory and Afshordi recognizes some of the same problems Grimstrup discusses. There’s a successful standard model of cosmology, but nobody has had any success in going beyond it. One is in danger of doing something more like religion than science in a way that makes me queasy. Afshordi has a lot of discussion of this in the book, and see some slides of his from a talk here.

• A fascinating book about mathematics and thinking, Mathematica by David Bessis, is now available in paperback. For more about this, I’ll refer you to a review by Michael Harris of the original French edition.

In positive news about science funding, there’s yet another new theory center, the Max Planck-IAS-NTU Center for Particle Physics, Cosmology, and Geometry. This is the second new IAS theory center in the past month, last one was the Leinweber Forum for Theoretical and Quantum Physics at IAS.

With US federal science research funding in the process of being drastically cut, an increasingly large fraction of funding for such research will be coming from the Simons Foundation. Their 2024 report is now out, with financial information here. In 2024 they were spending about $300 million in grants (there’s also more grants from the Simons Foundation International).

Update: For an interesting recent review that answers a lot of the questions about cosmology that have always bothered me (and provides a good supplement to the Afshordi/Halper book), see here.

News from here is that as of July 1 I’ll be on a one-semester sabbatical, then will officially retire at the end of the year. This won’t affect that much what I’m actually doing with my time. I’m not moving out of New York and will continue to spend most of my time coming in to the math department to work, but will from now on not be regularly teaching and will mostly give up the administrative things I have been doing for the department having to do with the computer system. In the future I do expect to at some point teach some more courses, but probably not until spring 2027.

In September I’ll be 68, and have always intended to not keep my current position past the age of 70. The reasons for retiring at the end of the year include wanting to focus full-time for a while on research, not wanting to be committed to teaching a course here for now given the disturbing things that have happened (with more likely to come) and the university announcing a financial incentive program for those willing to retire by the end of the year. I’m quite happy with the way this is working out.

I’ll be on vacation in Canada July 3-19. The rest of the summer will be working on trying to write up the progress I’ve been making on the ideas I’ve been pursuing the past few years. The latest version of such a paper focuses not on the twistor stuff but on explaining exactly how “Wick rotation” does something quite unexpected in the case of spinors, opening up new possibilities for unified theories. As always, the problem with writing the paper is that as I write my perspective on the subject changes and I keep changing the conception of the paper. Hopefully this process will converge on a finished version of some kind soon.

During the next academic year I won’t be teaching, look forward to spending a fair amount of time traveling. If you’d like to hear more about the ideas I’ve been working on in person, let me know since I might be near your town sometime during the coming year.

Blogging will continue in some form or other. Much of my recent blogging has been driven by trying to understand what’s going on at Columbia. Unfortunately at this point I’m starting to feel that I understand this all too well, so will continue to report on what’s happening, but spend less time thinking about (and getting disturbed by) it. Will try to find news worth writing about on the math and physics fronts, something that is becoming especially hard to do in physics.

Update: Maybe I should emphasize that, while I’m “retiring” in the sense that Columbia will no longer be paying me a salary and I won’t be teaching a course every semester, I’ll still be spending my time in much the same way as in recent years. The main difference I’m hoping for is to concentrate more on making progress on the research program I’ve been pursuing, so very much not “retiring” from that, quite the opposite.