A common theme in discussions online of the problems of fundamental theoretical physics is that the subject has gotten “lost in math”, losing touch with “physical intuition”. In such discussions, when people refer to “math” it’s hard to figure out what they mean by this. In the case of Sabine Hossenfelder’s “Lost in Math” you can read her book and get some idea of what specifically she is referring to, but usually the references to “math” don’t come with any way of finding out what the person using the term means by it. Here I’ll mostly leave “math” in quotation marks, since the interesting issue of what this means is not being addressed.
“Physical intuition” is also a term whose meaning is not so clear. Sometimes I see it used in an obviously naive way, referring to our understanding of the physical world that comes from our everyday interaction with it and the feeling this gives us for how classical mechanics, electromagnetism, thermodynamics work. Some people are quite devoted to the idea that this is the way to understand fundamental physics, sometimes taking this as far as skepticism about subjects like quantum mechanics.
Usually though, the term is not being used in this naive sense, but as meaning something more like “the sort of understanding of physical phenomena someone has who has spent a great deal of time working out many examples of how to apply physics theory, so can use this to see patterns and guess how some new example will work out”. This is contrasted to the person lacking such intuition, who will have to fall back on “math”, in this situation meaning writing down the general textbook equations and mathematically manipulating them to produce an answer appropriate for the given example, without any intuitive understanding of the result of the calculation. This is what we expect to see in students who are just learning a new subject, haven’t yet worked out enough examples to have the right intuition.
If the question though is not how to apply well understood fundamental theory to a new example, but how to come up with a better fundamental theory, I’d like to make the provocative claim that “physical intuition” is not going to be that helpful. New breakthroughs in fundamental theory have the characteristic of being unexpectedly different than earlier theory. The best way to come up with such breakthroughs is from new experimental results that conflict with the standard theory and point to a better one. But, what if you don’t have such results? It seems to me that in that case your best hope is “math”.
Here’s a list of the great breakthroughs of fundamental physics in the 20th century, with some comments on the role of “physical intuition” and “math”.
- Special relativity: According to physical intuition, if I’m emitting a light ray and speed up in its direction, so will the the speed of the light ray. The crucial input was from experiment (Michelson-Morley), which showed that light always travels at the same speed. Finding a sensible theory of mechanics with this property was largely “math”.
- General relativity: There’s a long argument about the role of “math” here, but I think the only way to develop “physical intuition” about curved spacetime is to start by learning Riemannian geometry (which Einstein did).
- Quantum mechanics: Here again, a crucial role was played by experimental results, those on atomic spectra. A large part of the development of the subject was applying “math” to the mysterious spectra for which there was zero “physical intuition”. Later on, a better understanding of the theory and better calculational methods involved bringing in a large amount of new “math” to physics, especially the theory of unitary representations of groups.
- Yang-Mills theory: This was pretty much pure “math”: replacing a U(1) gauge theory by an SU(2) gauge theory.
- Gell-Mann’s eight-fold way: Pure “math”.
- The Anderson-Higgs mechanism: The funny thing here is that Anderson did get this out of “physical intuition”, based on what he knew from superconductivity. Particle theorists ignored him (especially when it came time for a Nobel Prize), and their papers about this were often mainly “math”, more specifically argumentation about how the mathematics of gauge symmetry could give a loophole to a theorem (the Goldstone theorem).
- The unified electroweak theory: Looks to me more like “math” than “physical intuition”.
- QCD and asymptotic freedom: David Gross famously had the “physical intuition” that the effective coupling grows in the ultraviolet for all QFTs, based on experience with a wide range of examples. He set a mathematical problem for his student (Frank Wilczek), and when the “mathematics” was finally sorted out, they realized the usual physical intuition for QFTs had to be replaced by something completely different.
Making a list instead of the great disasters of 20th century theoretical physics, there’s
- Supersymmetry: OK, this one is “math”. I suspect though that the problem here is that the “math” is not quite right, but missing some other needed new ideas.
- String theory: As we’re told in countless books and TV programs, this starts with a new “physical intuition”: instead of taking point particles as primitive objects, take the vibrational modes of a vibrating string. Developing the implications of this certainly involves a lot of “math”, but the new fundamental idea is a physical one (and it’s wrong, but that’s a different story…).
Let’s expand some of these a little bit though:
Special relativity–>Logic shows that relativity (the Galilean kind) is in conflict with electromagnetism and Galilei’s transformations, intuition shows the third must go and be substituted by something that respects the first two, math was used to derive that.
General Relativity–>Logic shows gravity must be updated to take special relativity into account and the most naive implementation will break the equivalence principle, intuition says the equivalence principle is fundamental and one must find a way to preserve it, logic and some thought experiments (Ehrenfests’s paradox) say that because of time dilation this inevitably involves Riemannian geometry, maths follows
Quantum mechanics—>Experiment gives funny results, intuition shows that to explain them one needs some seemingly counter-intuitive ansatze (Bohr’s momentum quantization, the photoelectric effect), which are then progressively systematized using more and more sophisticated maths.
Yang Mills/Gell-mann/Electroweak theory—>Intuition shows that perhaps it is worth to extent the techniques we used to classify spin to systems where there seem to be approximate degeneracies analogous to spin (isospin/strangeness), group theory follows.
Intuition also guided the analogy between strong and weak isospin
Asymptotic freedom—->Before Gross there was the parton model, with a mix of intuition and phenomenological maths by Feynman,Bjorken etc showing that “partons” weakly coupled at high Q² exhibit the sort of scaling seen experimentally.
Higgs—>The necessity of renormalizability follows the deeply intuitive work of Wilson, through there is quite a lot of sophisticated maths to connect this to Gauge symmetry.
The result that a theory broken by the Higgs mechanism is still renormalizeable is intuitive (the Higgs is in the IR, gauge symmetry needs to operate in the UV) but needs a lot of sophisticated maths to prove.
In each case intuition has a crucial role in selecting what math to pick. The set of consistent mathematical systems is almost certainly infinitely larger than the set of consistent mathematical systems relevant to physics. So it is an interplay between the two.
The path integral, Feynman diagrams, the Parton model are more physical intuition than math.
Rohrlich, F. The unreasonable effectiveness of physical intuition: Success while ignoring objections. Found Phys 26, 1617–1626 (1996). https://doi.org/10.1007/BF02282125
Abstract
The process of theory development in physics is a very complex one. The best scientists sometimes proceed on the basis of their physical intuition, ignoring serious conceptual or mathematical objections well known to them at the time.The results soon justify their actions: but the removal of these objections is often not possible for a very long time. Four examples are presented: Newton, Schrödinger, Dirac, Dyson. Some thoughts on this “unreasonableness≓ are offered.
Why are you acting like these are mutually exclusive? It’s clear to me that new ideas always come from a combination of qualitative and quantitative reasoning, in support of each other.
Einstein had the qualitative ideas that led to General Relativity in mind long before he learned Riemannian geometry. He also said that “every true theorist is a kind of tamed metaphysicist.” The taming here is the translation of qualitative insight (“physical intuition”) into quantitative form (“math”).
“Steven Weinberg, who was awarded a Nobel Prize for unifying the electromagnetic and weak interaction, likes to make an analogy with horse breeding: “[The horse breeder] looks at a horse and says “That’s a beautiful horse.” While he or she may be expressing a purely aesthetic emotion, I think there’s more to it than that. The horse breeder has seen lots of horses, and from experience with horses knows that that’s the kind of horse that wins races.”
But like experience with horses doesn’t help when building a racing car, experience with last century’s theories might not be of much help conceiving better ones. ”
That’s a quote from my book “Lost in Math” to say I agree with you.
This is a topic close to my heart. It can be called “physics vs formalism”. In my view, there has never been a successful *unguided* venture into theory space (like string theory) . None of the successful examples that you discussed were unguided. Their advent took place within the context of observed problems revealed by experiments, often amidst several competing theories at the time. Even with general relativity: Einstein spent a long time pondering the problem, considering everyday examples (like a man falling from a window) to build up his “physical intuition,” until he came up with the equivalence principle. Only then did he venture into math, guided by this principle.
“Math” v “physical intuition” is surely about choosing a strategy when confronted with a challenging problem in theoretical physics. The choice obviously depends on the background and experiences of the chooser, and history shows that there is no ‘best’ choice. The real issue is the potential (or lack thereof) for a proper interplay between theory and experiment. All the examples of successful theoretical developments, no matter what strategy was adopted, involve experiment, because experiment in physics is the arbiter of ‘success’. In the cases of supersymmetry and string theory, these are unsuccessful either because these approaches make no contact with experiment, or because the ‘predictions’ they have made have simply not be upheld by experiment. The more substantial issue then arises from theorists justifying their continued commitment to an unsuccessful theory by seeking to change the definition of ‘success’.
Doesn’t the canonical history of general relativity involve a rather “naive” form of physical intuition? “I am falling in an elevator”, “I am moving on a rotating disk”, etc.
Pardon me for being facetious but many years ago I decided that “intuition” is just a word one uses for a body of knowledge learned so deeply and so many years ago that one doesn’t recall not knowing it. E.g., much of at least non-rel. QM seems superficially to be intuitive to me–which is patently ridiculous.
This is a fascinating topic but I come down on the other side. Quantum mechanics, SR and GR were all driven by physical observation and intuition. “Math” was the available tool to be used. If math was dominant, Hilbert would have figured out GR.
Nikita,
Yes, it somehow has become “canonical history” that “math” wasn’t important, that Einstein figured out general relativity by using his physical intuition to think through what happens when you fall in an elevator. Enough of this kind of dubious argument about GR. If someone wants to point to a serious discussion of the topic, that fine. Otherwise, enough of that.
After years of talking with physicists, I concluded with some amusement that when they said “physical intuition,” they just meant “guesses of which math to use for physics problems, whenever the guesses turn out to be correct.” 😀
To talk about a somewhat different field, statistical mechanics, Giorgio Parisi won the Nobel Prize in part for inventing the replica method. This is a case where it seems to me that the math is completely broken, but the physical intuition works anyway. But there’s not “no math”. There’s lots of equations, but none of them can be justified rigorously.
It seems to me that papers with few equations, but lots of hand-waving about physical intuition, are likely to be completely wrong.
This is a great idea for a post Peter.
I worry that the term ‘math’ is being used for two very different things: Geometric vs Analytic thinking.
Likely the term ‘Physical intuition” is also being bound to two very different ideas: Constructing Lagrangians vs Unpacking Lagrangians.
And, oddly I think these two subdivisions are mirroring each other. A mathematician trained in Geometry is often much closer to a Field Theorist trying to develop new Lagrangians than (s)he is to most hard core analysts. There are very things that separate mathematical from physical intuition in such cases but there are a few. In particular the concept of what is geometrically natural is slightly different to what is physicalliy natural in a Lagrangian. The “Mexican Hat Potential” quartic potential is obviously geometrically awkward as is the Yukawa Coupling as are the CKM/PMNS matrices. They come from experiment, but they don’t correspond to natural fiber bundle theory that mathematicians would stumble upon. Yet they are natural enough to a field theorist who might not see why the Yang-Mills term is incredibly more geometrically natural because the field theorist may believe that renormalizability is the proxy for naturality.
Conversely, the mathematical analyst might be more focused on keeping control of infinities in mathematical work while the perturbation/regularization/renormalization concerns mirror those in analysis need to keep infinities from spoiling everthing. Yet this is often about unpacking Lagrangians rather than constructing them.
This leads to two related but distinct concepts of elegance and beauty. I have never understood the argument that String Theory is beautiful…but at least I’d like to think I know WHY I don’t understand it. It is because it is largely based on a mathematics problem that doesn’t strike me as the way forward: “How do we treat the metric so that it doesn’t blow up or misbehave when added to the field content to be quantized?” That is really an analysis question to my mind which drags in a lot of geometry kicking and screaming. So you end up with beautiful geometric objects like Calabi-Yau manifolds but for very weird and often ugly reasons.
Ultimately, I think that what we are struggling with here is four quadrants: geometry, model building and phenomenology, QFT, and hardcore analysis on manifolds. We are looking for beautiful geometry to become a compelling natural Lagrangian. That is two quadrants. Then it will need to be unpacked by QFT types to discern what degrees of freedom would be observed at our effective energy scales. And that in turn would need to be made into rigorous analysis to be fully satisfactory. What I don’t understand is why those of us who base oursevels in one of these quadrants or the other too often try to discount those in the other three. It just seems totally self-defeating. One man’s opinion.
Perhaps a slight aside, but as far as I can tell “the math” means anything one does not have a really good intuition about.
And it’s not just physicists who talk this way either. Ask any (well-educated) person on the street what an average is and they’ll explain it in plain English with clarity and precision. Ask them about standard deviation and you’ll get a slightly muddier answer.
Ask them about the third moment of a random variable, and they’ll tell you that for something like that you gotta do the math.
So “math” seems to be the psychological state where one starts manipulating equations with pen and paper and knowledge in the hands begins to take over from knowledge in the mind.
Just my theory.
Robert James Parkinson,
I put “math” in quotation marks because I think different people mean different things by it. That it is often getting used by non-mathematicians to mean “stuff I don’t really understand” sounds right.
I like a lot Scott Aaronson’s observation that “physical intuition” often doesn’t mean a non-mathematical understanding of a problem but does mean knowing what the right mathematical understanding of the problem is. To me the big problem of finding a way forward in fundamental theory is exactly that of finding the right mathematics. Mathematics is a set of languages and tools for using these languages, we need to identify which of these is needed to make progress. Eric Weinstein’s comment is along similar lines, with a specific proposal, mine would be different.
Wow this is a wildly out of touch point of view of how physics is done.
To a physicist, “the math” means doing a formal calculation from first principles. There’s often an implication that the calculation is being done with a high degree of generality.
Working physicists often think it’s not a good idea to only use “the math” when doing real world problems. There’s a famous quote by Wheeler: “Never make a calculation before you know the answer.” There are a variety of *heuristic techniques* that physicists use to get a feel for the answer, including:
* Dimensional analysis
* Knowledge of relevant experimental results
* Experience with related calculations
* Knowing that the calculation must reproduce some known result as a special case
* Toy models
In fact, I don’t think this heuristic mode that physicists use is particularly different from what mathematicians use; I think it is quite analogous to what Terry Tao refers to as the “post rigorous” stage of mathematics: https://terrytao.wordpress.com/career-advice/theres-more-to-mathematics-than-rigour-and-proofs/
Trying to find “a general solution” to some equation without a *physical question* motivating the endeavor is a classic path to getting lost in equations and not deriving insight from them.
In terms of finding new theories, I think all of your examples can be reframed in terms of a physical motivation. For example, Einstein’s paper on special relativity starts by talking about how it is strange the current produced in a coil of wire moving relative to a magnet has two apparently different explanations in two different reference frames. This motivated him to take equations discovered earlier by Poincaire and others (whether or not he was actually aware that they were previously discovered) and *interpret* them physically in terms of various thought experiments, making them manifestly relevant for physics. Of course math still has a role, especially when it expands our conceptual understanding — like when Minkowski reinterpreted the Poincaire transformations in terms of spacetime geometry. But even then, the benefit of Minkowski’s work was not blindly applying a formalism, but developing new ideas that allowed us to reinterpret existing results in a new framework where different intuition can be applied.
In reality, research is messy, and I think all tools and points of view are needed. But I think the way your article is framed that all discoveries in the 20th century are based on “math” and string theory is based on “physical intuition” is unabashedly political and not a serious attempt to understand the philosophy of science.
Gavin Storm,
I think you’re missing the point of my posting. It was specifically an intentionally provocative argument against the idea that, in a situation where current theory doesn’t apply and you don’t have any experimental guidance, the way forward is “physical intuition”, not “math”. I often hear this in discussions of the current lack of progress in fundamental physics (“the problem is that the string theorists are doing math, not using physical intuition”), and I think it’s completely wrong-headed.
You’re arguing about something completely different, about whether one should start trying to solve a problem with heuristic methods or with a rigorous formal calculation. Of course both mathematicians and physicists generally start with the first, later move on to the second.
My argument is about what heuristic method you should use. Should you look to the mathematical tools you have successfully used in the past (your built up “physical intuition”) or should you look for new tools? The best example of the ones I gave in the posting might be the Gell-Mann one. He and everyone else stared a long time at the patterns of strongly interacting states they were seeing, not recognizing any pattern they were used to or anything that fit the mathematical tools they had in hand. He finally got around to talking to a mathematician who told him about the representations of SU(3), giving him a new mathematical tool that explained the patterns.
In the context of string theory, some people argue that it failed because, seduced by Witten, string theorists went out and adopted all sorts of new mathematical tools not previously useful in physics. I think it failed for a different reason: it was a completely wrong heuristic idea based on picking a wrong physical system (the vibrating string) as starting point.
I started my career as a theoretical high energy physicist and am now an experimental biological physicist. I have published in major refereed journals in both fields. I find Jim Baggot’s comment above to be especially relevant and cogent. A simple question is always “What are the experimental data?” that either do or do not validate any “conclusion(s)” coming from math or physical intuition. Experiment is always the final arbiter in science although math, physical intuition as well as experiment can interact along the way to a final determination. Peter your last entry gives some very cogent examples relevant to both Baggot’s comment and what I try to express.
Bob Y,
The question I’m interested in is that of research in areas where there is no experimental evidence to interact with. Should theoretical quantum gravity research, for instance, be described as “not physics” because there is no experiment to interact with?
The thing that really bugs me is people acting as if mathematics has nothing to do with intuition, therefore if there is any intuition at all, it must be physical intuition.
I’ve thought about this often, mostly in bafflement before I realized other people were defining ‘math’ very differently from me. To me, mathematics is just being incredibly careful and precise with your logical relationships (as even ‘true/false’ can be mapped to 0/1, partially true to some other number or a tensor, etc), so that even if you use different definitions from other people they’ll still be able to follow your steps *because you defined your terms*; like how ‘field’ can mean two different technical things, the mathematical kind and the physics kind.
Intuitions about geometry are inherently intuitions also about math, they’re not two separate things. The problem is the space of possible geometries in math is much wider than in physics, so usually, some guidance from the real world is useful.
However, this will break down in domains where we cannot even in principle do an experiment. From there, we will need mathematics: specifically, metamathematics of how different axiom systems relate to each other and why one might get ‘picked out’ as the more ‘inertial’ or ‘equality preserving’ of the two. But what I say here by ‘mathematics’ and what someone else hears when I say it may be two different things, and lead to an ugly and ultimately pointless argument: if you think mathematics is only calculations, you are not going to be thinking about all the work (axioms) that has to go in before you ever get to calculations so much.
If theoretical quantum gravity research is “not physics”, it’s hard to see what else it could be. It certainly isn’t rigorous enough to be called “mathematics”. I suppose a case could be made for it being “nonsense”, but I think this is much too pessimistic (although it’s true for some current theoretical quantum gravity research).
Thank you for promoting and sympathetically moderating such an interesting and wide ranging discussion, as you previously did on Sabine Hossenfelder’s piece about academic life.
First, intuition. I see this as a subconscious, unwritten and often unverbalised collection of prior knowledge which can be called on when action is needed and there is no specific data and no time to collect data. The more specialised the individual and the more typical for him or her the problem, the more reliable it becomes.
It is not reliable in new situations, merely a guide to a possible starting point. But in research situations it may save a lot of time by suggesting a first line of attack (e.g. an appropriate mathematical framework) based directly or subconsciously on previous experience. The analysis and calculations still need to be done and the approach may be unsuccessful. In this sense, it applies to math as well as physics – there are many famous conjectures which have been disproven.
On your examples, first, special relativity. Einstein’s original 1905 paper on special relativity was entitled (in translation) “On the electrodynamics of moving bodies”. Einstein’s father ran an electrical business and the young Albert was fascinated by Maxwell’s Equations, later saying that his predecessor was not Newton but Maxwell. His intention was to resolve a problem of asymmetry in the application of the equations. Special Relativity is just Part I of the paper.
He also refers but only in passing to the failure of experiment to show “motion of the earth relative to the light medium”. He later said he knew of the M&M experiments but had not read the paper. As to math, there is nothing in his paper beyond the reach of a high school math graduate and the basic calculations are simple.
Next, general relativity. This is far more complex story and impossible to summarise briefly and accurately. The first steps taken by Einstein and others were the extension of special relativity to include special cases of accelerating frames and objects which do not require general relativity. Einstein wished to extend it to arbitrary accelerating frames and gravity and proposed using the principle of general covariance, which extends invariance of physical laws to arbitrary differentiable coordinate transformations and is in many ways was a natural extension of SR.
He did not have the mathematical knowledge to do this and sought help from an earlier classmate and friend, Grossmann, who pointed him towards tensors and the Ricci tensor in particular and assisted him with the math. But Grossmann believed, in error, in 1913 that the resulting mathematical framework for general relativity did not lead asymptotically to Newtonian gravity and they went round in circles for two years before returning to the 1913 tensor approach in 1915.
The basic mathematical framework needed was already in existence, but Einstein needed Grossmann to point it out, and even then, there were major difficulties in its application and finding solutions in practical circumstances, with initially erroneous predictions for key tests.
Leaving aside quantum mechanics and particle physics, your final question is whether research should be described as “not physics” if there is no experimental evidence. My view is quite conventional – research should be either explanatory of existing phenomena (i.e. based on data, even if only qualitative) or should lead to testable predictions. Its mathematical basis should as far as possible be well founded; this is usually the case, the problem is finding the right structure and adapting it to the application. Both special and general relativity did just that.
Peter,
Referring to your exchange with “Nikita”, I don’t see why the involvement of a naïve form of physical intuition in the history of general relativity would imply that math wasn’t important.
In this context, the essential thing to me is to identify the spark that inspires the pursuit of a theory. It may very well be naïve physical intuition even though the formulation of the theory itself requires a lot of math, possibly even novel mathematical ideas.
I don’t know if the following counts as pointing to a serious discussion of the topic, as you requested, but I’ll give it a try. Sparks of inspiration are personal, so we have to listen to the physicists themselves.
Einstein described in a lecture in Kyoto in 1922 how “the luckiest thought” of his life came to him in 1907 (Physics Today 35(8) pp. 45-47, 1982):
“The breakthrough came suddenly one day. I was sitting on a chair in my patent office in Bern. Suddenly a thought struck me: If a man falls freely, he would not feel his weight. I was taken aback. This simple thought experiment made a deep impression on me. This led to the theory of gravity.”
Similarly, special relativity had been brewing in Einstein’s mind since his teenage years (Autobiographical notes, Open Court, 1949):
“After ten years of reflection such a principle resulted from a paradox upon which I had already hit at the age of sixteen: if I pursue a beam of light with the velocity c (velocity of light in a vacuum), I should observe such a beam of light as a spatially oscillatory electromagnetic field at rest. However, there seems to be no such thing, whether on the basis of experience or according to Maxwell’s equations. From the very beginning it appeared to me intuitively clear that, judged from the standpoint of such an observer, everything would have to happen according to the same laws as for an observer who, relative to the earth, was at rest. For how, otherwise, should the first observer know, i.e., be able to determine, that he is in a state of fast uniform motion? One sees that in this paradox the germ of the special relativity theory is already contained.”
When it comes to the development of new fundamental theory, I would say that such sparks or general ideas are necessary. It is not enough to search for a mathematical framework that can accommodate new experimental results, it seems to me.
Michelson and Morely did their experiments in the 1880s, and mathematical efforts to account for the lack of aether drag began immediately, via Fitzgerald contractions and the like. However, Einstein’s relativity postulate was necessary to jump from ad hoc theories to a new mechanics based on general principles that allowed the derivation of a lot of other, seemingly unrelated results.
I would say that the history of quantum mechanics is similar. Decades of experimental results that deviated from the predictions of classical physics were accounted for in an array of phenomenological mathematical models. However, it was not until the work of Heisenberg and Schrödinger that a single, coherent theory emerged that could be used to derive a lot of new predictions. And Heisenberg and Schrödinger both seem to have been inspired by simple, general ideas.
The abstract of Heisenberg’s groundbreaking paper from 1925 reads in its entirety (Über quantentheoretische Umdeutung kinematischer und mechanischer Beziehungen, Zeitschrift für Physik 33(1) pp. 879–893, 1925):
“The present paper seeks to establish a basis for theoretical quantum mechanics founded exclusively upon relationships between quantities which in principle are observable.”
He goes on to write:
“It is well known that the formal rules which are used in quantum theory for calculating observable quantities such as the energy of the hydrogen atom may be seriously criticized on the grounds that they contain, as basic element, relationships between quantities that are apparently unobservable in principle, e.g., position and period of revolution of the electron.”
Schrödinger had a different angle of attack. He was inspired by de Broglie’s matter waves and seems to have fallen in love with the idea that continuously interacting waves is the substrate for the entire physical world. In the last paper before those that define wave mechanics, Schrödinger wrote (On Einstein’s gas theory, Physikalische Zeitschrift 27 pp. 95–101, 1925):
“This means nothing other than to be serious about the De Broglie–Einstein wave theory of moving particles according to which these are nothing but a kind of ‘foam crest’ on a fundamental radiation wave.”
The relativity postulate is a metaphysical principle that goes deeper than the apparent constancy of the speed of light. Heisenberg’s strict epistemic ansatz in his 1925 paper predicates and goes beyond the choice to express mechanics by means of non-commuting matrices. Schrödinger’s (presumed) idea that ‘everything is waves’ is more far reaching than the Schrödinger equation.
I think new such general ideas are needed to spark further development of fundamental physics. They don’t have to be entirely correct to be fruitful. Whether Heisenberg’s epistemic approach is appropriate is a matter of philosophical taste. Schrödinger’s vision of waves was clearly insufficient. Newton’s idea that space and time are absolute is wrong, but it was nevertheless immensely fruitful, of course, since it allowed the expression of the relations between all objects by means of their individual coordinates, the evolution of which could be expressed with differential equations.
Could the same be said about string theory? Could there be something to its mathematical structure even though the basic physical intuition about the “vibrational modes of a vibrating string” is misguided? If so, could the theory have been developed without this idea?
Sorry for the length of this comment, and for repeating lots of standard material. I guess my point is to defend “physical intuition”. However, I would like to rephrase it as “metaphysical intuition”, meaning intuition about which general principles describing the physical world are fruitful guides in the formulation of fundamental physics.
“Metamathematical intuition” is surely also needed, understood as intuition about which mathematical structures and requirements are fruitful for the same purpose. Dirac’s equation seems to have been the result of such intuition, for example.
All theories in physics, including those mentioned in the post, are based on intuition, almost by definition. It is not possible to develop a correct and entirely new physical theory purely through rigorous formal derivation without incorporating one’s perception of the phenomenon in question. Importantly, this perception pertains to one’s understanding of the phenomenon rather than the phenomenon itself. When developing a new hypothesis, one’s understanding cannot be guided by the not-yet-existing theory and, therefore, must be intuitive.
I (respectfully) believe that the examples provided by the author are misinterpreted and treated in an ahistorical manner. The most striking example is that of special relativity. The Lorentz transformation and related mathematical formalism were indeed derived years before Einstein’s article through symmetry-based considerations of Maxwell’s theory. However, it was Einstein’s understanding of the important (non-mathematical) principle of relativity, particularly his intuition of time and distance as measured with clocks and rulers, that constituted the actual breakthrough. This breakthrough (a) changed our conceptual understanding of reality and (b) inspired further development of vast mathematical formalism beyond the Lorentz transformation. Notably, Einstein’s discovery did not rely on the Michelson-Morley experiment, as the validity of the principle of relativity was not dependent on the existence of aether.
I believe the greatest breakthrough in our understanding of nature came from a physicist who had limited knowledge of and mistrusted mathematics: Sir Michael Faraday. Faraday introduced the concept of a “field” purely through intuition, departing from the idea of action-at-a-distance along straight lines (Newton’s description of physics with ordinary differential equations) and instead proposing a “medium” and curved “force lines.” This led to Maxwell representing physics with partial differential equations. Theorists often overlook that field theory, the foundation of our understanding of fundamental forces, was introduced by Faraday through his meticulously conducted and brilliant experiments on electromagnetism. Faraday’s discovery process contradicts the author’s statement: “The best way to come up with such breakthroughs is from new experimental results that conflict with the standard theory and point to a better one.” Faraday did not have a “standard” theory to compare to, nor did he attempt to understand existing mathematical theories. Instead, he built a consistent conceptual framework of electric and magnetic force by formulating hypotheses about reality and testing them experimentally. He was able to extrapolate his non-mathematical ideas to make claims about the electromagnetic nature of light without experimental methods to prove it or any mathematical formalism. Learning about Faraday’s story could teach many theorists a valuable lesson in humility and help prevent unsound philosophical claims about the connections between mathematics and reality.
There is no reason to believe that a story similar to Faraday’s can not happen again in the XXI, XXII, or XXIII century, despite “big science” and “mathematics” being widely considered the only way of ensuring scientific progress.
Sebastian,
This is exactly the point of view that I am arguing against. Given the conceptual and mathematical sophistication of our theories of modern physics, there is zero chance that they are going to be improved by a a new Faraday as you suggest.