I’ve just finished reading Lee Smolin’s new book The Trouble With Physics, which should be released and available for sale very soon. It’s a great book, covering some of the same ground as mine, but with significant differences.
This won’t be a usual sort of review, since I’ll mainly concentrate on discussing the parts of Smolin’s book that I found most interesting, and my perspective here is kind of unique, having spent a lot of time writing about many of the same subjects that he covers. I will offer some capsule consumer advice: if you have any interest at all in what is going on these days in fundamental physics, you should buy and read both books. If you really are on a tight budget, and your main interest is in the relation of mathematics and physics, you should get mine. If your main interest is in quantum gravity or the foundations of quantum mechanics, you should get Smolin’s. His is more appropriate for someone with little background in this area, mine contains some significantly more demanding material which requires some expertise to appreciate.
What most fascinated me about Smolin’s book is the personal story behind it. He was a graduate student at Harvard during the same years that I was an undergraduate there, and describes well that place and time. The standard model had just been formulated a few years earlier, and experimental confirmation was pouring in. Many of the people responsible for the standard model were there at Harvard, and there was more than a bit of justifiable pride and arrogance. Smolin was of a philosophical bent, and initially put off:
The atmosphere was not philosophical; it was harsh and aggressive, dominated by people who were brash, cocky, confident, and in some cases insulting to people who disagreed with them.
He studied the philosophy of science and was very struck by Paul Feyerabend’s Against Method (there are also has some amusing tales of later personal encounters with Feyerabend). Feyerabend’s philosophy of science has been described as “anarchistic”; he sees no one “scientific method”, but science as a very human activity, in which all sorts of different tactics are used to make progress towards better understanding. Smolin recognized that much as he would prefer a more deeply philosophical approach, it was the much more pragmatic tactics of people like Coleman, Glashow and Weinberg, who wouldn’t be caught dead talking about the nature of space and time, or foundational problems of quantum mechanics, that was what was really having success.
Smolin begins his book by explaining what he (and I) see as the most important fact about the past thirty years of theoretical particle physics research. We’re in a historically unprecedented situation, with virtually no progress being made on the fundamental problems of particle physics for a very long time, despite huge efforts. In his description, the field has “hit a wall”; I like to describe it as a victim of its own success. The standard model is just too good. It’s too hard to find an experimental result that disagrees with it, and too hard to come up with theoretical advances that will address some of the things it leaves unexplained. Smolin sees the source of the problem in the field’s insistence on sticking with a way of doing science which worked until 30 years ago, but now has become dysfunctional, with string theory only a symptom of the underlying problem. He writes:
I have mentored several talented young people through crises very similar to my own. But I cannot tell them what I told my younger self – that the dominant style was so dramatically successful that it must be respected and accomodated. Now I have to agree with my younger colleagues that the dominant style is not succeeding.
Elsewhere he writes:
My hypothesis is that what’s wrong with string theory is the fact that it was developed using the elementary-particle-physics style of research, which is ill-suited to the discovery of new theoretical frameworks… This competitive, fashion-driven style worked when it was fueled by experimental discoveries but failed when there was nothing driving fashion but the views and tastes of a few prominent individuals.
Smolin was a student of Stanley Deser’s, and during his graduate student years supergravity was a field that was just taking off. He describes getting to know Peter van Nieuwenhuizen and Martin Rocek and being offered a chance to get into the field at the ground floor, one he passed up because he couldn’t believe that the kind of lengthy algebraic calculations they were doing could give real insight:
It was like being offered one of the first jobs at Microsoft or Google. Rocek, van Niewuwenhuizen, and many of those I met through them have made brilliant careers out of supersymmetry and supergravity. I’m sure that from their point of view, I acted like a fool and blew a brilliant opportunity.
Smolin didn’t join the Stony Brook supergravity group, but found that he could make a place for himself in the physics community working on quantum gravity, but using particle physicist’s methods:
… an easy opportunity opened up while I was a graduate student, which was to attack the problem of quantum gravity using recent methods developed to study the standard model. So I dould pretend to be a normal-science kind of physicist and train as a particle physicist. I then took what I learned and applied it to quantum gravity.
Smolin ended up with a post-doc at the new ITP in Santa Barbara, which luckily was running a program on quantum gravity that year. His career tactic almost didn’t pay off:
One day, as we were waiting for the results of our applications, a friend came by to tell me that I was unlikely to get any jobs, because it was impossible to compare me with other people. If I wanted a career, I had to stop working on my own ideas and work on what other people were doing, because only then could they rank me against my peers.
The most powerful parts of the book are the chapters entitled How Do You Fight Sociology?, and How Science Really Works. They give a detailed and clear diagnosis of the problematic way string theory research is being conducted, and decisions are being made about who deserves a job. Smolin has an insider’s point of view, particularly because he himself worked on string theory:
… during the years I worked on string theory, I cared very much what the leaders of the community thought of my work. Just like an adolescent, I wanted to be accepted by those who were the most influential in my little circle. If I didn’t actually take their advice and devote my life to the theory, it’s only because I have a stubborn streak that usually wins out in these situations. For me, this is not an issue of “us” versus “them,” or a struggle between two communities for dominance. These are very personal problems which I have been contending with internally for as long as I have been a scientist.
So I sympathize strongly with the plight of string theorists, who want both to be good scientists and to have the approval of the powerful people in their field. I understand the difficulty of thinking clearly and independently when acceptance in your community requires belief in a complicated set of ideas that you don’t know how to prove yourself. This is a trap it took me years to think my way out of.
Smolin gives many examples of the “groupthink” behavior of the string theory community, while characterizing string theorists as “almost all more open-minded and self-critical and less dogmatic than they are en masse.” He describes string theorists as:
… supremely confident both of the truth of string theory and of their superiority over those unable or unwilling to do it. To many string theorists, especially the young ones with no memory of physics before their time, it is incomprehensible that a talented physicist, given the chance, would choose to be anything but a string theorist.
…Anyone who hangs out with string theorists encounters this kind of supreme confidence regularly. No matter what the problem under discussion, the one option that never comes up (unless introduced by an outsider) is that the theory might simply be wrong. If the discussion veers to the fact that string theory predicts a landscape and hence makes no predictions, some string theorists will rhapsodize about changing the definition of science.
Some string theorists prefer to believe that string theory is too arcane to be understood by human beings, rather than consider the possibility that it might just be wrong.
Smolin finds in the string theory community a sense of entitlement and disdain for anyone who works on alternatives to the theory, with major string theory conferences never inviting people who work on alternatives to speak. An editor from Cambridge University Press told him that one string theorist said he would never consider publishing with the press because it had put out a book on LQG (I see why their publishing my book was out of the question…). At string theory conferences Smolin would be asked “what are you doing here?” or told “It’s so nice to see you here! We’ve been worried about you.” Some friends explained to him that if he wanted to be considered part of the string theory community he had to work not just on string theory, but on the particular string theory problems that were fashionable at the moment.
One problem for physicists trying to get tenured positions that Smolin mentions is that most universities now require letters from 10-15 people evaluating their work, with a small number of negative evaluations sufficient to sink their chances. If you’re working on something other than a mainstream topic, finding 10-15 people who can comment knowledgeably on your work can be impossible. He describes string theorists as mostly submitting the same two or three research proposals. This narrow concentration on a small number of problems is defended by some senior theorists as a “disciplined” approach, one that will more surely lead to progress than encouraging people to pursue a variety of different research directions.
Very recently, Smolin sees things changing:
Until last year I had hardly ever encountered an expression of doubt from a string theorist. Now I sometimes hear from young people that there is a “crisis” in string theory. “We have lost our leaders,” some of them will say. “Before this, it was always clear what the hot direction was, what people should be working on. Now there’s no real guidance,” or (to each other, nervously) “Is it true that Witten is no longer doing string theory?”
One can quantify this new situation by noting that there have been virtually no heavily cited new papers during the past few years, except perhaps for the KKLT one that is part of the landscape story.
Smolin notes that many string theorists (including himself) have often been ill-informed about the exact state of knowledge concerning crucial conjectures about string theory. One example he discusses in detail is that of the finiteness of multi-loop string amplitudes. The state of the subject is that one knows how to precisely formulate them and can show lack of divergences only up to two loops (this is due to the work of d’Hoker and Phong). At higher genus d’Hoker and Phong have a conjectural definition, but have not yet been able to show that divergences cancel. Few string theorists seem to be aware of this, and some of them react with great hostility and shower with insults anyone who mentions this issue (as I’ve done here on this blog).
There’s much else of interest in Smolin’s book, including a lot of material about what he sees as promising ideas in quantum gravity, discussion of research on the foundations of quantum mechanics, and a chapter on “seers”, people doing original work on foundations. These include ‘t Hooft, Penrose, and many others less well-known.
While I agree with just about all of what Smolin has to say about string theory, my own background is different and I see promise in very different lines of research than he does. I’m much more skeptical than him about our ability to get useful experimental data on quantum gravity, and see questions about quantum mechanics rather differently. My prejudice is that, lacking experimental guidance, the thing to do is to try and better understand the mathematical structures underlying the standard model. In the past, better physical models have gone hand in hand with deeper mathematics, and I’ll bet this will continue to be true in the future. Quantum mechanics has deep connections to representation theory, a part of mathematics that unifies many different subfields. It seems likely to me that a better understanding of quantum mechanics will come from better understanding representation theory and its connections to physics.
There’s a lot of other sorts of material in the book that I haven’t discussed, and I strongly recommend that people read the whole thing. It’s very, very good, and anyone interested enough to follow this blog will find it highly rewarding.
