Some commenters here a while ago made the excellent suggestion that I should take a look at a book published this spring, Helge Kragh’s Higher Speculations: Grand Theories and Failed Revolutions in Physics and Cosmology. I’ve always wondered what historians of science would make of the increasing dominance of research in fundamental physics by unsuccessful highly speculative research programs, and have also often wondered if there are any relevant historical parallels to this situation. This book does a great job of addressing those questions, and it’s pretty much unique in doing so.
Kragh spends the first half of the book on history, the second half on currently popular (of varying degrees of popularity…) topics including varying constants of nature, cyclic cosmological models, anthropics, the multiverse and string theory. He doesn’t explicitly make any attempt to evaluate how successful these current efforts are, but they are discussed in the context of previous failures and parallels are drawn. I didn’t know much about the history of “vortex theory” in nineteenth century physics, and this turns out to be possibly the best historical parallel to the story of string theory. Here’s an extract from the extensive and enlightening discussion of that bit of scientific history:
From its beginnings in 1867 to its end at about 1900, the [vortex] theory was frequently justified on methodological and aesthetic grounds rather than its ability to explain and predict physical phenomena. In an 1883 review of ether physics, Lodge described the vortex atom theory as ‘beautiful’ and ‘the simplest conception of the material universe which has yet occurred to man’. He added, just as Michelson would do twenty years later, that it was a ‘theory about which one many almost dare to say that it deserves to be true’.
The audience listening to William Hicks’ address at the 1895 meeting of the British Association for the Advancement of Science would not suspect that the vortex theory of atoms was dying. Without paying much attention to the theory’s disappointing record with regard to empirical physics, Hicks reviewed in an optimistic tone the theory of various vortex objects such as rings, spheres and sponges. He realized that relatively little progress had been made over the years in the mathematical development of the theory, and that progress was even more lacking in the theory’s contact with experiments. However, these problems he deftly turned into a defence of the theory, for the undeveloped mathematical framework meant that the theory could not be rigorously tested. Hicks was convinced that the road towards progress would be to develop still more advanced mathematical models. The vortex theory, he said ‘is at present a subject in which the mathematicians must lead the attack’.
Surely many physicists of the day would have described vortex theory as “our best hope for a unified theory”, and one wonders if any of them thought of it as a “part of 20th century physics that fell by chance into the 19th century.”
Kragh’s book does something really remarkable and valuable: it starts to put some aspects of the last 30 years of fundamental physical theory into a plausible historical context. The future of the subject remains a mystery though, but one can hope that on the vortex theory timeline we’re about to hit the analog of 1900, with successful rather than failed revolutions ahead of us.
