Not Even Wrong

Roger Penrose’s Fashion, Faith and Fantasy in the New Physics of the Universe is finally being published this week. This is a bit of a landmark event, since the book has a long history, going back to a series of lectures that he delivered in 2003 at the university and IAS. At one point I remember watching videos of these lectures hosted by Princeton here, but these no longer seem to be available.

In giving these lectures, Penrose was walking into the lion’s den, bringing a forceful critique of string theory to the academic institution where it is most popular. Congratulations to Princeton University Press for publishing this despite it challenging the hegemonic viewpoint at Princeton (I had less luck with them: when my British publisher sent them my book for consideration way back when, they hired Lubos Motl to write an evaluation of it…).

Besides a mathematical appendix, the book is divided up into four parts:

• Fashion: This is the section that deals with string theory, and Penrose’s central objection is to the use of extra spatial dimensions as a crucial part of the theory. When trying to use string theory as a unified theory, an assumption is made that one can take four space-time dimensions very large, and the rest very small, decoupling the large and small dimensions. Penrose argues that there is no reason to believe one can consistently do this, that there should be couplings between these degrees of freedom that cannot be ignored, leading to instability of the theory, rather than a stable ground state with large dimensions.

  The problem is that one doesn’t actually have a non-perturbative string theory in which one could properly study this issue. There’s no consistent theory, so Penrose can’t rigorously prove there’s a problem of this kind. He faces the generic problem of arguing not with a well-defined theory, but with people’s speculative hopes of what kind of theory might exist. I agree with him though that the extra dimensions are a deadly problem for the theory. Even if you accept the most optimistic hopes that Penrose’s and other problems will go away, you are still going to be left with the landscape problem. Everything known about conjecturally stable states with 4 large dimensions indicates an infinite complexity of such conjectural things, capable of giving you any physics you want, leaving the theory able to predict nothing and empty of explanatory power.

• Faith: In this section Penrose addresses the measurement problem of quantum mechanics, pointing out correctly that our standard story about quantum mechanics introduces an “ontological shift”, indicating that something more is going on than a well-understood consistent framework. He favors the idea that perhaps the introduction of gravity into the usual framework could resolve this problem, backing this up with a dimensional analysis argument that a relevant effect could come from gravity, while being too small to be observable so far.

  Here I think he does an excellent job of explaining the usual story and why there’s a problem, but personally I’ve never been convinced that this problem requires new physical laws, non-linearities, or the introduction of gravitational effects. To me the “ontological shift” has always seemed due to the standard story being not a full theory of what happens in a real measurement process, but being a phenomenological approximation of what happens, with approximation needed to get a tractable description. As people build and study more complicated and larger fully quantum systems, the inadequacies of the standard story about “measurement” I think will become clearer, and we’ll get a better understanding of how classical behavior emerges from quantum laws, with no need to change those laws.

• Fantasy: Here Penrose describes in detail some basic problems in the theory of cosmology, and how they are supposedly resolved by the theory of inflation. He explains that characterizing this as “fantasy” is not meant to be purely critical, that “fantasizing” about the moment of the big bang is what theorists do in the absence of compelling evidence, and that he just has other fantasies he thinks worthwhile.

  I don’t think I can do justice here to the depth and complexities of his arguments in this section. This is a topic involving subtle questions about the behavior of general relativity where Penrose is one of our deepest thinkers and greatest experts. While acknowledging some of the achievements of inflationary theory, part of his critique is related to that of Paul Steinhardt and others, showing that the theory doesn’t accomplish what it sets out to do, with the exponential expansion not providing a way to get observed homogeneity from arbitrary initial conditions. At the same time there is a lot more there, and this section seems to me that it should be required reading for anyone trying to make sense of fantasies of the description of the big bang itself.

• A new physics for the universe?: In a final section, Penrose describes some of his more positive ideas addressing the problems pointed out in the earlier sections. This begins with a wonderful summary of the theory of twistors, and I strongly suspect that he’s right that this very different way of thinking about space-time geometry will ultimately be part of any successful integration of our understanding of quantization and geometry. That this geometry is very specific to four space-time dimensions provides yet another reason for skepticism about the fashion of theories with more spatial dimensions.

  I’m less convinced by his speculation about quantum state reduction, and by what he refers to as his “Conformal Crazy Cosmology”, although the emphasis on conformal invariance may very well be a correct one.

  In a final “Personal Coda” he explains that he sees himself not as a “maverick”, but as rather embodying an “inner conservatism”, somewhat allergic to the appeals of fashion. In particular:

  So when I heard that string theory – to which I had been distinctly attracted, partly because of its early use of Riemann surfaces – had moved itself in the direction of requiring all those extra spatial dimensions, I was horrified, and far from being tempted by the romantic attractions of a higher-dimensional universe. I found it impossible to believe that nature would have rejected all those beautiful connections with Lorentzian 4-space – and I still do.

A wonderful aspect of the book are Penrose’s many and detailed graphical illustrations, which have been made available separately by Princeton here. At their website you can also read the Preface, and an interview. Unfortunately I’ll be in Germany next week, missing Penrose’s book tour events here in New York, at the American Museum of Natural History and MoMath.

The range of non-crackpot speculative ideas about fundamental physics that normally get much attention is unfortunately quite narrow. In this environment Penrose is a breath of fresh air, providing here a different point of view on several topics, backed by serious and detailed argument. In some ways this is a popular book, but in others it is something else, deserving the attention of experts in the subject. I can’t recommend it too highly to anyone with a serious interest in fundamental questions about physics.

Update: A somewhat different version of this review is up at MAA Reviews.

Cathy O’Neil’s important new book Weapons of Math Destruction, is out today, and if you’re at all interested in the social significance of how mathematics is now being used, you should go out and get a copy. She has been blogging for quite a while at Mathbabe, which you should be following, and is a good place to start if your attention span is too short for the book.

Cathy has had an interesting career path, including a period as my colleague here in the math department at Columbia. She left here to pursue excitement and fortune at a top hedge fund, D.E. Shaw, where she had a front-row seat at the 2008 financial industry collapse. A large factor in that collapse was the role played by mathematical models, and her book explains some of that story (for another take on this, there’s Models.Behaving.Badly from another Columbia colleague, Emanuel Derman). As far as I’ve ever been able to figure out, the role of mathematical modeling in the mortgage backed securities debacle was as a straightforward accessory to fraud. Dubious and fraudulent lending was packaged using mathematics into something that could be marketed as a relatively safe investment, with one main role of the model that of making it hard for others to figure out what was going on. This worked quite well for those selling these things, with the models successfully doing their job of obscuring the fraud and keeping most everyone out of jail.

While this part of the story is now an old and well-worn one, what’s new and important about Weapons of Math Destruction is its examination of the much wider role that mathematical modeling now plays in our society. Cathy went on from the job at D.E. Shaw to work first in risk management and later as a data scientist at an internet media start-up. There she saw some of the same processes at work:

In fact, I saw all kinds of parallels between finance and Big Data. Both industries gobble up the same pool of talent, much of it from elite universities like MIT, Princeton and Stanford. These new hires are ravenous for success and have been focused on external metrics – like SAT scores and college admissions – their entire lives. Whether in finance or tech, the message they’ve received is that they will be rich, they they will run the world…

In both of these industries, the real world, with all its messiness, sits apart. The inclination is to replace people with data trails turning them into more effective shoppers, voters, or workers to optimize some objective… More and more I worried about the separation between technical models and real people, and about the moral repercussions of that separation. If fact, I saw the same pattern emerging that I’d witnessed in finance: a false sense of security was leading to widespread use of imperfect models, self-serving definitions of success, and growing feedback loops. Those who objected were regarded as nostalgic Luddites.

I wondered what the analogue to the credit crisis might be in Big Data. Instead of a bust, I saw a growing dystopia, with inequality rising. The algorithms would make sure that those deemed losers would remain that way. A lucky minority would gain ever more control over the data economy, taking in outrageous fortunes and convincing themselves that they deserved it.

The book then goes on to examine various examples of how Big Data and complex algorithms are working out in practice. Some of these include:

• The effect of the US News and World Report algorithm for college ranking, as colleges try and game the algorithm, while at the same time well-off families are at work gaming the complexities of elite college admissions systems.
• The effects of targeted advertising, especially the way it allows predatory advertisers (some for profit educational institutions, payday lenders, etc.) to very efficiently go after those most vulnerable to the scam.
• The effects of predictive policing, with equality before the law replaced by an algorithm that sends different degrees of law enforcement into different communities.
• The effects of automated algorithms sorting and rejecting job applications, with indirect consequences of discrimination against classes of people.
• The effects of poorly thought-out algorithms for evaluating teachers, sometimes driving excellent teachers from their jobs .
• The effects of algorithms that score credit, determine access to mortgages and to insurance, often with the effect of making sure that those deemed losers stay that way.

Finally, there’s a chapter on Facebook and the way political interests are taking advantage of the detailed information it provides to target their messages, to the detriment of democracy.

To me, Facebook is perhaps the most worrisome of all the Big Data concerns of the book. It now exercises an incredible amount of influence over what information people see, with this influence sometimes being sold to the highest bidder. Together with Amazon, Google and Apple, our economy and society have become controlled by monopolies to an unparalleled degree, monopolies that monitor our every move. In the context of government surveillance, Edward Snowden remarked that we are now “tagged animals, the primary difference being that we paid for the tags and they’re in our pockets.” A very small number of huge extremely wealthy corporations have even greater access to those tags than the government does, recording every movement, communication with others, and even every train of thought as we interact with the web.

These organizations are just starting to explore how to optimize their use of our tags, and thus of us. Of the students starting classes here today in the math department, our training will allow many of them to go on to careers working for these companies. As they go off to work on the algorithms that will govern the lives of all of us, I hope they’ll start by reading this book and thinking about the issues it raises.