Some new items mostly updating older ones:
- Natalie Wolchover has a very good article at Quanta, entitled A Fight for the Soul of Science, reporting on the recent Munich conference discussed here. David Gross sounds a little bit like John Horgan, emphasizing the problem of HEP physics getting too difficult, with an “End of Science” danger. I think he has the problem right:
“The issue in confronting the next step,” said Gross, “is not one of ideology but strategy: What is the most useful way of doing science?”
I hadn’t realized quite how radical Dawid is. He seems to have moved on from discussing theory “assessment” to theory “confirmation”. Even the most devoted string theorists like Gross may be unwilling to sign on to this, comfortable with the idea that string theory deserves a positive assessment, as a promising idea still worth working on, much less so with claims from Dawid that one can sensibly discuss string theory as a “confirmed” theory, one that belongs in our school textbooks.
There was much discussion evidently of Bayesian confirmation theory, and Gross was enthusiastic about this
Gross concurred, saying that, upon learning about Bayesian confirmation theory from Dawid’s book, he felt “somewhat like the Molière character who said, ‘Oh my God, I’ve been talking prose all my life!’”
He may have become a bit less enthusiastic later when faced with Joe Polchinski’s Bayesian calculation showing a 94% probability confirming the multiverse.
Sabine Hossenfelder and Carlo Rovelli both explained well the danger of such claims of non-empirical Bayesian confirmation of one’s ideas:
The German physicist Sabine Hossenfelder, in her talk, argued that progress in fundamental physics very often comes from abandoning cherished prejudices (such as, perhaps, the assumption that the forces of nature must be unified). Echoing this point, Rovelli said “Dawid’s idea of non-empirical confirmation [forms] an obstacle to this possibility of progress, because it bases our credence on our own previous credences.” It “takes away one of the tools — maybe the soul itself — of scientific thinking,” he continued, “which is ‘do not trust your own thinking.’”
- For more on the “non-empirical science” front, see Alan Lightman’s long piece in Harper’s on “Quantum Cosmologists” Sean Carroll, Alexander Vilenkin, James Hartle and Don Page. Lightman waxes poetic on the importance of “The need to ask the really big questions”, but unless I missed it, there’s nothing there about the need to provide any evidence for the answers that one comes up with. At least one of the four quantum cosmologists, Don Page, seems to see no particular distinction between theology and science.
- On the Mochizuki front, there’s this report in Nature about the Oxford conference. On the question of what went wrong with the later talks
But Conrad and many other participants say they found the later lectures indigestible. Kim counters that part of the difficulty lay in cultural differences: Japanese mathematicians have a more formal style of lecturing than do those in the West and they are not as used to being questioned by a testy audience, he says.
I don’t think the “Japanese culture” explanation of the problem holds water. Of the three problematic speakers, one (Mok) is not Japanese at all (from Hong Kong), and the other two certainly understand that explaining mathematics to someone is about more than reading a lecture to an audience. It’s not plausible that the reason they didn’t have satisfactory answers to questions from the audience is their Japanese cultural background. For the case of Mochizuki himself, he grew up here in New York, went to prep school, undergraduate and graduate school in the US. The question of why following him is so difficult is a fascinating one, but I don’t think the answer to it has anything to do with his choice to move to Japan.
A must-read detailed report on the situation is Brian Conrad’s, available here.
Update: For some other commentary on the Munich workshop and relevant issues, see Sabine Hossenfelder and my Columbia colleague Andrew Gelman (who I think, unlike anyone in the theoretical physics community, actually knows something about Bayesian methods).
Update: Fesenko has a comment at the Nature article, where he makes the claim: “There are no questions about the theory which are left unanswered.” I agree with my Columbia colleague David Hansen’s response to this, that this seems to be an absurd statement.
Update: There’s a very good report on the abc conjecture workshop from Kevin Hartnett at Quanta. His take on what happened agrees with others:
Kedlaya’s exposition of Frobenioids had provided the assembled mathematicians with their first real sense of how Mochizuki’s techniques might circle back to the original formulation of Szpiro’s conjecture. The next step was the essential one — to show how the reformulation in terms of Frobenioids made it possible to bring genuinely new and powerful techniques to bear on a potential proof. These techniques appear in Mochizuki’s four IUT theory papers, which were the subject of the last two days of the conference. The job of explaining those papers fell to Chung Pang Mok of Purdue University and Yuichiro Hoshi and Go Yamashita, both colleagues of Mochizuki’s at the Research Institute for Mathematical Sciences at Kyoto University. The three are among a small handful of people who have devoted intense effort to understanding Mochizuki’s IUT theory. By all accounts, their talks were impossible to follow.
There’s also a report now from Fesenko, available here. His take on this is that the problem wasn’t the talks, it was the audience:
Labor omnia vincit. Progress in understanding the talks correlated with preparation efforts for the workshop. There were participants who came unprepared but were active in asking questions, many of which were already answered in recommended surveys and some of which were rather puerile.
Unclear what the point of such remarks is, unless the goal is to make sure that many of the experts who attended the workshop won’t come to any more of them.
Update: The paragraph from Fesenko’s report on the workshop quoted about has been removed, replaced by
Без труда не выловишь и рыбку из пруда. Progress in understanding the talks correlated with preparation efforts for the workshop. Lack of reading of non-classical papers of the theory often correlated with the depth of asked questions.
Update: Nature covers the Munich conference as Feuding physicists turn to philosophy for help.
Update: The Fesenko report has been further edited, with the paragraph mentioned above now reading
Без труда не выловишь и рыбку из пруда. According to the feedback, progress in understanding the talks and quality of questions often correlated with preparation efforts for the workshop and reading of non-classical papers of the theory.
Update: Michael Harris’s take on the press coverage of the Oxford conference is here.
Well…I was recently in a video conference with some potential Japanese collaborators, and it was impossible to get answers to all but the most straightforward and anodyne questions. Incredibly awkward. The translator sometimes would just sit there, fidgeting slightly, apparently so abashed by what we were saying she couldn’t speak it aloud in Japanese. You could tell how well the men (all men) around the table could understand English, though, by how red their faces were getting with whatever mix of mortification and rage they were attempting to conceal beneath stone-faced stoicism. I’ve worked with loads of Japanese researchers who’ve done a lot of post-graduate or post-doctoral work in the West and were fairly acclimated. Never had a problem. But perhaps without some amount of pre-exposure, there can be substantial cultural barriers to useful dialog, and a lot still gets “lost in translation”. Or not translated at all!
Bayesian confirmation theory is very, very popular in philosophy right now, especially in epistemology. My guess is that this is mainly because (i) it is simple and often lots of fun to play around with, (ii) it is mathematical and therefore rigorous in at least one way, and (iii) you can use it to prove more or less anything.
Dawid’s treatment of the no-alternatives argument (published in a paper in the British Journal for the Philosophy of Science this year, written together with Hartmann and Sprenger) is a good example of (iii). Given the model (or “framing”) they use, and their assumptions about the priors, the argument is certainly valid. But the model is just one possibility out of countless, and while most of their assumptions sound prima facie reasonable in that specific model, other models would give quite different recommendations.
Peter,
Regarding Lightman’s piece, I got the same impression you did–he seems to think that when asking “the really big questions” one needn’t bother with meaningful searches for evidence for the answers that one comes up with. But that said, I don’t think Don Page is blurring science and theology. As a Christian, he believes the hand of God is manifested in the workings of nature and that fills him with wonder. But rest assured he is quite clear that this is a metaphysical stance well outside the realm of his science. It certainly hasn’t kept him from contributing to quantum cosmology research–he’s a leader in the field.
I believe his comments are in response to Sean Carroll and Lawrence Krauss (also mentioned in the essay), both of whom are outspoken atheists who actually have made a point of publicly blurring the distinction between science and theology (or “a-theology” if you will), and on many occasions. Page is simply setting the record straight by pointing out that his faith coexists quite nicely with his quantum cosmology research–that science and theology are separate but complementary realms that are neither incompatible, nor mutually exclusive, and they shouldn’t be encroaching on each other’s turf. If anything, it is he who is who is clear about this rather than the others.
Scott Church,
I don’t think Page always so cleanly separates his science from his theology, see for instance
http://arxiv.org/abs/1412.7544
I see that Carroll has just finished writing a book: “ON THE ORIGINS OF LIFE, MEANING, AND THE UNIVERSE ITSELF”, which starts out with the fundamental nature of reality and ends up with a code of ethics. This strikes me as designed to compete with the great religious texts, although presumably in the parts not quantum cosmology he is often invoking well-grounded scientific facts.
Thanks Peter! I hadn’t seen that paper of Page’s, but I plan on reading it more closely soon as I get a chance. At first blush it looks as though he’s attempting to argue via inference to the best explanation in both scientific and philosophical areas, invoking (among other things) well-grounded facts wherever they seem relevant. I haven’t seen Carroll’s book yet, but based on what you said I get the impression he’s trying to do the same.
Part of the problem with this sort of thing is that fields like quantum cosmology unavoidably take us into realms where the line between physics and metaphysics is easily blurred and it becomes increasingly difficult keep the two separate, and all the more important that we do so. This is a problem because (in my experience at least) not many physicists are well-trained in philosophy–particularly philosophy of science and religion which is what’s really needed for that. Some folks do cross the line from time to time, but I don’t think that necessarily means they aren’t doing their best to appropriately honor it. More than one physicist of late has suggested that perhaps it’s time for more dialog between physicists and philosophers, and I for one, agree. Speaking of which… I find it telling that Krauss–who runs rough-shod over that line every chance he gets–is openly contemptuous of philosophy, including the philosophy of science, and has been called out for it by many commentators including Carroll himself.
Peter,
Dawid’s usage of the term “confirmation” was a matter of vigorous analysis and debate during the conference.
As Dawid explained during the Q&A session, he uses the term confirmation as synonymous to “increasing the Bayesian probability that for the viability of the given theory”, or something to that effect (there was also a big discussion what “viability” means in this context, but that’s another story). Here the word “confirmation” is being used as a highly specialized technical term, defined in the context of Bayesian analysis. Dawid’s point was that, according to latest understanding in philosophy of epistemology, the term “confirmation” can actually have no other meaning except the above Bayesian one.
Despite the fact that he is probably right about latest results in epistemology, he was still heavily criticized for using the word confirmation, from two sides. First, if you ask a random person on the street what does it mean that a theory is confirmed, they are not going to start talking about Bayesian epistemology etc. Common meaning of the word “confirmation” in everyday usage is waaay different than the specialized technical meaning that Dawid uses. Of course, the common meaning is ill-defined, does not stand up to epistemological scrutiny, and eventually boils down to the technical Bayesian meaning when one thinks about it really hard. But despite this, ordinary people still understand the word in that common everyday sense (“being confirmed means that it’s true”), just like a whole bunch of other ill-defined terms (like truth, proof, reality, etc…). So this is bound to introduce confusion.
Second, George Ellis mounted a very strong criticism of Bayesian analysis itself, distinguishing between adjusting the prior in light of new data, versus adjusting the prior in light of new theories. The former is useful and good, the latter does not really make sense. In short, Bayesian analysis is legitimately applied to information. However, theory is not information, and Bayesian analysis cannot be applied to theories.
My impression is that Dawid acknowledged these criticisms. The biggest part of the problem (although not the only one) turned out to be just the poor choice of terminology.
Also, another thing that became overwhelmingly obvious was the misinterpretation due to ignoring the proper context. When you hear the main single-sentence claim by a philosopher, it can sound scandalous or overstated. But when you listen to their lecture which explains the claim in more detail, you become aware of all the footnotes, side-remarks and technical definitions that reinterpret the scandalous main claim into something much more reasonable, well-measured and uncontroversial. It’s like in journalism — the title of the newspaper article produces a “Wow!!! Seriously?!” reaction, but when you read the whole article, your reaction reduces to “Oh, of course, it’s obvious and trivial, nothing to see here really (yawn)…”.
So Dawid’s statement is much less controversial when one understands the appropriate context and technical definitions of the terminology being used. The only criticism beyond that is basically the one given by George Ellis.
Best, 🙂
Marko
Brian Conrad’s notes on the IUT workshop are absolutely outstanding for both lucidity and informativeness and I can’t recommend them strongly enough.
Peter, You might have seen it already, but just in case, I have a summary here
http://www.forbes.com/sites/startswithabang/2015/12/10/why-trust-a-theory-physicists-and-philosophers-debate-the-scientific-method/
that also has a few words on the assessment vs confirmation issue.
If the new scalar at 750 GeV really exists and if it will turn out to be there for no natural reason, it would make a direct contradiction with anthropic selection arguments
As the pragmatist (and radical empiricist) C.S. Peirce already argued convincingly over a century ago, Bayesian inference has no place in valid scientific inference (or in inference of any sort, for that matter, e.g. in business decision making). That Bayesian inference now plays an increasingly prominent role in particle physics, astrophysics/cosmology, and now biology is unfortunate and personally very depressing (I have watched this develop over my time working in these three different communities). This is what happens when scientists hand over the keys to statisticians and philosophers of science (don’t expect the philosophers to come to the rescue either…I strongly dispute Dawid’s claim that Bayesian inference is the only solution to the problem of confirmation, as reported above by Marko). It would probably help if there were more dialogue among these communities. What would be useful for many would be to simply revisit Peirce’s writings on scientific inference, which were way ahead of his time (and still ahead of our own time apparently…he would have been sorely disappointed with the Munich conference). The enormous amount of nonsense written in the intervening century (in both statistics and in the philosophy of science, and in their intersection, e.g. Popper) may still prevent most from appreciating how fundamental and relevant his writings remain.
**************************************************************************
“The relative probability of this or that arrangement of Nature is something which we should have a right to talk about if universes were as plenty as blackberries, if we could put a quantity of them in a bag, shake them well up, draw out a sample, and examine them to see what proportion of them had one arrangement and what proportion another. But, even in that case, a higher universe would contain us, in regard to whose arrangements the conception of probability could have no applicability.” (Peirce 2.684, i.e., volume 2, section 684 of his collected papers)
**************************************************************************
“Every argument or inference professes to conform to a general method or type of reasoning, which method, it is held, has one kind of virtue or another in producing truth. In order to be valid the argument or inference must really pursue the method it professes to pursue, and furthermore, that method must have the kind of truth-producing virtue which it is supposed to have. For example, an induction may conform to the formula of induction; but it may be conceived, and often is conceived, that induction lends a probability to its conclusion. Now that is not the way in which induction leads to the truth. It lends no definite probability to its conclusion. It is nonsense to talk of the probability of a law, as if we could pick universes out of a grab-bag and find in what proportion of them the law held good. Therefore, such an induction is not valid; for it does not do what it professes to do, namely, to make its conclusion probable. But yet if it had only professed to do what induction does (namely, to commence a proceeding which must in the long run approximate to the truth), which is infinitely more to the purpose than what it professes, it would have been valid.” (Peirce 2.780)
**************************************************************************
For those curious about the further details of this history, you can refer to my book draft. In this draft, I also present a novel and optimal approach to general statistical inference, one that is strictly frequentist in nature and in complete accord with Peirce’s guidelines, providing a concrete example of just how relevant his ideas continue to be:
http://arxiv.org/abs/1301.5186
(Apologies to Peter for the self-promotion, but this history and perspective is unfortunately not presented anywhere else in the literature.)
Pingback: Gathering of philosophers and physicists unaware of modern reconciliation of Bayes and Popper - Statistical Modeling, Causal Inference, and Social Science
Ali,
In the excerpt from your comment below, are you indicating that Popper was the source of some of that nonsense? (I should note that he was a great admirer of Peirce’s contributions.)
Andrew Gelman’s post (the pingback above) is highly relevant.
Bee,
Thanks, I did see that, strongly recommend that those interested in this read your article, so added another link to it (and to the Gelman posting).
Chris,
Peirce of course considered falsification an important aspect of inference, but he also recognized the narrowness of this purely “negative” view of inference (which Popper never properly conceded). What is missing in Popper is the “positive” basis of the process of scientific inference on Occam’s razor. Peirce recognized the importance of Occam’s razor, but even he didn’t go as far as I would in stressing its essential nature. There are really just two aspects to scientific inference (and epistemology, period), the statistical frequentist notion of goodness-of-fit or what I more generally refer to as “concordance” (and which Peirce alludes to in the above quotes) and Occam’s simplicity. Without Occam, we are lost in a sea of possible theories; we need simple theories to make the fastest growth in our knowledge of the universe (see Peirce’s discussion of the game of twenty questions, which I also write about in my book draft). Without goodness-of-fit applied to real-world observations (e.g., most famously through Pearson’s chi-squared test, but even more fundamentally through the statistical approach that I develop in my book draft), we are slaves to arbitrary conventions (Bayesian “flat priors”). But the latter is a deeper argument, the details of which you will find in my book draft.
I have long thought that Peter’s view of string theory as “not simple” is indeed the most important criticism of the entire endeavor. That it is “not even wrong”, i.e. not testable (easily), is also of course important, but its lack of simplicity is more critical. These two critiques embody well the dual competing aspects of scientific inference, simplicity and concordance.
For more details on Popper’s views, I recommend Deborah Mayo’s book “Error and the Growth of Experimental Knowledge”. While she also admires Peirce, she is an avowed Neyman-Pearsonian (“confidence intervals” are unfortunately merely a “confidence trick” as Arthur L. Bowley noted long ago and Edwin B. Wilson, Gibbs’s sole student, did his best to inform us of). While I disagree strongly with her on her defense of Neyman-Pearson (see my book draft for further details), her presentation of Popper’s views is very interesting.
I need not point out that it’s of course better to be a Karl-Pearsonian or Fisherian or Neyman-Pearsonian or Popperian frequentist than to be a Bayesian.
But even better is to be a Peircean frequentist.
Pingback: Gelman on ‘Gathering of philosophers and physicists unaware of modern reconciliation of Bayes and Popper’ | Error Statistics Philosophy
Dogmatic frequentists and Bayesian comments read to the outsider much more like religious than scientific claims. Both are inveterate proselytizers of the unchurched.
That’s a very nice summary of the Munich meeting from Bee in Forbes magazine, Bee.
However, there is one aspect of the debate that still puzzles me: one lesson from the history of theoretical physics is that many ‘sleeping beauty’ theories showed little sign of connecting with experiment at first, but did eventually throw up predictions that could be tested. It seems to me that we should not assume that the difficulties in connecting string theory to the real world may one day be overcome…the real question is, how long along the road does one travel before losing heart?
P.S. Indeed, one can draw a useful link between the Munich conference and the recent conference on the history of GR in Berlin as follows:
After the initial excitement following Eddington’s observations of the bending of light in 1919, GR went through something of a ‘low watermark’ period. It was not until the observation of evidence for exotic phenomena such as quasars, pulsars and black holes in the 1960s that GR became a topic of major interest to the community. So it took 50 years for a theory of gravity to connect with the macroscopic world – how long should we allow for a theory concerning the sub-atomic world to mature such that it can make predictions about that world? Who can know that such predictions will not be rendered possible through some unimaginable mathematical breakthroughs down the road?
Cormac
I think trying to compare string theory and GR is pushing it. After Eddington GR was essentially universally acknowledged to be the theory of gravity. It wasn’t necessarily such a big deal for a long time because no one thought there was much to do with it. I mean Newton works just fine for almost anything anyway, and there weren’t any big problems out there that anyone thought GR would apply to. And there were very good people working on it, coming up with all sorts of solutions of the field equations. And it was THE theory. String theory is almost the opposite, there are plenty of big question out there people hoped it would apply to, but of course there has not ever been any way to test any of it. Beyond what is going on at the LHC now, where SUSY is NOT showing up, which as best I understand it would be a real problem for ST.
Cormac,
It is true that “sleeping beauty” theories sometimes surprise us by turning out to be more testable than we thought. There certainly are times when deciding how far to travel a road before giving up isn’t easy. But that said, there are at least two differences between string theory and your GR analogy.
First, Eddington’s observation of gravitational light bending was in fact an unambiguous test of GR. For that matter, so was the perihelion precession of Mercury. In both cases GR made specific predictions that could be accurately measured, and they were verified. These two observations alone were reason enough to embrace GR no matter how long a low-watermark period ensued. Despite 40+ years of effort no such thing has occurred with string theory.
Second, as already noted, GR makes clear predictions–the kind that can be tested and falsified if they’re wrong. String theory doesn’t. The problem isn’t just that testing it is likely to be out of reach for the foreseeable future–the larger problem is that it’s fluid enough to predict literally anything we might observe. Hence, the title of Peter’s book and this blog: Not Even Wrong. GR could have been wrong… and it wasn’t. As it stands today string theory can’t be, and that situation isn’t likely to change anytime soon.
IMHO, these problems are more than enough reason to justify turning down a different road.
I’d rather if Cormac’s comment not be the excuse for rehashing rather old arguments about string theory, and GR isn’t a great analogy, for many reasons.
I don’t know how much it got discussed at the Munich workshop, but the “when do you give up and try something else” question is obviously a central one for string theory, as well as for the philosophy of science in general. I’ve noticed in recent years a change in how string theorists respond to the “when can it be tested” question, from “by finding SUSY at the LHC, by observing cosmic strings, by Planck CMB results, etc.” to “maybe something unexpected will appear”. While unexpected progress can’t be ruled out, and is a good argument that some people who think they see a way forward should keep trying, the shift to this argument is an indication of a problem. How do you keep this argument from being used to justify sticking with a failed research program, since, no matter how bad things look, it’s always the case that maybe a miracle will happen?
Scott Church’s reply to Cormac is exactly what I was thinking of saying. The “something unexpected” that is needed is a large reduction in the flexibility of string theory to which he refers. That does not refer to relevant observations or the lack thereof. It refers to how the theory responds to observations.
Ali,
In my mind this flexibility is what the lack of simplicity of string theory is really about. Simplicity is essential to making a theory meaningfully testable, and not merely capable of cranking out quantitative assertions that one might try to compare with observation. If GR’s predictions of gravitational light bending and the perihelion precession of Mercury had been conditioned on hidden structure that could be freely adjusted to achieve agreement with observation, then GR would have been viewed in a different light—maybe a signpost on the road to a good theory, but not satisfactory as it stood. And that inadequacy would have nothing to do with an inability to produce numbers that could be compared with experiment, either in theory or in practice.
PS: Another way to put this is to say that ability to generate assertions about observable objects or events is a prerequisite for testability of a theory, but by itself falls well short of testability in the usual sense.
This is probably just semantics, but is the perihelion precession of Mercury considered a “prediction” of GR? It adequately explains the deviation from Newtonian gravitation but the effect had been known for quite a while. I would think that any new theory of gravitation which failed to explain the precession would have failed to gain attention. In that sense, it’s more of a “post-diction”. The deflection of light and gravitational redshifting of light strike me as the true “predictions”.
Peter,
“I don’t know how much it got discussed at the Munich workshop, but the “when do you give up and try something else” question is obviously a central one for string theory, as well as for the philosophy of science in general.”
It did get discussed, that was one of the main topics actually. David Gross kept repeating the word “strategy” all too often, there was a big debate what does it mean to say that a theory is “viable”, etc. If Gross and Rovelli agreed on anything at all, it was the operational definition of “viability”. A theory is considered viable “if I want to keep working on it”, Gross said, and Carlo shouted “Yes! Yes! That’s it!”. 🙂
And in the absence of experiments, what criteria do we have to estimate the viability of the theory? The whole motivation for the conference was Dawid’s proposed answer to that question, condensed in his three criteria: No Alternatives Argument (NAA), Meta-Inductive Hypothesis (MEH) and Unexpected Explanations Argument (UEA). The big debate was whether NAA, MEH and UEA taken together can be considered a good or useful criterion for viability of a given scientific approach (they obviously don’t work when taken separately one by one, but together they are more robust).
I did not see any sort of consensus being reached on the issue during the conference. And the issue is certainly important, not just for string theory, but for all other approaches to quantum gravity as well — some due to practical problems, some due to conceptual problems, but none of the QG theories can actually be tested. So the issue is wider than just ST.
If anything, the Munich conference raised awareness among people (philosophers and physicists alike) that this is an important question to answer.
Best, 🙂
Marko
srp,
There are three modern (and major) viewpoints in statistics as far as I can tell: (1) Frequentism is the “only game in town” (e.g. Deborah Mayo), (2) Bayesian approaches are the “only game in town” (there are many that claim this), (3) Frequentism and Bayesianism have been reconciled. You will find that all three views have their dogmatic defenders. Which camp do you fall into? I suppose category #3, in order to acknowledge the “worth” of camps #1 and #2 (they did work on their ideas for such a long time…it would be a shame if this were all for nothing)? But this is an equally untenable dogmatism. As statistics now increasingly impacts how science is done (how it is reported and even planned…see biology), this is an important fight. You can either participate in it or you can watch from the sidelines. I actually fall into a subcategory of camp #1, as I dispute the worth of Neyman-Pearson confidence intervals (as did many frequentists before me before, until dogmatism and simple inertia allowed confidence intervals to supplant other approaches and the search for better ideas…now Bayesianism is doing the same…sound familiar?).
The main reason I don’t acknowledge Bayesian approaches is that they are based on an obvious logical fallacy, that we can construct a “flat” prior (in one coordinate system) and use it to construct a meaningful posterior probability distribution on which we can be confident actually applies to our universe. But this “flat” prior will be “flat” in only one coordinate system. As physicists, it is obvious that a simple coordinate change will completely alter what is meant by “flat” and therefore what one then gets as a “posterior” distribution. The whole deep desire to construct probability distributions on data is the really the problem (equally a problem with frequentist “confidence intervals”). As Edwin B. Wilson tried to tell us, all we really have is the p-value for certain statistics (e.g. Pearson’s chi-squared/p-test, which is the physicists’ favorite tool, for good reason). This is the only thing we can all agree on. It should be noted that Wilson is often reported as the discoverer of “confidence intervals” (by Neyman himself), a false characterization of his work that he disputed his entire life. So essentially, Neyman was lumping Wilson in with Neyman-Pearson. I guess you could say Wilson was being “dogmatic” about this. I think it’s the lack of people willing to be “dogmatic” that has led to the current complete disarray and confusion within the field of statistics and certainly to outsiders (no one criticizes any more, at least not like they did in the past). So here’s to dogmatism (especially when it’s in the defense of science)!
Chris,
I was a bit glib in the above discussion of “not even wrong” and “not simple” criticisms of string theory. These claims cannot be so easily separated, as simplicity is what ultimately allows for testability.
Chris and G.S.,
The prediction of light bending vs. the postdiction of Mercury’s perihelion represent an important focal point in the field of the philosophy of science (it comes up everywhere). Basically, the prediction of light bending was not really seen as convincing as the postdiction of the perihelion. An interesting mental exercise: Imagine if there were no historically/currently testable predictions from G.R., only the postdiction of the perihelion of Mercury. G.R. would still be heralded as a fantastic triumph, as it is simple and gives exactly the right postdicted result for the perihelion. If all string theory did was postdict the mass spectrum with either zero or even only a handful of free parameters (less than the number of masses, but not 10^100), then it would also be celebrated (and not just by those working on it!). It would need to make no new predictions. That’s also, by the way, why Popper’s views are restricted. They don’t acknowledge the importance of the simplicity of certain theories that postdict successfully.
The ongoing discussion about the theory of confirmation of scientific theories is certainly very important. However, concerning the work of Don Page I would like to remark, that he wrote quite a few fundamental papers which I carefully studied and I never had the slightest impression that he was mixing up theological with scientific reeasoning whatever his own beliefs are.
Peter, many thanks for linking to my article and facilitating a great discussion.
Following up on vmarko’s last comment, I wanted to clarify that “viability” as “something worth working on” was not the definition used in Munich, at least not by the philosophers in the room. Dawid said repeatedly that what he means by the “viability” of a theory is its ability to account for observations at the scales it intends to describe. Newton’s theory is “viable” as a theory of gravity at low energies and speeds, for example. “Viable” is basically a substitute for “true,” since a viable theory will ultimately turn out to be encompassed by a larger theory and is not therefore true in a universal sense.
So, when Dawid argues that the NAA, MEA, and UEA arguments increase the Bayesian confirmation that string theory is viable, he’s not merely saying that they increase the probability that you should work on string theory, he’s saying that these arguments increase the Bayesian confirmation that string theory correctly describes nature at the Planck scale. It’s a stronger claim.
The reasoning is that these three non-empirical arguments impose limits on underdetermination; they decrease the likelihood that there are other theories out there that are also able to describe nature at the Planck scale, which in turn increases the likelihood that string theory is viable. Dawid doesn’t claim that these arguments can ever completely kill underdetermination or (in other words) drive the confirmation of string theory’s viability up to 100 percent, but he argues that they can get us pretty far — far enough to account for the trust/confidence in string theory exhibited by many of its proponents.
Thanks again!
-Natalie
In my field, we have to contend with greedy sociopaths trying to circumvent scientific scrutiny by suppressing the evidence. From a philosophical point of view, that threat pales in comparison to an open-ended proposition to eschew testable hypotheses so long as something strikes enough people as beautiful and/or unique. Liars get caught, eventually, if the evidence exists. In science, it always should, from the greatest physicist to the lowliest stamp collector. How can it possibly be otherwise? I’ve been reading about this conference for days, and it just seems like…madness! Bayesian epistemology in place of evidence, built around a “framework” of theory generation of nearly infinite malleability? For decades? With no end in sight? How is that not unthinkable? A days-long dialogue between some of the brightest minds in the world devoted to…this! I’m grateful for all the wonderful coverage, but I still can’t quite believe it.
LMMI,
A lot of this doesn’t make much sense until you realize that Dawid, Gross and others are taking as starting proposition a well-entrenched ideology about string theory, then working backwards to find a philosophical framework to justify it as science. I believe Dawid is sometimes even quite explicit about this: he’s just writing down the arguments string theorists use to justify what they do in the face of conventional failure as science.
There are `sleeping experiments’ too… superconductivity was discovered in 1911 but wasn’t well described by theory until 1957. CP violation was discovered in 1964 but didn’t have a good theory until 1973.
You don’t always need a theory to make experimental progress. Maybe, if we finally get the mass scale of the neutrinos established, we’ll see a pattern that still isn’t evident. Ditto with the CP violating phase in neutrinos.
Dark Matter is a long, long sleeping experimental discovery, one that has had lots of phenomenological impact but still doesn’t have a good theory.
Maybe science is a process, and maybe theories aren’t things defined by falsifiability or degree of Bayesian probability. Maybe theory is just a repository for the discussion in a sort of standardized format. And maybe the highest calling is to evade the problem of the most charismatic or insistent human defining right from wrong, but be guided by something bigger than a human (at least ideally) like experiments, observations, and some sort of durable beauty in the theory.
I’m a little surprised that in this discussion no one has taken issue with Wolchover’s analogy that no one has observed an atom e.g. ” modern scientific theories typically make claims far beyond what can be directly observed — no one has ever seen an atom — and so today’s theories often resist a falsified-unfalsified dichotomy”.. Individual atoms have been pretty clearly observed. The most dramatic example being STM images of Silicon, but x-ray diffraction, observation of fluorescence from a single atom in an atomic trap, even radioactive decay of a single nucleus in a Geiger counter, all observe single atoms pretty clearly. The observability of atoms seems a remarkably poor example of how theories can’t be falsified.
Laurence,
“I’m a little surprised that in this discussion no one has taken issue with Wolchover’s analogy that no one has observed an atom […] Individual atoms have been pretty clearly observed.”
I guess what Natalie meant was that atoms have not been *directly* observed back at the beginning of 20th century, when the modern atomic theory was first introduced. There were things like Brownian motion and such, but these were all *indirect* observations of (various effects of) atoms, and I guess back then nobody was even contemplating any possibility of direct observation of individual atoms. Direct observations (via X-diffraction etc.) came quite some time later.
Best, 🙂
Marko
@LMMI: I think “Bayesian epistemology in place of evidence” is misphrasing the point. Rather, it would be “Bayesian epistemology as a generalization of hypothetico-deductive evidence.” After all, evidence is taken into account in the updating rule… the question is what to do with evidence which is not set up from an experiment.
I’d like to note that “flat priors” – while usually used (as the principle of indifference) for naive guesses – are generally not taken as acceptable, for the reasons Ali mentions (flat priors are no less special than any other set for a different problem). I think a lot of the issues that physicists run into occur when they try and apply really naive methods (such as the principle of indifference) and try and get results… it would be akin to saying “well, we don’t know what our error is, so we’ll say 5%… and then look, this result is statistically significant!” You make one statistically irresponsible decision and get nonsense results. This isn’t necessarily a problem with a Bayesian interpretation itself, but rather with trying to apply Bayesian methods with a fast-and-loose approach.
Accordingly, the retrospective alterations that Peter and LMMI refer to, I would guess (although I don’t know the community well enough to say for certain anything about the particular of the individuals), come from the way they choose their priors and their probabilities to fit their conclusion rather than their choice of BCT as a theory of philosophy of science in its entirely. I.e., classical overconfidence in their probability determinations rather than reconstructing their entire philosophy of science.
As another error physicists make: as noted in the “Gathering of philosophers and physicists…” article above: Bayesian methods are designed for when one can explore the complete parameter space. When one only has access to a selection of parameters in consideration, there’s no guarantee you’ll get accurate results, and physicists are (generally) trying to apply those fast-and-loose results to what is already a limited parameter space.
Magnema,
To construct the posterior probability distribution, you need to assume a prior probability distribution (which can be updated with whatever data are at hand). The posterior is however completely set by the prior (which is not unique). There is no way around this in Bayesian statistics. This is the definition. The only argument from Bayesians that I have ever seen against this is that we should not take the posterior distribution so seriously, but then what’s the point?
The desire to construct the posterior probability distribution is of course strong (not only in Bayesian statistics, but also in traditional frequentist approaches), but it’s not logically sound, as Peirce points out above. Bayesians have largely won over a sizable chunk of the scientific community because they give scientists what scientists they think they want (you can even find many Bayesians who admit this). But scientists are also no experts at understanding the subtleties of the probabilistic assessment of data. Again, as both Peirce and Wilson pointed out, the only thing we really have is the p-value for particular statistics based on the cumulative distribution (which is invariant to changes in the underlying coordinate system). In my manuscript I develop an entire framework for optimal statistical inference based on this idea and using a particular and remarkable statistic that I call the “fidelity” (a specific interpretation of the Gibbs entropy). I show that it is superior to maximum likelihood (the previous “gold standard”) for parameter estimation and superior to the chi-squared/p-test (the previous “gold standard”) for goodness-of-fit assessment. I also generalize Student’s t-test to completely arbitrary distributions, with now an automatic means for goodness-of-fit assessment. Additionally, I straightforwardly extend the notion of the “fidelity” to non-parameteric testing, developing a test that is superior to the KS test (also considered the “gold standard” for such testing). That all of this is possible in a way that is coordinate-system invariant is remarkable. Again, the entire system is based on the p-value of the fidelity, a statistic constructed from the coordinate-independent cumulative distribution. My approach avoids the thorny problems that characterize both Bayesian and traditional frequentist approaches. For example, my approach is the only solution that I’m aware of to Neyman’s paradox, which led Neyman to believe that there is no fundamental approach to statistical inference and that we should only use good judgement or “good behavior” in assessing data in order to not be misled. But Neyman’s paradox (a bit of mathematical sleight-of-hand by Neyman) is simply resolved by sticking to the cumulative distribution and respecting the topological ordering of the coordinate system.
I only give the above details to show that there are other possible ways of performing statistical inference. But, if you are a Bayesian, you are done. You already have a complete system for doing any kind of inference (one that is of course fatally flawed). The only work to be done is to find better applied math techniques and algorithms to more efficiently perform the necessary computations (which was the real barrier to applying Bayesian approaches in the past…now we have cheap and much more powerful computers).
Please, stop the repetitive rants about Bayesian inference. I’m deleting them and any more on the topic. This has little to do with the topic of the posting and drives away people with relevant comments.
” I believe Dawid is sometimes even quite explicit about this: he’s just writing down the arguments string theorists use to justify what they do in the face of conventional failure as science.”
Does Dawid anywhere explain the reasons underlying his apparently naive and uncritical acceptance of such arguments? That’s a question potentially more interesting than the theoretical apparatus he’s constructed. I would hope that his reasons amount to more than a simple appeal to authority.
Visitor,
“Does Dawid anywhere explain the reasons underlying his apparently naive and uncritical acceptance of such arguments?”
I think it’s naive to call his arguments naive and uncritical. Despite the fact that I disagree with everything he said in his lecture, I have to admit that the arguments and reasoning presented in his lecture (and I guess his book as well) is very thought-through, and anything but naive and uncritical.
What he did was to look at what string theorists were actually doing, provide an abstract description of it in terms of the NAA, MIH and UEA, and then promote this description into a normative statement (“this” is what they are actually doing, and it is also what they should be doing). And he was very up-front with all that, while including a great deal of analysis of the interactions between NAA, MIH and UEA, assumptions, pitfalls, caveats, characterizations, etc…
You are welcome to disagree with his proposals (as many people do, myself included), but it is just plain wrong to call them naive, uncritical or appealing to authority.
Best, 🙂
Marko
Marko,
I think what “Visitor” means by “simple appeal to authority” is much the same as what you describe as “promote this description into a normative statement”. As far as I can tell, what Dawid is doing simply ignores the main question: isn’t all this just an attempt to prop up a degenerating research program? The tactic seems to be to forgo that obvious interpretation of the situation in conventional terms, in favor of a point of view taking string theorists as authorities, with the job just to justify their arguments.
Visitor, vmarko, Peter,
Dawid says there is one way in which his arguments are normative, or at least, following conventional policies of how science should work. Any of his three criteria, he argues, could disconfirm a hypothesis, just as easily as it might confirm it. This makes each one a sort of test.
Also, to the question of what I meant by “no one has ever seen an atom,” the STM images are extremely impressive, but aren’t we still interpreting data? Fantastic interpretation, I agree – hence the near-100 percent confirmation of atomic theory. In the late 20th century, physicists worried they would never manage to rule out all possible alternative interpretations of the indirect evidence they possessed in favor of the existence of atoms. Atomic theory was “undetermined.” But eventually, everyone decided that the indirect effects of atoms (like the way dust flits through the air) should count as empirical data. Almost everyone, that is — Ernst Mach refused to believe in “the reality of atoms” to his dying day, in 1916.
-Natalie
Natalie and Marko,
Looking again at Dawid’s three criteria, I think that they might more accurately be characterized as a typology of arguments for a degenerating research program:
1. No Alternatives: no matter how bad things look for our ideas, they’re better than anyone else’s.
2. Meta-inductive: we had a good reason to start looking at these ideas, since this kind of thing had worked before. That reason is still good, no matter had bad things turned out and look now.
3. Unexpected explanatory coherence: the more we study our ideas, the more devoted to them we get, so there must be something to them.
I’m not an active physicist anymore, but these three criteria seem to be unconvincing.
“No alternatives” might have a sociological explanation. “Unexpected explanatory coherence” depends on what it is that you explain – black hole entropy etc. seems, to me at least, fairly unconvincing. But the weakest criterium is “meta-inductive”. What do people mean when they say this kind of thing worked before? What is “this kind of thing”? The graveyard of physics is littered with things that didn’t work. I bet most physicists have had more ideas that don’t work than ideas that do work. If you only look at the published ideas that did work, you’re suffering from a serious case of confirmation bias.
Peter,
“I think what “Visitor” means by “simple appeal to authority” is much the same as what you describe as “promote this description into a normative statement”.”
When listening to Dawid’s lecture, I didn’t get that kind of impression (despite the fact that I was expecting to hear precisely something like that). Dawid doesn’t promote descriptive to normative because string theory physicists are very smart, but because he believes that his three arguments are the only sensible way one can assess a theory absent any observational evidence. So he’s not appealing to authority, but rather saying there’s nothing else we can do.
Of course, I disagree with him on that, and yes his arguments do indeed support a degenerating research program. But his arguments were not designed so that they would support string theory, that is somehow a side-effect. The arguments were designed to capture what we can say about the quality of a theory when we have no access to experiment. And of course, I believe they are very very wrong, but I failed to see any appeal to authority there.
This is funny, btw. I heavily disagree with Dawid’s points, but somehow I keep sounding like I’m defending his ideas here. That’s certainly not my intention.
Natalie,
“Also, to the question of what I meant by “no one has ever seen an atom,” the STM images are extremely impressive, but aren’t we still interpreting data?”
If you put it that way, then no one has ever seen a chair either — all we see is light coming to our eyes, and then we’re interpreting that data. Interpreting the data from X-ray experiments is really no worse in any way (neither qualitatively nor quantitatively) than interpreting the data when we look at bacteria, red blood cells, chairs and tables, or other galaxies.
Best, 🙂
Marko
You can see an individual trapped barium ion glowing with the naked eye.
They should do more of these conferences for unresolved problems/speculative theories in physics. Everything from Multiverse to dark matter all the way down to quantum foundations. I suspect if you allowed physicists to talk about things in such a setting you’d get to hear more dissatisfaction that you would suspect.
That said, from the coverage of this conference things might just boil over altogether into a furious series of shouting matches. Shamefully, I must confess this prospect would make me even more likely to buy a ticket.
When you win a Nobel prize you earn the right to fly to nice cities and have conferences about the meaning of simple English words like “test”, “verify” and “true”. I am sure the coffee break pastries were fantastic, but I really don’t think there was anything there to inspire so much hand-wringing debate. The outcomes were completely banal.
In stark contrast to the abc conference which is scientifically and sociologically fascinating on many levels.
Today I read the Consilience section in the recent Sci.Am. (Dec. 2015) and I think there are some useful remarks in it concerning the ongoing debate about scientific verification. Shermer mentions the 19-th century philosopher William Whewell and his notion of “consilience by induction´´. He argues that for a theory to be accepted it must be based on more than one induction or a single generalization drawn from a specific fact. It must have multiple induction that converge on one another , independently but in conjunction (Book: The Philosophy of inductive Science).
Magnema: I tend to think of “evidence” used to update priors in science as something other than a human judgment about, say, the beautiful unifying properties of a theoretical framework. Rather “evidence” is the result of an objective measurement of a natural phenomenon. Of course, when one has a functionally infinite landscape of possibilities to choose from, I suppose one can exclude some of those from the newly-inferred probability distribution (say, when a particular model predicts sparticles of a certain mass that are not observed). But that process seems to have made little dent in the posterior assessment. “Evidence” in this new paradigm seems to mean something quite different than what is healthy. The ability to calculate probabilities using Bayes’ shouldn’t obscure this. If, functionally, this epistemic paradigm allows so many lines of “evidence” but cannot assign to them a proper weight, perhaps it is suspect.
You comment “…Andrew Gelman (who I think, unlike anyone in the theoretical physics community, actually knows something about Bayesian methods).” If you and Golman bothered to read the article, you’d see this was indeed debated at the meeting by both physicists and philosophers. This kind of snide remark undermines your attempt to provide useful comments on what is going on.