Killer Asteroids

During my recent vacation I visited my old friend Nathan Myhrvold, and got a tour of his company’s lab near Bellevue. At that time he told me about what he had been working on recently, which has now appeared on the arXiv here, and is the subject of news stories today at the New York Times and Science magazine.

I confess I’ve never worried much about killer asteroids, but am glad that someone is doing this. Nathan has always pursued a wide range of different interests, and killer asteroids has evidently been one of them. I first heard from him a year or two ago about how he had gotten interested in the question of how to model the observability of such objects. Such modeling affects choices to be made about how to optimally search for these things (space-based or earth-based telescopes? what kind?). He wrote a paper last year about this, which was published in March.

What Nathan told me when I saw him was that he had found significant problems with the modeling done by the NEOWISE/WISE group at NASA, and you can now judge for yourself by reading his paper. I’m very far from being able to understand the details of this story well enough to judge who’s right here. I do know Nathan well enough to know that his work on this deserves to be taken very seriously, and would bet that he has identified real problems. As noted in the comments there, the reaction from one of the NASA WISE people quoted at the end of the Science article wasn’t exactly confidence inspiring.

Update: There’s a press release about this out from NASA today, pretty much devoted to attacking Nathan’s work.

Update: For some specific criticisms of Nathan’s work, see the comment thread here. For a response to some of this from Nathan, see here.

Update: Scientific American has an article about this here.

Update
: As pointed out by Wayt Gibbs in a comment, those interested in some discussion of the main point at issue might want to read the exchange here.

This entry was posted in Uncategorized. Bookmark the permalink.

74 Responses to Killer Asteroids

  1. Ethan Vishniac says:

    Ned Wright is a professor at UCLA, not a NASA person. I urge anyone interested to read the papers rather than judge by your opinion of Ned Wright or Nathan Myhrvold. For those without the patience to do so, I note that Ned has been usually acerbic, and usually correct, throughout a long career in science.

  2. Peter Woit says:

    Ethan Vishniac,
    Thanks, fixed the affiliation, I think.

    I know nothing about Wright, but I do have a lot of experience with what happens when you tell smart scientists they’re wrong about something. If you’re the one who is wrong, you quickly learn immediately why. If it’s them, you get this kind of thing (for which “juvenile” is more accurate than “acerbic”).

  3. Henry Lichtenstein says:

    If triple “l” “Killler” is not merely a typo, please explain what you mean by it. Thanx.

  4. edmeasure says:

    It’s entirely possible that some significant threats among the smaller asteroids could be missed, whether for these reasons or others, including ones that could flatten a city or a small country. The big ones, extinction level event size, not likely. That’s not the case for comets. Giants could be out there coming and we wouldn’t see them until it was entirely too late.

  5. Peter Woit says:

    Henry Lichtenstein,
    I do think the triple “l” looks quite good there, but I fear it is a typo that I’ll fix.

  6. Henry Lichtenstein says:

    Whew! Glad it wasn’t meant to imply another Chicxulub impactor was heading our way.

    🙂

  7. Dwayne Day says:

    “I do have a lot of experience with what happens when you tell smart scientists they’re wrong about something.”
    .
    I note that while you complain about the reaction of one of the WISE scientists, you neglect to address a very serious issue here: Myhrvold did NOT bring up these issues in a peer-reviewed paper, but by posting it to a blog and then having his public relations people notify various science journalists and bloggers about it. That’s bypassing normal (and proper) scientific procedures. It’s not really telling “smart scientists they’re wrong about something,”–the way you do that is by publishing with peer review. This is unprofessional behavior.
    .
    In addition, Myhrvold also interjected himself into an ongoing NASA mission selection competition in an obvious effort to influence that selection. That too is unprofessional. He’s not a knight in shining armor, he’s a spoiler.

  8. Peter Woit says:

    Dwayne Day,
    Nathan did not “post the paper to a blog”. He posted it on the arxiv and submitted it (if you believe the NYT) to the journal Icarus. Among the fields I’m familiar with, posting a preprint to the arXiv at the same time as submitting it to a journal is the standard behavior.

    I have no idea whether he would like to influence the NASA selection, but don’t see why, if he’s convinced there’s a problem with the NEOWISE numbers, it is “unprofessional” for him to raise the issue. Put differently, if you knew there was a problem with the numbers being used in such a selection process, it seems to me that you would have some duty to raise the issue. He’s an unusual case of someone who has deeply immersed himself in the relevant technical issues, while not being affiliated with any of the groups seeking funding (even more unusual in that not only is he not seeking funding, others have been seeking it from him).

  9. Henry Lichtenstein says:

    @ Dwayne Day

    “… –the way you do that is by publishing with peer review.”
    .
    That is certainly one way, especially if one is at leisure to do so. But it is not the only way. I’ve been to a few conferences in my day, and I can vouch for there being many other ways of telling people they are wrong.

  10. Henry Lichtenstein says:

    “… , while not being affiliated with any of the groups seeking funding”
    .
    A laudable way to be, indeed. Beware of geeks seeking funding, like some folks at East Anglia University.

  11. Jeffrey M says:

    Peter,

    While posting to arxiv while you submit is totally standard, in math at least that’s not what you would do if you thought you found an error in someone’s paper. In my experience, the person who thinks they found the error would email the author, and let them know so they can fix it. This literally just happened to me, someone had emailed a while ago about a paper I wrote 15 years ago, and my co-author and I finally had the time to look at what they said. I spent all day emailing back and forth with her today, and we came to the conclusion that we were right in the first place, so we’re emailing the person who thought we had an error to see what they say. If we had decided instead they were right, we would have posted a correction to arxiv, and to the original journal, acknowledging who found the error. Seems like a much nicer way to do things.

  12. Dwayne Day says:

    “Submitted” is not the same as “published.” He’s made his paper available online–prior to any peer review–and then had his flacks publicize it (see the comments over at NASAWatch).

    And he not only raised the issue of NEOWISE data, but injected himself into the Discovery selection process re NEOCam. He’s playing spoiler.

    The scientists quoted in the article have pointed out that he should go through peer review instead of engaging in a public relations campaign bypassing the scientific process and going straight to the press. If his analysis is correct it will survive peer review.

  13. AdamT says:

    Jeffery M,

    What makes you believe that Myrhvold didn’t contact them in private beforehand? My reading of the article indicates they were talking before he published.

  14. Jeffrey M says:

    AdamT

    Where do you see that? They clearly talked, but from what I read it looked like it was after his paper was on ArXiv, when errors in Myrhvold’s paper were pointed out to him. I see nothing to indicate he wanted to give anyone time to fix anything, he pretty clearly wanted to make a splash himself.

  15. Peter Woit says:

    Jeffrey M,
    As far as I know, there was discussion of this between the parties before the preprint appeared. Would of course have been better if that led to agreement on what was right and what was wrong. Resolving such an issue tends to be more straightforward in math than in other fields.

  16. G. S. says:

    There are a few things which bother me about this story:

    – Normally, when a single author writes a 111 page article criticizing a large scientific collaboration’s data analysis, I am a bit suspect. It doesn’t mean that the person is wrong, but I assign a high prior belief to the author being in error. Regardless of the author’s institutional affiliation, I am unlikely to give much credence to the claims until after they have survived peer review.

    – According to the Science article, it sounds like there was quite a bit of back and forth between Myrhvold and NEOWISE. NEOWISE disagreed with his analysis and suggested that he submit the paper to peer review. That seems like the proper course of action to settle a scientific disagreement. I am troubled by the additional media blitz (initiated by Myrhvold?) where the author claims the scientific establishment is staying the course because they are worried about their NEOCam proposal. This is similar to the argument constantly used by climate change deniers for why climate scientists repeatedly publish papers confirming global warming. It’s possible that Myrhvold is correct, but these tactics are moving my belief even further out of his favor.

    – The media seem to love these stories of “lone non-expert proves scientific establishment wrong”. It seems there is a new one that gets promoted every week. Almost all of them eventually get debunked (e.g., the Canadian teenager who found a lost Mayan city … that turned out to be an old cornfield). Unfortunately, the public never reads the debunking article and they are left with the mistaken belief that the scientific community is constantly being proven wrong by non-experts. This makes it extremely difficult for the scientific community to convince the public that its findings are a solid foundation for public policy.

  17. Peter Woit says:

    G.S.,
    I think this is a rather unusual situation and cast of characters, so I don’t think trying to figure out who is right by analogies of the kinds you are making is sensible. In addition, trying to paint Nathan as analogous to a climate change denier is pretty offensive.

    And no, as usual, I’m not going to allow comments here from people who want to argue about climate change.

  18. G. S. says:

    I certainly did not intend to cause offence. I merely intended to point out which aspects of this story bias me a particular way and why. In my experience, my line of reasoning has biased me toward the eventual correct conclusion much more often than not. However I realize that my line of reasoning is not proof, which is why I fully encourage peer review from (hopefully) initially unbiased experts in the field.

  19. Henry L. says:

    @ G. S.
    “…, I am a bit suspect.”

    Did you really mean to say that you are not to be trusted?

  20. G. S. says:

    @Henry L.

    Ha ha, no. I always thought that “suspect” could be used in this way but the dictionary claims otherwise. I learn something every day.

    Interestingly, if I did say that I was not to be trusted, would you believe me?

  21. Henry L. says:

    G. S.,

    Thank you for taking my remark in the spirit that I intended it to be taken.

    I would believe you but I wouldn’t entirely trust you. 🙂

  22. Scott Church says:

    Peter, I’m not familiar with this issue, but it seems to me that your friend is entirely within legitimate bounds. Dwayne Day says,

    “‘Submitted’ is not the same as ‘published.’ He’s made his paper available online–prior to any peer review–and then had his flacks publicize it (see the comments over at NASAWatch)…”

    I’ll grant that submitted doesn’t equate to published, but the Arxiv is well-known as a preprint forum where researchers routinely post work while it’s in review. In addition to the journal/s it’s been submitted to, this gets it in front of a larger audience where it can receive whatever expansion and/or correction it may need that much faster. Everyone does this… we’re all the better off for it, and we don’t accuse anyone of duplicity for doing so.

    The press routinely trolls the Arxiv for anything that might fuel PM-Edition potboiler stories and regardless of what anyone says or publishes, “Killer Asteroids Threaten Humanity!” is bound to be like tossing a 10 lb bloody pot roast into a shark tank. It’s little wonder they were all over it, and whether Myrhvold spoke to them first or not, you can bet they would’ve sought him and others out for comment soon enough anyway. If he, or anyone else chose to respond that hardly makes them “flacks” or “spoilers.” I don’t recall anyone labeling Stephen Hawking as either when he publicly bet Gordon Kane that the LHC wouldn’t find the Higgs, only to turn around and proclaim that it could destroy the universe at any moment when it was.

    As for NASAWatch, I’ve seen countless such *****Watch sites over the years. In my experience all were agenda-driven forums run by “activists” who invariably saw ***** as a cabal of nefarious evil doers against whom they had been anointed to be “whistle blowers” delivering “the truth” to the masses. Not surprisingly, NASAWatch is owned by SpaceRef–a self-described “new media company focused on the space sector” that among other things, has an “Intelligence Unit.” According to the NASAWatch website header, “This is not a NASA Website. You might learn something. It’s YOUR space agency. Get involved. Take it back. Make it work – for YOU.” [sic]. The only references I was able to turn up to their content were at conspiracy websites. The words objective and trustworthy don’t exactly leap to mind here.

  23. Henry L. says:

    @ Scott Church

    I agree that Hawking’s bets and proclamations do not make him a spoiler, but I fail to see how the outcome of his bet with Kane has anything to do with his proclamation about the Higgs possibly destroying the universe. The discovery of the Higgs at the LHC did not create the possibility that it could destroy the universe.

    Perhaps I am misreading your comment?

  24. Scott Church says:

    Henry L.,

    It didn’t… I cited the two stories together only because both involved the Higgs, and both struck me as examples of situations similar to that involving Peter’s friend that didn’t lead to anyone being accused of being a flack or a spoiler. Other than that, you’re correct–there is no similarity between the bet and the doomsday comment. Best.

  25. JeanTate says:

    I’m a bit surprised no one has yet commented on the actual content of the Myhrvold paper (arxiv preprint version anyway). There are sites where this is discussed, and which include posts by actual astronomers, professional and amateur, e.g. Challenge to Asteriod Size Estimates, at Yahoo Groups.

    It seems that, among other things, Myhrvold did not do a good job of checking for consistency, both internal (e.g. radius vs diameter), and external (e.g. independent estimates of the sizes of asteroids, from studies independent of WISE, occultations, etc; there are quite a few such independent datasets, available publicly, for free).”

    As far as I know, these sorts of simple, if sometimes rather tedious – astronomers tend to use rather a lot of different systems, and conventions, not all immediately obvious (surprisingly, radius vs diameter is one!) – consistency checks peer reviewers expect authors to do, before submitting to arXiv. Sure, if you’re an outsider, you’ll likely miss some, but I find it strange that a trained physicist would trip up over radius vs diameter, no matter how deeply you’d have to dig to get complete certainty (“size” clearly doesn’t cut it).

  26. Anonyrat says:

    “Amy Mainzer, principal investigator for NEOWISE, pointed out that many of her projects’ measurements have in fact been reproduced and confirmed. Japan recently operated an infrared telescope called Akari that was similar to WISE, and it measured many of the same asteroids.”

    https://www.washingtonpost.com/news/speaking-of-science/wp/2016/05/25/that-study-critiquing-nasas-bad-science-on-asteroids-is-pretty-bad-science/

  27. Anonyrat says:

    http://arxiv.org/abs/1403.7854
    A Comparative Study of Infrared Asteroid Surveys: IRAS, AKARI, and WISE
    Fumihiko Usui, Sunao Hasegawa, Masateru Ishiguro, Thomas G. Mueller, Takafumi Ootsubo

  28. Anonyrat says:

    From the WaPo link:

    He {Myhrvold} added that he tried to get WISE scientists to comment on his work for months, but that they’d refused to respond until he made his work public. Mainzer disputes this, saying she has offered corrections to the paper that Myhrvold has ignored, including an issue that he sees as a possible sign of fraud.

    “In more than 100 cases, the reported asteroid sizes they listed were an exact match to prior papers, that’s within the meter. That’s just not possible to have happen accidentally,” he said.

    “”We’ve tried to explain this many times,” Mainzer said. Those measurements, she continued, are intentionally pulled from other sources — they’re used to calibrate the measurements made using the spacecraft with previously observed data. It’s not the case of a mistake or fraud, but a standard procedure.”

    “She also takes issue with the underlying thesis of the critique, which is that the NEOWISE calculations ignore Kirchhoff’s law of thermal radiation.”

    “”Of course this model doesn’t perfectly conserve energy, and no one has ever said it does,” Mainzer said. The model, she explains, sweeps away some of the complexities of an asteroid’s heat distribution to make up for the fact that asteroids have incredibly complex surfaces. Scientists using these models don’t have every data point they would need to do a perfect calculation, and that’s the whole point of using the model in the first place.

    “The paper is basically useless in the sense that he’s complaining about a model which it has always been clear does not satisfy physical laws,” Ned Wright told The Post. “So then the question is, does it work, is it a good approximation anyway, and the answer to that seems to be generally yes.””

    https://www.washingtonpost.com/news/speaking-of-science/wp/2016/05/25/that-study-critiquing-nasas-bad-science-on-asteroids-is-pretty-bad-science/

  29. Anonyrat says:

    On a yahoo egroup, Joseph Masiero, Ph.D. Scientist, NASA JPL/Caltech, points out a NASA-JPL response (provides no link to that response)

    https://groups.yahoo.com/neo/groups/mpml/conversations/messages/32026

    Quote:

    A sampling of the errors include:

    — Equation 3 (page 7), equation 26 (page 20) and equation 33 (page 21) are all wrong by a factor of 4. This is because of a fundamental error confusing diameter and radius. It hasn’t been possible to review the actual code used by the author(s) in implementing the thermal model, but if it follows these equations it will consistently get incorrect results for diameters, albedos and predicted fluxes. One example of this can be seen in Figure 21 (page 68), which shows a diameter for asteroid 295 of about 660 kilometers. Such a size would make this object bigger than asteroid Vesta, the second largest object in the asteroid belt. The actual diameter of asteroid 295 is estimated at about 30 kilometers from the Infrared Astronomical Satellite (IRAS), Japan’s AKARI satellite, and WISE data.

    — The paper mischaracterizes the use of the radar/occultation/flyby diameters. In Mainzer et al. 2011 ApJ 736, 100, the NEOWISE team uses these diameters to compute model brightnesses and compare them to the measured brightnesses for the objects. They are in excellent agreement. The radar/occultation/flyby sources are cited in this paper; later papers reference this calibration paper.

    — The paper erroneously states that the NEOWISE data are not reproducible. On the contrary, in Usui et al. 2014 the authors find that IRAS, AKARI, and WISE are all in agreement to within +/-10% 1-sigma for the ~2000 asteroid diameters measured by all three observatories. This point is given in the abstract (See: http://arxiv.org/abs/1403.7854). These measurements were independently taken over a 30-year span beginning with the joint UK-US-Dutch IRAS mission and Japan’s AKARI mission, as has been documented in peer reviewed papers. While Usui et al. 2014 is mentioned in the new report (on page 2), the central point of its finding was omitted.

    –The paper has changed the standard asteroid thermal model (Harris 1998) that is widely used and has been extensively peer reviewed, but the paper doesn’t show that the proposed model works better than the standard model at replicating the radar diameters.

    — The paper mischaracterizes the NEOWISE use of visible brightness H; it is an observational constraint and not a free parameter (page 48.)

    — In equation 32 (page 21), the brightness of the Sun is computed as being of order 10E-11 Jy. This is off by more than 20 orders of magnitude as currently written.

    Data from the mission is publicly available. A quick guide to the NEOWISE data release, data access instructions and supporting documentation is available at http://wise2.ipac.caltech.edu/docs/release/neowise/. Access to the NEOWISE data products is available via the on-line and API services of the NASA/IPAC Infrared Science Archive.”

  30. Anonyrat says:

    The above comments, including page numbers, from Joseph Masiero are in reference to
    http://arxiv.org/abs/1605.06490
    “Asteroid thermal modeling in the presence of reflected sunlight with an application to WISE/NEOWISE observational data”

  31. Anonyrat says:

    Dave Herald of IOTA –
    https://groups.yahoo.com/neo/groups/mpml/conversations/messages/32032
    Quote:

    There is a world-wide group of (mainly) amateurs who regularly observe occultations of stars by asteroids The main groups are located in Nth America, Australia/New Zealand, Europe, & Japan. These groups collectively observe over 200 actual occultations per year (and have high hopes for this number to increase significantly as Gaia data becomes available over the next few years). Their observations and results are annually archived at NASA’s Planetary data System, Small Bodies node, Asteroid/Dust Archive – at http://sbn.psi.edu/pds/resource/occ.html.

    The observed length of a single occultation chord routinely has a precision of about 1 km for a main belt asteroid. That is, we are measuring the outline of an asteroid at a 1km precision, at which level surface irregularities (let alone gross shape) are evident. To obtain a reasonable estimate of the asteroid size, you generally need at least 3 occultation chords spread across the asteroid. On occasions we have many 10’s of chords located across the asteroid – with the shape and irregularities of the asteroid profile being blindingly obvious. We generally fit an ellipse to the observations – as asteroids are rarely spherical. And this raises the issue of the rotational orientation of the asteroid – as the profile is continually changing as the asteroid rotates. A consequence is that we do not get (or expect) a ‘single’ diameter measurement for an asteroid. On the other hand the observations can (and are) linked to light-curve inversion models to provide a scale (as well as validation) for those models. It seems to me that any study of the accuracy of asteroid diameter determinations from missions like NEOWISE is entirely academic unless it taps into this publically available data. The failure of Myhrvold to undertake this in the course of his criticism of the work of others is (IMHO) bordering on the inexcusable.

    Turning now to a specific critique of Myhrvold’s paper (which I find extremely tedious reading…) Fig 23 (on page 72!) is (from my perusal) the first (only?) point at which he presents diameters derived by his approach. It lists just three asteroids, and interestingly we have a single reasonably-well-determined occultation diameter for each of them. Importantly, for these three asteroids we have a measured diameter two compare against the two ‘inferred’ diameters, with the obvious ability to assess which inferred diameter is best in each case, and whether there is any consistency across different asteroids. To summarise the various results:

    Asteroid # 208 306 757
    NEOWISE 45.0km 51.6km 36.7km
    Fig 23 146.5km 83.8km 6.6km
    Occultations 48 x 42km 61 x 44 km 39 x 34km

    Clearly the occultation results align extremely well with NEOWISE. In contrast there is major disagreement with the results of the author’s ‘bootstrap’ solution – with the strong implication that his bootstrap methodology is seriously flawed. IMHO the consequence of this on the paper as a whole doesn’t need to be stated…

    A final comment. We (in the occultation community) have not undertaken a formal comparison of our occultation results with those from NEOWISE/IRAS etc. However I believe it is a fair comment to assert that the NEOWISE/IRAS diameters are ‘about right’ – and certainly more than reliable enough for our current prediction purposes. Furthermore I cannot think of any instance where the diameter determined from a well-observed asteroidal occultation was greatly different from a NEOWISE.IRAS diameter – especially when a shape model is available such that we can assess the effects of the shape model on the expected diameters.

  32. Dwayne Day says:

    NASA Response to Recent Paper on NEOWISE Asteroid Size Results
    .
    Press Release From: NASA HQ
    Posted: Wednesday, May 25, 2016
    .
    A paper posted Sunday by Nathan Myhrvold to ArXiv.org and described in an article by reporter Ken Chang in the May 23 New York Times discusses interpretations of data on asteroids from NASA’s NEOWISE mission. The paper was posted before undergoing the essential scientific peer-review process to catch and remove significant errors.
    .
    Examination of the paper by members of the science community studying near-Earth objects has found several fundamental errors in Myhrvold’s approach and analysis—mistakes that an independent peer review process is designed to catch. The errors in the paper lead to results that are easily refuted, such as sizes for well-known asteroids that are significantly larger or smaller than their already-verified sizes. While critique and re-examination of published results are essential to the scientific process, it is important that any paper undergo peer review by an independent journal before it can be seriously considered. This completes a necessary step to ensure science results are independently validated, reproducible, and of value to the science community.
    .
    All of the published NEOWISE team papers providing their results have endured the peer-review process. NASA is confident that the processes and analyses performed by the NEOWISE team are valid and verified and stands by its data and scientific findings.
    .
    Data from the NEOWISE mission is available on a website for the public and scientific community to use. A guide to the NEOWISE data release, data access instructions and supporting documentation is available at http://wise2.ipac.caltech.edu/docs/release/neowise/. Access to the NEOWISE data products is available via the on-line and API services of the NASA/IPAC Infrared Science Archive. A list of peer-reviewed papers using the NEOWISE data is available at http://neowise.ipac.caltech.edu/publications.html

    // end //

  33. south says:

    Having read the comments here and some testimonies online, Myhrvold comes across as an individual more interested in seeing his name and ideas talked about in the media than contributing to science. Similar to some of the individuals and their over-reaching ideas in the string theory that receive some criticism on this blog.

    Also, all I see in the NASA statement is a dispassionate assertion that (1) the peer-review process is the proper hurdle to overcome, and that (2) experts have identified several errors in Myhrvold’s work. I don’t consider it an “attack” to state that someone’s calculations are wrong — in fact, this is what Myhrvold himself has done to NASA. But he did so publicly and these exchanges should carried out professionally through a peer-review process. Since Myhrvold eschewed this process and was first to blast to the media about NASA’s errors, I think it’s necessary and responsible for NASA to defend their reputation by publicly stating their disagreement.

  34. Dwayne Day says:

    From the Washington Post article:
    .
    “Some of the things I’ve heard from third parties are, ‘wow, we’ll see if it makes it through peer review,’ which makes me think they’ll ambush it in peer review,” Myhrvold said.
    .
    So he had his people contact the NY Times, Science, and apparently other websites pushing his claims before peer review.
    .
    https://www.washingtonpost.com/news/speaking-of-science/wp/2016/05/25/that-study-critiquing-nasas-bad-science-on-asteroids-is-pretty-bad-science/

  35. Anonyrat says:

    Myhrvold has evidently put in a lot of work, and it would be tragic if none of it is of value. What I don’t get is that, from all accounts, he has resources at his command that I can only dream of, and he certainly could have hired a couple of smart students or even professionals to vet his work before publishing it. It is not as though someone was going to scoop him. If this turns out to be a fiasco, anything important that he has to say in the future will get little attention.

  36. Jeffrey M says:

    I think this makes clear that arXiv is now treated in a way it really shouldn’t be. arXiv is for preprints. The odds that there are errors in things posted on arxiv are noticeably higher than for things published in reputable journals. I think a lot of people have forgotten that, they treat arXiv like a journal, which it’s not. If Myhrvold tried to get the authors to fix mistakes he thought they made, and they didn’t (either because they couldn’t, or because they didn’t feel they were mistakes), he could have gone to the journal editors where it was published. If even after that he got nowhere, he could put something on arXiv, and send it to a journal, but to make a big deal out of it is really out of bounds, no one has checked his work carefully, and someone did check the original work. The original referee might have screwed up, but Myhrvold hasn’t been refereed yet. This shouldn’t be happening in public. Myhrvold and NASA should just be waiting on the referee report on his paper, and if the referee says junk then no story, if the referee thinks Myhrvold is right, then it’s a story.

  37. JeanTate says:

    I would be interested to hear how mathematics arXiv submissions are different from astronomy/astrophysics ones.

    In the latter field, most authors clearly indicate whether the document has been submitted to a peer-reviewed journal, when/whether it has been accepted for publication, etc. How timely, or accurate, this info is is left up to the reader to decide. Some post the published version (modulo formatting etc), but I think that’s rare. There are, almost always, significant differences between the “submitted to”, and the “accepted for publication in” versions; sometimes these can be quite major. There are also quite a few arXiv documents that have, apparently, never been submitted to a journal, and some which apparently were submitted many years ago, but are still not published (in a peer-reviewed) journal.

    In almost any published paper, you’ll find “errors”. Mostly these are trivial – a misplaced comma, say – but occasionally you’ll find some howlers. A figure’s axes that are mislabeled, say, or an equation that omits a key term. I’ve asked astronomers about some of these, and most seem quite unconcerned; the mistakes are “minor” enough that a fellow professional, working in the same sub-sub-field can work out what was intended; indeed, I’ve had responses where mislabeled axes, say, we’re not even noticed!

    So, how bad are the – many – “errors” in the Myrhvold preprint? Of the ones already noted, and copied here, some would require just minor tweaks, and the main conclusions would be unaffected. Others, not so clear (the misunderstanding/misrepresentation of H, for example). The basic physics, and statistical analyses seem OK, but their application seems flawed, in several places. Also, one of the central themes – picked up by the popsci articles, i.e. the “unphysical” modeling – is misplaced (as Mainzer notes).

  38. Ignatz Ratzkywatzky says:

    “Anonyrat says:”

    Thank you for posting the only comments worth reading about this controversy.
    The ones with astronomical content.

    As for the rest

    “What dire Offence from astro’mous Causes springs,
    What mighty Contests rise from trivial Things,”

    with apologies to Pope.

  39. Jeffrey M says:

    JeanTate

    In math, from my experience, many postings on arXiv are done before something is submitted somewhere, but not long before. People are often trying to work out where to send something once they’re confident it’s right, so they’ll put in on arXiv while they’re trying to figure that out. People usually repost published versions, and corrected versions before publication. There are things on arXiv that have never been published, there are various reasons for that. I have a paper on arXiv which never appeared anywhere, I am sure it is correct, but it’s in a field not mine, and my co-author and I could never get our language to where the referees in that area would deal with the paper, so we just gave up. As you note, plenty of published papers have errors, referees miss things all the time. Usually not a big deal, but not always. There’s a famous example in my area, a paper of McKean where he claimed to prove that 1/4 is the bottom of the Laplace spectrum on surfaces. This was a big big deal, which unfortunately isn’t true.

  40. Anonyrat says:

    NASA pointed out that Myhrvold estimates the diameter of asteroid 295 to be 660 kilometers – this would be larger than the asteroid Vesta which is supposedly the second largest object in the asteroid belt. The diameter estimated by WISE, etc., is around 30 kilometers.

  41. Anonymous says:

    I would hardly characterize the NASA response as an attack. It simply states that peer review is appropriate for such claims. That’s pretty mild compared with what they could have said.

    I’m frankly startled, given your critiques of the meddling of billionaires in the physics and math worlds, and of the rockstar culture in physics, that you are embracing Myhrvold’s science-by-public relations efforts.

  42. Peter Woit says:

    I’ve added as an update a link to something from Nathan explaining his point of view on this controversy.

    While I have no expertise or ability to figure out who is right and who is wrong on the technical issues being raised on both sides here, I do have some expertise on the question of Nathan’s motivations, based on observations conducted over the long-term and over the shorter term of the last couple years. Some commenters seem to think this is a usual story of a wealthy person who wants to see his name in the papers, throw his money around and “meddle”in the world of science. They should note that devoting a huge amount of time and energy to rather tedious and complex issues of data analysis in a highly-obscure subject, and writing 60 page (successfully peer-reviewed) and new 110 page (out for peer-review) papers about this data analysis, are a funny way to go about this. Nathan has gotten a lot of media attention for various of his activities, and to whatever extent he might want more, he has lots of much easier ways to go about this. Among all the scientists I’ve met over the years, he’s one with an extremely unusual degree of enthusiasm and love for learning as much as possible about the technical details of a wide variety of scientific subjects, coupled with a brilliant mind and an insane amount of energy. Those criticizing his motivations seem to me to be missing the obvious: the guy loves real science with a passion and engages with it very seriously.

  43. Narad says:

    The errors in the paper lead to results that are easily refuted, such as sizes for well-known asteroids that are significantly larger or smaller than their already-verified sizes. While critique and re-examination of published results are essential to the scientific process, it is important that any paper undergo peer review by an independent journal before it can be seriously considered.

    This seems to be a very priggish approach coming from NASA. I know that the context is different (and I also worked on the make-the-final-product side of the AAS journals for over a decade a while ago and prevented more than a few errata), but virologist Vincent Racaniello recently was the subject of similar complaints.

    Is there some sort of crucial difference between postpublication peer review and the prepublication version? I see no particular reason to delay the “more eyeballs” part until after solemnization.

  44. No Math, Meekly Interacting says:

    “But the most egregious problems with the NEOWISE project are dead simple to explain; indeed, this might be a good project for a middle-school science class.”

    Ooooh-kay then! No mistaking the size of Dr. Myhrvold’s stones!

  45. Henry L. says:

    “No mistaking the size of Dr. Myhrvold’s stones!”

    Would that be based on using the thermal model?

  46. Anonymous says:

    While I still think Myhrvold’s pre-publication publicity efforts are inappropriate, and still loath him as a billionaire patent troll, I’ve now looked at the paper and withdraw my comment about meddling. His argument (about the copied data, I can’t evaluate the thermal modeling) is pretty damning. The NEOWISE people have some explaining to do.

    I do have some sympathy for their disinclination to engage with him, whatever the truth of the matter. Myhrvold hurts his case by his tone, which comes across as too similar to that of the many cranks that everyone at NASA or funded by NASA has to deal with on a daily basis.

  47. Anonyrat says:

    http://arxiv.org/pdf/1604.03444v1.pdf
    MODELING THE PERFORMANCE OF THE LSST IN SURVEYING THE NEAR-EARTH OBJECT POPULATION

    cites Myhrvold, if only to disagree with him. But they weren’t ignoring him.

    “The limiting magnitudes and solar elongation coverage presented here are at odds with Myhrvold (2016); his Figure 7 shows much fainter limiting magnitudes than presented here. While Myhrvold (2016) cites the possibility and capability of LSST observing at much smaller solar elongations, this is inconsistent with the published cadences provided by the LSST project and would potentially interfere with
    its other science goals. Additionally, Myhrvold (2016) incorrectly assumes that an object need only be detected once for it to be counted as discovered, cataloged, and tracked”

  48. Anonyrat says:

    In https://arxiv.org/ftp/arxiv/papers/1605/1605.06490.pdf, Myhrvold writes about the NEOWISE claims of high accuracy, and that all the caveats are not included.

    “This implies that the correspondence between NEATM model estimates and those from radar or other means has been calculated yielding a quantified numerical answer
    ≤±10%. In the passage above there is a caveat that the error estimate applies only with certain pre-conditions, but those caveats are frequently dropped from other references (as in the quote from (Masiero et al., 2011) in section 1 above.)”

    A look at Masiero et al. 2011 is NEATM applied to main belt asteroids, not near-earth asteroids. NEATM is “Near Earth Asteroid Thermal Model”. It was developed in 1998. This abstract (doi:10.1006/icar.1997.5865) essentially says that the thermal models for large main-belt asteroids were wrongly being applied to Near Earth Asteroids, that have very different properties, and that is the motivation for the NEATM model.

    But it seems after that, NEATM was found to be much more widely applicable, giving good results with main-belt asteroids. Therefore, when Myhrvold complains that Masiero et. al. write “Using a NEATM thermal model fitting routine we compute diameters for over 100,000 Main Belt asteroids from their IR thermal flux, with errors better than 10%.” (this is in the abstract) it is possible that NEATM model accuracies for main-belt asteroids can indeed by 10% while not being that accurate for actual NEOs. It is not clear to me.

    Most of the Mainz, Masiero, etc. papers that are cited have preprints on arxiv. You can look at them and see if Myhrvold’s criticisms are justified.

  49. Low Math, Meekly Interacting says:

    “Would that be based on using the thermal model?”

    Indeed. There’s a lot of heat here, clearly. We’ll see if there’s light. Tenured professors openly cutting rivals to the quick is hardly surprising, based on what I’ve observed in my career, and barely warrants a shrug. I normally do expect similar behavior from an “outsider” to be indicative of a crackpot at work. Obviously, Dr. Myhrvold doesn’t fit that description. But on the subject of stones, his status as a “billionaire patent troll” of considerable renown does mean that if he’s wrong, it’s going to be a very, very satisfying experience for some people to point that out, given the current sociopolitical climate. I doubt they will resist the opportunity to gloat in a most public fashion.

  50. JeanTate says:

    Thanks for posting the link to Nathan’s response, Peter.

    He makes a strong case for some apparent, strange inconsistencies in the published NEOWISE diameter estimates of some asteroids, and at first blush it seems pretty sound. However, despite Anonyrat pointing to many papers being freely available, it’s not clear how different the preprint versions are from the actual, published in peer-reviewed journals … some at least are behind paywalls (this points to yet another barrier to outsiders doing front-line, independent research in astronomy; unless you are rich, or have a backdoor, simply obtaining the necessary papers, not preprints, can be prohibitively expensive. Ironic really, as taxpayers, you pay for almost all the published research, yet you have to pay, yet again, to actually read what your dollars resulted in).

    I’m a little confused, however; my read of Nathan’s paper is that the NEOWISE diameter inconsistencies are a relatively minor part. The main part has to do with the models which are used to produce diameter estimates, and in this regard, it seems that Nathan is doing a dodge of his own.

    Finally, Nathan claims that the shortcomings in his equations (etc) are minor, easily fixed, and do not significantly change his main conclusions. That may well be so; however, until he makes the changes, publishes them, and we can all evaluate them for ourselves, his claim has no legs, IMHO.

Comments are closed.