Skip to main content

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

  • Matters Arising
  • Published:

Overconfidence in Bayesian analyses of galaxy rotation curves

Nature Astronomy volume 4, pages 132–133 (2020)Cite this article

Subjects

Matters Arising to this article was published on 27 January 2020

The Original Article was published on 13 November 2018

The Original Article was published on 13 November 2018

The Original Article was published on 18 June 2018

arising from D. C. Rodrigues et al. Nature Astronomy https://doi.org/10.1038/s41550-018-0498-9 (2018)

Over the past year or so, a vigorous debate has played out in the pages of Nature Astronomy (and other journals) between the opponents1,2 and exponents3,4 of universal galactic radial acceleration relationships (RARs) deriving from an alternative gravitational paradigm. The modified theory of Newtonian dynamics (MOND) offers a potentially cold dark matter-free explanation for the flattening of certain galaxy rotation curves at increasing radial distance, observed in contrast to the decline that would be expected under standard Newtonian gravity for a mass budget that is assumed to be dominated by baryonic matter. The exchange in question was sparked by the nominally Bayesian analysis of Rodrigues et al.1, which claims rejection at greater than 10σ confidence level of a MOND-inspired RAR model. In this Matters Arising, we argue that the above conclusion rests on a flawed approach to statistical inference: namely, the application of an ad hoc (and inherently non-Bayesian) test in a misspecified setting. We detail our concerns below and offer recommendations for best-practice procedures for Bayesian model development.

This is a preview of subscription content, access via your institution

Access options

Buy this article

  • Purchase on Springer Link
  • Instant access to full article PDF
Buy now

Prices may be subject to local taxes which are calculated during checkout

References

  1. Rodrigues, D. C., Marra, V., Del Popolo, A. & Davari, Z. Absence of a fundamental acceleration scale in galaxies. Nat. Astron. 2, 668–672 (2018).

    Article  ADS  Google Scholar 

  2. Rodrigues, D. C., Marra, V., Del Popolo, A. & Davari, Z. Reply to ‘Presence of a fundamental acceleration scale in galaxies’ and ‘A common Milgromian acceleration scale in nature’ Nat. Astron. 2, 927–929 (2018).

    Article  ADS  Google Scholar 

  3. McGaugh, S. S., Li, P., Lelli, F. & Schombert, J. M. Presence of a fundamental acceleration scale in galaxies. Nat. Astron. 2, 924 (2018).

    Article  ADS  Google Scholar 

  4. Kroupa, P. et al. A common Milgromian acceleration scale in nature. Nat. Astron. 2, 925–926 (2018).

    Article  ADS  Google Scholar 

  5. Lelli, F., McGaugh, S. S. & Schombert, J. M. SPARC: Mass models FOR 175 disk galaxies with SPITZ. Astron. J. 152, 157 (2016).

    Article  ADS  Google Scholar 

  6. de Blok, W. J. G. et al. High-resolution rotation curves and galaxy mass models from things. Astron. J. 136, 2648–2719 (2008).

    Article  ADS  Google Scholar 

  7. Andrae, R., Schulze-Hartung, T. & Melchior, P. Dos and don’ts of reduced chi-squared. Preprint at https://arxiv.org/abs/1012.3754 (2010).

  8. Clopper, C. J. & Pearson, E. S. The use of confidence or fiducial limits illustrated in the case of the binomial. Biometrika 26, 404–413 (1934).

    Article  Google Scholar 

  9. Jin, S., Thulin, M. & Larsson, R. Approximate Bayesianity of frequentist confidence intervals for a binomial proportion. Am. Stat. 71, 106–111 (2017).

    Article  MathSciNet  Google Scholar 

  10. Chang, Z. & Zhou, Y. Is there a fundamental acceleration scale in galaxies? Mon. Not. R. Astron. Soc. 486, 1658–1666 (2019).

    Article  ADS  Google Scholar 

  11. Marchand, E. & Strawderman, W. E. On Bayesian credible sets, restricted parameter spaces and frequentist coverage. Electron. J. Stat. 7, 1419–1431 (2013).

    Article  MathSciNet  Google Scholar 

  12. Sweeting, T. J. On predictive probability matching priors. Inst. Mat. Stat. Collections 3, 46–59 (2008).

    Article  MathSciNet  Google Scholar 

  13. Bayarri, M. J. & Berger, J. O. The interplay of Bayesian and frequentist analysis. Stat. Sci. 19, 58–80 (2004).

    Article  MathSciNet  Google Scholar 

  14. Berger, J. O., Boukai, B. & Wang, Y. Unified frequentist and Bayesian testing of a precise hypothesis. Stat. Sci. 12, 133–160 (1997).

    Article  MathSciNet  Google Scholar 

  15. Trotta, R. Bayes in the sky: Bayesian inference and model selection in cosmology. Contemp. Phys. 49, 71–104 (2008).

    Article  ADS  Google Scholar 

  16. Kennedy, L. A. et al. Not every credible interval is credible: evaluating robustness in the presence of contamination in Bayesian data analysis. Behav. Res. Methods 49, 2219–2234 (2017).

    Article  Google Scholar 

  17. Yang, Z. & Zhu, T. Bayesian selection of misspecified models is overconfident and may cause spurious posterior probabilities for phylogenetic trees. Proc. Natl Acad. Sci. USA 115, 1854–1859 (2018).

    Article  MathSciNet  Google Scholar 

  18. Lv, J. & Liu, J. S. Model selection principles in misspecified models. J. R. Stat. Soc. Ser. B 76, 141–167 (2014).

    Article  MathSciNet  Google Scholar 

  19. Gelman, A. & Shalizi, C. R. Philosophy and the practice of Bayesian statistics. Br. J. Math. Stat. Psychol. 66, 8–38 (2013).

    Article  MathSciNet  Google Scholar 

  20. Gabry, J. et al. Visualization in Bayesian workflow. J. R. Stat. Soc. Ser. A 182, 389–402 (2019).

    Article  MathSciNet  Google Scholar 

  21. Gelman, A., Simpson, D. & Betancourt, M. The prior can often only be understood in the context of the likelihood. Entropy 19, 555 (2017).

    Article  ADS  Google Scholar 

  22. Lyddon, S. P., Holmes, C. C. & Walker, S. G. General Bayesian updating and the loss-likelihood bootstrap. Biometrika 106, 465–478 (2019).

    Article  MathSciNet  Google Scholar 

  23. Bernton, E. et al. On parameter estimation with the Wasserstein distance. Inf. Inference 8, 657–676 (2019).

    Article  MathSciNet  Google Scholar 

  24. Gelman, A. & Hill, J. Data Analysis Using Regression and Multilevel/Hierarchical Models (Cambridge Univ. Press, 2006).

  25. Li, P., Lelli, F., McGaugh, S. & Schombert, J. Fitting the radial acceleration relation to individual SPARC galaxies. Astron. Astrophys. 615, A3 (2018).

    Article  ADS  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Malaria Atlas Project, Oxford Big Data Institute, Oxford, UK

    Ewan Cameron

  2. Dipartimento di Fisica, Università di Torino, Torino, Italy

    Garry W. Angus

  3. Istituto Nazionale di Fisica Nucleare (INFN), Sezione di Torino, Torino, Italy

    Garry W. Angus

  4. Max Planck Institute for Extraterrestrial Physics, Garching, Germany

    J. Michael Burgess

Authors
  1. Ewan Cameron
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Garry W. Angus
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. Michael Burgess
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

E.C. recognized the core statistical problem, conceived the project, and wrote an initial draft. G.W.A. and J.M.B. contributed expertise towards understanding key domain-specific issues and refining the arguments presented. All authors participated in revising the final manuscript.

Corresponding author

Correspondence to Ewan Cameron.

Ethics declarations

Competing interests

The authors declare no competing interests.

Additional information

Peer review information Nature Astronomy thanks Johannes Buchner, Roberto Trotta and the other, anonymous, reviewer(s) for their contribution to the peer review of this work.

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cameron, E., Angus, G.W. & Burgess, J.M. Overconfidence in Bayesian analyses of galaxy rotation curves. Nat Astron 4, 132–133 (2020). https://doi.org/10.1038/s41550-019-0998-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1038/s41550-019-0998-2

This article is cited by

Search

Advanced search

Quick links

Nature Briefing AI and Robotics

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

Get the most important science stories of the day, free in your inbox. Sign up for Nature Briefing: AI and Robotics