Nature | Comment

Good data are not enough

A vibrant scientific culture encourages many interpretations of evidence, argues Avi Loeb.

Article tools

Reidar Hahn/Fermilab

The Dark Energy Survey at the Cerro Tololo Inter-American Observatory in Chile is mapping the large-scale structure of the Universe traced by galaxies.

This summer, I visited the Mayan city of Chichén Itzá in the Yucatán Peninsula, Mexico. It has an ancient observatory where priest-astronomers made detailed astronomical observations around AD 600–1200. The ruins — stepped pyramids, temples, columned arcades and other stone structures — reveal that astronomy was at the heart of this sophisticated society.

The Mayans accurately tracked changes in the positions and relative brightness of the Sun, Moon, planets and stars. They documented their astronomical data in folding books called codices, with many more quantitative details than other civilizations at the time. The priest-astronomers used observations and advanced mathematical calculations to predict eclipses, and devised a 365-day solar calendar that was off by just one month every 100 years.

LISTEN

Avi Loeb describes his view that astronomers should entertain multiple interpretations of their data

You may need a more recent browser or to install the latest version of the Adobe Flash Plugin.

So why, I wondered, didn't the Mayans go further and infer aspects of our modern understanding of astronomy? They determined the orbital periods of Venus, Mars and Mercury around the Sun, but Earth was at the centre of their Universe.

I came to appreciate how limiting prevailing world views can be. Just as geological and other evidence for the great age of Earth was rejected before the nineteenth century as being hard to square with biblical history, the Mayans used their fine data to support a mythological culture of astrology. They correlated the periodic motions of celestial objects with human history and, rather than seeking a physical explanation for their astronomical data, they used it to initiate wars or rituals such as human sacrifice.

Have we learned our lesson, or is today's science similarly trapped by cultural and societal forces? Most research funding is allocated assuming that the highest-quality data will inevitably deliver useful scientific interpretation and theoretical concepts, which can be tested and refined by future data. The astronomy division of the US National Science Foundation, for example, devotes most of its funds to major facilities and large surveys, which are performed by big teams to collect better data within mainstream paradigms. Fields from particle physics to genomics do the same.

The consequences of a closed scientific culture are wasted resources and misguided 'progress' — witness the dead end that was Soviet evolutionary biology. To truly move forward, free thought must be encouraged outside the mainstream. Multiple interpretations of existing data and alternative motivations for collecting new data must be supported.

Blinkered view

Mayan cosmologists had high social status. They got generous support because of their promises to forecast the future. Cosmologists today collect vast amounts of exquisite data in surveys of large parts of the sky, costing billions of dollars.

Surveys of the large-scale structure of the Universe traced by galaxies include the US Baryon Oscillation Spectroscopic Survey and the international Dark Energy Survey, as well as forthcoming facilities such as the US Dark Energy Spectroscopic Instrument, the European Space Agency's (ESA's) Euclid mission, NASA's Wide Field Infrared Survey Telescope and the Large Synoptic Survey Telescope in Chile. Others mapping the primordial seeds of these structures as traced by cosmic microwave background radiation include ESA's Planck satellite, the US South Pole Telescope and international collaborations such as the Atacama Cosmology Telescope and the Simons Array.

myLoupe/UIG/Getty

The Mayan pyramid at Chichén Itzá in Mexico.

Such projects have a narrow aim — pinning down the parameters of one theoretical model. The model comprises an expanding Universe composed of dark matter, dark energy and normal matter (from which stars, planets and people are made), with initial conditions dictated by an early phase of rapid expansion called cosmic inflation. The data are reduced to a few numbers. Surprises in the rest are tossed away.

I noticed this bias recently while assessing a PhD thesis. The student was asked to test whether a data set from a large cosmological survey was in line with the standard cosmological model. But when a discrepancy was found, the student's goal shifted to explaining why the data set was incomplete. In such a culture, the current model can never be ruled out, even though everyone knows that its major constituents (dark matter, dark energy and inflation) are not understood at a fundamental level.

Instead, observers should present results in a theory-neutral way. Observations should not converge on one model but aim to find anomalies that carry clues about the nature of dark matter, dark energy or initial conditions of the Universe. Further observations should be motivated by testing unconventional interpretations of those anomalies (such as exotic forms of dark matter or modified theories of gravity). Vast data sets may contain evidence for unusual behaviour that was unanticipated when the projects were conceived. If all results are expected and planned for, babies may be thrown out with the bathwater.

Blinded by beauty

How each culture views the Universe is guided by its beliefs in, for example, mathematical beauty or the structure of reality. If these ideas are deeply rooted, people tend to interpret all data as supportive of them — adding parameters or performing mathematical gymnastics to force the fit. Recall how the belief that the Sun moves around Earth led to the mathematically beautiful (and incorrect) theory of epicycles advocated by the ancient Greek philosopher Ptolemy.

Similarly, modern cosmology is augmented by unsubstantiated, mathematically sophisticated ideas — of the multiverse, anthropic reasoning and string theory. The multiverse idea postulates the existence of numerous other regions of space-time, to which we have no access and in which the cosmological parameters have different values.

The anthropic argument is then often applied. It holds that our own region has the parameters it does (including those of dark energy and dark matter) because other, more likely values would not have allowed life to develop near a star like the Sun in a galaxy such as the Milky Way1, 2, 3. An overlooked problem with this argument is that, according to one analysis4, life is 1,000 times more likely to exist 10 trillion years from now around stars that weigh one-tenth the mass of the Sun. This means that terrestrial life might be premature and not the most likely form of life, even in our own Universe4.

The anthropic argument, which as yet has no empirical support, suppresses much-needed efforts to understand dark energy through an alternative theory that unifies quantum mechanics and gravity. The fact that we have not yet converged on such a theory is indicated by paradoxes in other areas of physics. For example, information contained in, say, an encyclopaedia is lost if it is swallowed by a black hole that ultimately evaporates into heat known as Hawking radiation. This contradicts a basic premise of quantum mechanics that information is preserved, and is known as the 'information paradox'5. In addition, currently viable models of cosmic inflation require fine-tuning of the conditions of the Universe before and during inflation6.

Cultivating other approaches avoids stalling progress by investing only in chasing what might turn out to be 'epicycles'. After all, the standard model of cosmology is merely a precise account of our ignorance: we do not understand the nature of inflation, dark matter or dark energy. The model has difficulties accounting for the luminous gas and stars that we can see in galaxies, while leaving invisible what we can easily calculate (dark matter and dark energy). This state of affairs is clearly unsatisfactory.

The tendency to establish large projects and firm up mainstream ideas is a signature of a mature scientific discipline. In such a culture, the low-hanging fruit has already been picked by small, versatile teams that are long gone. Critics argue that when funds are limited, the focus of research should be on coordinated approaches that are likely to produce results in a predictable way. This advocacy fails to appreciate that our mainstream paradigm might be heading in the wrong direction. The opportunity for making mistakes is much greater than for real breakthroughs, so as any venture capitalist knows, investing part of the portfolio in risky endeavours is necessary to gain substantial profits.

Alternative paths

The only way to work out whether we are on the wrong path is to encourage competing interpretations of the known data.

I have been arguing for many years that funding agencies should promote the analysis of data for serendipitous purposes beyond major programmes and the mainstream dogma. The need for a change in course is even more pressing now. Empirical constraints on expected forms of dark matter (such as weakly interacting massive particles or supersymmetric partners to known particles) are getting tighter, and the hope of identifying testable consequences of string theory is receding. At a minimum, when funding is tight, a research frontier should maintain at least two ways of interpreting data so that new experiments will aim to select the correct one. This should apply to alternatives to inflation when dealing with new cosmological data, and to alternatives to cold dark matter when discrepancies are observed in the properties of dark-matter-dominated galaxies.

New funding streams should be established in other fields. The LIGO discovery of black-hole mergers should encourage a 'template-free' search for new sources of gravitational waves that were never imagined. The Kepler satellite's discovery that roughly one-quarter of all stars in the Galaxy host a habitable Earth-mass planet7 should lead to a renewed effort in the search for extraterrestrial life, including new methods for finding intelligent civilizations8. Indeed, a habitable planet was recently discovered9 around the nearest star to our Sun, Proxima Centauri, which could be probed with a future spacecraft (http://breakthroughinitiatives.org/Concept/3).

A healthy dialogue between different points of view should be fostered through multidisciplinary conferences that discuss conceptual issues, not just experimental results and phenomenology. A diversity of views fosters healthy progress and prevents stagnation. In September, I had the privilege of founding an interdisciplinary centre, the Black Hole Initiative at Harvard University in Cambridge, Massachusetts, which brings together astronomers, physicists, mathematicians and philosophers. Our experience is that a mix of scholars with different vocabularies and comfort zones can cultivate innovation and research outside the box. Already the centre has prompted exciting insights on the reality of naked singularities in space-time, the prospects for imaging black-hole silhouettes and the information paradox.

Such simple, off-the-shelf remedies could help us to avoid the scientific fate of the otherwise admirable Mayan civilization.

Journal name:
Nature
Volume:
539,
Pages:
23–25
Date published:
()
DOI:
doi:10.1038/539023a

References

  1. Weinberg, S. Phys. Rev. Lett. 59, 26072610 (1987).

  2. Vilenkin, A. Phys. Rev. Lett. 74, 846849 (1995).

  3. Bousso, R., Hall, L. J. & Nomura, Y. Phys. Rev. D 80, 063510 (2009).

  4. Loeb, A., Batista, R. A. & Sloan, D. J. Cosmol. Astropart. Phys. 8, 040 (2016).

  5. Hawking, S. W., Perry, M. J. & Strominger, A. Phys. Rev. Lett. 116, 231301 (2016).

  6. Ijjas, A., Steinhardt, P. J. & Loeb, A. Preprint at https://arxiv.org/abs/1402.6980 (2014).

  7. Dressing, C. D. & Charbonneau, D. Astrophys. J. 807, 45 (2015).

  8. Guillochon, J. & Loeb, A. Astrophys. J. Lett. 811, L20 (2015).

  9. Anglada-Escude, G. et al. Nature 536, 437440 (2016).

Author information

Affiliations

  1. Avi Loeb is professor of science and chair of the astronomy department at Harvard University in Cambridge, Massachusetts, USA.

Corresponding author

Correspondence to:

Author details

For the best commenting experience, please login or register as a user and agree to our Community Guidelines. You will be re-directed back to this page where you will see comments updating in real-time and have the ability to recommend comments to other users.

Comments

Commenting is currently unavailable.

sign up to Nature briefing

What matters in science — and why — free in your inbox every weekday.

Sign up

Listen

new-pod-red

Nature Podcast

Our award-winning show features highlights from the week's edition of Nature, interviews with the people behind the science, and in-depth commentary and analysis from journalists around the world.